[jira] [Created] (SOLR-14090) Schema API - Delete copy field not working when the field contains and underscore
Frank Iversen created SOLR-14090: Summary: Schema API - Delete copy field not working when the field contains and underscore Key: SOLR-14090 URL: https://issues.apache.org/jira/browse/SOLR-14090 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: v2 API Affects Versions: 8.3.1 Reporter: Frank Iversen Attachments: image-2019-12-16-08-54-46-719.png Delete statements for the Schema API are not working when the source field contain an underscore (havent tested this for "dest") . If the underscore is removed deletion is working as intended. Another user of Solr seems to have had the same problem and was asked to create a Jira ticket. I can however not find this ticket anywhere. [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] I upgraded to the newest version and the problem still persists. It should be simple to reproduce. Just include and underscore in a copy field in your schema file and try to delete the copy field. !image-2019-12-16-08-54-46-719.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14090) Schema API - Delete copy field not working when the field contains an underscore
[ https://issues.apache.org/jira/browse/SOLR-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Iversen updated SOLR-14090: - Summary: Schema API - Delete copy field not working when the field contains an underscore (was: Schema API - Delete copy field not working when the field contains and underscore) > Schema API - Delete copy field not working when the field contains an > underscore > > > Key: SOLR-14090 > URL: https://issues.apache.org/jira/browse/SOLR-14090 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: v2 API >Affects Versions: 8.3.1 >Reporter: Frank Iversen >Priority: Major > Attachments: image-2019-12-16-08-54-46-719.png > > > Copy field delete statements for the Schema API are not working when the > source field contain an underscore (havent tested this for "dest") . If the > underscore is removed deletion is working as intended. Another user of Solr > seems to have had the same problem and was asked to create a Jira ticket. I > can however not find this ticket anywhere. > [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html > |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] > I upgraded to the newest version and the problem still persists. > It should be simple to reproduce. Just include and underscore in a copy field > in your schema file and try to delete the copy field. > !image-2019-12-16-08-54-46-719.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14090) Schema API - Delete copy field not working when the field contains and underscore
[ https://issues.apache.org/jira/browse/SOLR-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Iversen updated SOLR-14090: - Description: Copy field delete statements for the Schema API are not working when the source field contain an underscore (havent tested this for "dest") . If the underscore is removed deletion is working as intended. Another user of Solr seems to have had the same problem and was asked to create a Jira ticket. I can however not find this ticket anywhere. [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] I upgraded to the newest version and the problem still persists. It should be simple to reproduce. Just include and underscore in a copy field in your schema file and try to delete the copy field. !image-2019-12-16-08-54-46-719.png! was: Delete statements for the Schema API are not working when the source field contain an underscore (havent tested this for "dest") . If the underscore is removed deletion is working as intended. Another user of Solr seems to have had the same problem and was asked to create a Jira ticket. I can however not find this ticket anywhere. [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] I upgraded to the newest version and the problem still persists. It should be simple to reproduce. Just include and underscore in a copy field in your schema file and try to delete the copy field. !image-2019-12-16-08-54-46-719.png! > Schema API - Delete copy field not working when the field contains and > underscore > - > > Key: SOLR-14090 > URL: https://issues.apache.org/jira/browse/SOLR-14090 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: v2 API >Affects Versions: 8.3.1 >Reporter: Frank Iversen >Priority: Major > Attachments: image-2019-12-16-08-54-46-719.png > > > Copy field delete statements for the Schema API are not working when the > source field contain an underscore (havent tested this for "dest") . If the > underscore is removed deletion is working as intended. Another user of Solr > seems to have had the same problem and was asked to create a Jira ticket. I > can however not find this ticket anywhere. > [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html > |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] > I upgraded to the newest version and the problem still persists. > It should be simple to reproduce. Just include and underscore in a copy field > in your schema file and try to delete the copy field. > !image-2019-12-16-08-54-46-719.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14066) Deprecate DIH
[ https://issues.apache.org/jira/browse/SOLR-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997082#comment-16997082 ] Adrien Grand commented on SOLR-14066: - Hello, should I wait for this one to build 8.4 or should we push it to 8.5? > Deprecate DIH > - > > Key: SOLR-14066 > URL: https://issues.apache.org/jira/browse/SOLR-14066 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - DataImportHandler >Reporter: Ishan Chattopadhyaya >Priority: Major > Attachments: image-2019-12-14-19-58-39-314.png > > Time Spent: 20m > Remaining Estimate: 0h > > DataImportHandler has outlived its utility. DIH doesn't need to remain inside > Solr anymore. Let us deprecate DIH in 8.4 (and remove it from the Solr distro > in 9x or 10x). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14091) Remove deprecated soLingerTime when configuring Jetty connector
Matthias Krueger created SOLR-14091:
---
Summary: Remove deprecated soLingerTime when configuring Jetty
connector
Key: SOLR-14091
URL: https://issues.apache.org/jira/browse/SOLR-14091
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 8.3.1, 7.7.2
Reporter: Matthias Krueger
Jetty has deprecated ServerConnector#soLingerTime since 9.4.12 as "linger
should not be set for NIO sockets" (see
[https://github.com/eclipse/jetty.project/issues/2468]). We should remove it
from configuration, too. Currently, there is a bunch of
{code:java}
[pool-1-thread-1] org.eclipse.jetty.server.AbstractConnector
Ignoring deprecated socket close linger time {code}
warnings spamming our logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14051) Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5 remove them in 9.0
[
https://issues.apache.org/jira/browse/SOLR-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997099#comment-16997099
]
Lucene/Solr QA commented on SOLR-14051:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 7 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 3m 16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 3m 11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 3m 11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate ref guide {color} |
{color:green} 3m 11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 87m
37s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 97m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-14051 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12988906/SOLR-14051.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns validaterefguide |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 2db4831 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/628/testReport/ |
| modules | C: solr/core solr/solr-ref-guide U: solr |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/628/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5 remove
> them in 9.0
> -
>
> Key: SOLR-14051
> URL: https://issues.apache.org/jira/browse/SOLR-14051
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-14051-branch_8x.patch, SOLR-14051.patch
>
>
> Please migrate to SOLR-8998 and SOLR-9510
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkr opened a new pull request #1087: SOLR-14091: Removing deprecated configuration of Jetty's soLingerTime…
mkr opened a new pull request #1087: SOLR-14091: Removing deprecated configuration of Jetty's soLingerTime… URL: https://github.com/apache/lucene-solr/pull/1087 … option # Description Please provide a short description of the changes you're making with this pull request. # Solution Please provide a short description of the approach taken to implement your solution. # Tests Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [ ] I have developed this patch against the `master` branch. - [ ] I have run `ant precommit` and the appropriate test suite. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14066) Deprecate DIH
[ https://issues.apache.org/jira/browse/SOLR-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997101#comment-16997101 ] Jan Høydahl commented on SOLR-14066: Please do not hold up the 8.4 build for this. > Deprecate DIH > - > > Key: SOLR-14066 > URL: https://issues.apache.org/jira/browse/SOLR-14066 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - DataImportHandler >Reporter: Ishan Chattopadhyaya >Priority: Major > Attachments: image-2019-12-14-19-58-39-314.png > > Time Spent: 20m > Remaining Estimate: 0h > > DataImportHandler has outlived its utility. DIH doesn't need to remain inside > Solr anymore. Let us deprecate DIH in 8.4 (and remove it from the Solr distro > in 9x or 10x). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14091) Remove deprecated soLingerTime when configuring Jetty connector
[
https://issues.apache.org/jira/browse/SOLR-14091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997103#comment-16997103
]
Matthias Krueger commented on SOLR-14091:
-
PR: [https://github.com/apache/lucene-solr/pull/1087/files]
> Remove deprecated soLingerTime when configuring Jetty connector
> ---
>
> Key: SOLR-14091
> URL: https://issues.apache.org/jira/browse/SOLR-14091
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 7.7.2, 8.3.1
>Reporter: Matthias Krueger
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Jetty has deprecated ServerConnector#soLingerTime since 9.4.12 as "linger
> should not be set for NIO sockets" (see
> [https://github.com/eclipse/jetty.project/issues/2468]). We should remove it
> from configuration, too. Currently, there is a bunch of
> {code:java}
> [pool-1-thread-1] org.eclipse.jetty.server.AbstractConnector
> Ignoring deprecated socket close linger time {code}
> warnings spamming our logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14092) Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5
Mikhail Khludnev created SOLR-14092: --- Summary: Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5 Key: SOLR-14092 URL: https://issues.apache.org/jira/browse/SOLR-14092 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Reporter: Mikhail Khludnev Assignee: Mikhail Khludnev Please migrate to SOLR-8998 and SOLR-9510 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14051) Remove BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 9.0
[ https://issues.apache.org/jira/browse/SOLR-14051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated SOLR-14051: Summary: Remove BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 9.0 (was: Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5 remove them in 9.0) > Remove BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 9.0 > --- > > Key: SOLR-14051 > URL: https://issues.apache.org/jira/browse/SOLR-14051 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-14051-branch_8x.patch, SOLR-14051.patch > > > Please migrate to SOLR-8998 and SOLR-9510 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14092) Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5
[ https://issues.apache.org/jira/browse/SOLR-14092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated SOLR-14092: Attachment: SOLR-14051-branch_8x.patch > Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5 > -- > > Key: SOLR-14092 > URL: https://issues.apache.org/jira/browse/SOLR-14092 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-14051-branch_8x.patch > > > Please migrate to SOLR-8998 and SOLR-9510 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14092) Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5
[ https://issues.apache.org/jira/browse/SOLR-14092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997128#comment-16997128 ] Mikhail Khludnev commented on SOLR-14092: - Trying to convince Yetus to pickup branch_8x patch. > Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5 > -- > > Key: SOLR-14092 > URL: https://issues.apache.org/jira/browse/SOLR-14092 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-14051-branch_8x.patch > > > Please migrate to SOLR-8998 and SOLR-9510 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14092) Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5
[ https://issues.apache.org/jira/browse/SOLR-14092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated SOLR-14092: Status: Patch Available (was: Open) > Deprecate BlockJoinFacetComponent&BlockJoinDocSetFacetComponent in 8.5 > -- > > Key: SOLR-14092 > URL: https://issues.apache.org/jira/browse/SOLR-14092 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-14051-branch_8x.patch > > > Please migrate to SOLR-8998 and SOLR-9510 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14090) Schema API - Delete copy field not working when the field contains an underscore
[ https://issues.apache.org/jira/browse/SOLR-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Iversen updated SOLR-14090: - Description: Copy field delete statements for the Schema API are not working when the source field contains an underscore (havent tested this for "dest") . If the underscore is removed deletion is working as intended. Another user of Solr seems to have had the same problem and was asked to create a Jira ticket. I can however not find this ticket anywhere hence this ticket. [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] I upgraded to the newest version and the problem still persists. It should be simple to reproduce. Just include and underscore in a copy field in your schema file and try to delete the copy field. !image-2019-12-16-08-54-46-719.png! was: Copy field delete statements for the Schema API are not working when the source field contain an underscore (havent tested this for "dest") . If the underscore is removed deletion is working as intended. Another user of Solr seems to have had the same problem and was asked to create a Jira ticket. I can however not find this ticket anywhere. [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] I upgraded to the newest version and the problem still persists. It should be simple to reproduce. Just include and underscore in a copy field in your schema file and try to delete the copy field. !image-2019-12-16-08-54-46-719.png! > Schema API - Delete copy field not working when the field contains an > underscore > > > Key: SOLR-14090 > URL: https://issues.apache.org/jira/browse/SOLR-14090 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: v2 API >Affects Versions: 8.3.1 >Reporter: Frank Iversen >Priority: Major > Attachments: image-2019-12-16-08-54-46-719.png > > > Copy field delete statements for the Schema API are not working when the > source field contains an underscore (havent tested this for "dest") . If the > underscore is removed deletion is working as intended. Another user of Solr > seems to have had the same problem and was asked to create a Jira ticket. I > can however not find this ticket anywhere hence this ticket. > [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html > |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] > I upgraded to the newest version and the problem still persists. > It should be simple to reproduce. Just include and underscore in a copy field > in your schema file and try to delete the copy field. > !image-2019-12-16-08-54-46-719.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14090) Schema API - Delete copy field not working when the field contains an underscore
[ https://issues.apache.org/jira/browse/SOLR-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Iversen updated SOLR-14090: - Description: Copy field delete statements for the Schema API are not working when the source field contains an underscore (havent tested this for "dest") . If the underscore is removed deletion is working as intended. Another user of Solr seems to have had the same problem and was asked to create a Jira ticket. I can however not find this ticket anywhere hence this ticket. [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] I upgraded to the newest version and the problem still persists. It should be simple to reproduce. Just include and underscore in a copy field statement in your schema file and try to delete the copy field using the api, as seen below: !image-2019-12-16-08-54-46-719.png! was: Copy field delete statements for the Schema API are not working when the source field contains an underscore (havent tested this for "dest") . If the underscore is removed deletion is working as intended. Another user of Solr seems to have had the same problem and was asked to create a Jira ticket. I can however not find this ticket anywhere hence this ticket. [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] I upgraded to the newest version and the problem still persists. It should be simple to reproduce. Just include and underscore in a copy field in your schema file and try to delete the copy field. !image-2019-12-16-08-54-46-719.png! > Schema API - Delete copy field not working when the field contains an > underscore > > > Key: SOLR-14090 > URL: https://issues.apache.org/jira/browse/SOLR-14090 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: v2 API >Affects Versions: 8.3.1 >Reporter: Frank Iversen >Priority: Major > Attachments: image-2019-12-16-08-54-46-719.png > > > Copy field delete statements for the Schema API are not working when the > source field contains an underscore (havent tested this for "dest") . If the > underscore is removed deletion is working as intended. Another user of Solr > seems to have had the same problem and was asked to create a Jira ticket. I > can however not find this ticket anywhere hence this ticket. > [https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html > |https://lucene.472066.n3.nabble.com/Error-deleting-copy-field-td4393097.html] > I upgraded to the newest version and the problem still persists. > It should be simple to reproduce. Just include and underscore in a copy field > statement in your schema file and try to delete the copy field using the api, > as seen below: > !image-2019-12-16-08-54-46-719.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14033) Fix Hadoop tests with security manager
[ https://issues.apache.org/jira/browse/SOLR-14033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997138#comment-16997138 ] ASF subversion and git services commented on SOLR-14033: Commit 2f051a4bfea12ea2c66d16680968697ca4573884 in lucene-solr's branch refs/heads/gradle-master from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2f051a4 ] SOLR-14086: Tika ClassNotFound error due to commons-compress in solr-core dependency Introduced in SOLR-14033 by including commons-compress as a compile time dependency in Solr core instead of as as test only dependency. Signed-off-by: Kevin Risden > Fix Hadoop tests with security manager > -- > > Key: SOLR-14033 > URL: https://issues.apache.org/jira/browse/SOLR-14033 > Project: Solr > Issue Type: Test > Security Level: Public(Default Security Level. Issues are Public) > Components: Hadoop Integration, hdfs, Tests >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Fix For: 8.4 > > Attachments: SOLR-14033.patch > > Time Spent: 6.5h > Remaining Estimate: 0h > > SOLR-14000 and SOLR-14002 have been cleaning this up. Seeing some HDFS suite > failures due to missing nulling out static variables and Hadoop being stupid > with the security manager. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13662) Package manager CLI
[ https://issues.apache.org/jira/browse/SOLR-13662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997135#comment-16997135 ] ASF subversion and git services commented on SOLR-13662: Commit 640ff96522eeff891d10cbc49bc8eadfcb0826f1 in lucene-solr's branch refs/heads/gradle-master from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=640ff96 ] SOLR-13662: Fixes to package manager * Better logging and error reporting * Fixing deploy command to handle previously undeployed packages * Test now uses @LogLevel annotation * Deploy command had a hard coded collection name by mistake, fix it > Package manager CLI > --- > > Key: SOLR-13662 > URL: https://issues.apache.org/jira/browse/SOLR-13662 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Assignee: Ishan Chattopadhyaya >Priority: Major > Fix For: 8.4 > > Attachments: plugin-cli.png > > Time Spent: 19h > Remaining Estimate: 0h > > Design details and usage details are here: > https://docs.google.com/document/d/15b3m3i3NFDKbhkhX_BN0MgvPGZaBj34TKNF2-UNC3U8/edit?ts=5d86a8ad# -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14086) Tika ClassNotFound error due to commons-compress in solr-core dependency
[
https://issues.apache.org/jira/browse/SOLR-14086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997137#comment-16997137
]
ASF subversion and git services commented on SOLR-14086:
Commit 2f051a4bfea12ea2c66d16680968697ca4573884 in lucene-solr's branch
refs/heads/gradle-master from Kevin Risden
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2f051a4 ]
SOLR-14086: Tika ClassNotFound error due to commons-compress in solr-core
dependency
Introduced in SOLR-14033 by including
commons-compress as a compile time
dependency in Solr core instead of as
as test only dependency.
Signed-off-by: Kevin Risden
> Tika ClassNotFound error due to commons-compress in solr-core dependency
>
>
> Key: SOLR-14086
> URL: https://issues.apache.org/jira/browse/SOLR-14086
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - Solr Cell (Tika extraction)
>Reporter: Tim Allison
>Assignee: Kevin Risden
>Priority: Major
> Fix For: 8.4
>
> Attachments: SOLR-14086.patch, test-documents.7z,
> tika-integration-example-9.0.0-SNAPSHOT.tgz
>
>
> Opening on behalf of [~tallison] since he found this in SOLR-14054.
> From lucene-solr repo directory:
> {code:java}
> ant clean clean-maven-build clean-jars jar-checksums
> cd solr
> ant package
> cp package/solr-9.0.0-SNAPSHOT.tgz ~/Downloads
> {code}
> New terminal
> {code:java}
> cd ~/Downloads
> wget
> https://issues.apache.org/jira/secure/attachment/12988870/tika-integration-example-9.0.0-SNAPSHOT.tgz
> tar zxvf tika-integration-example-9.0.0-SNAPSHOT.tgz
> wget
> https://issues.apache.org/jira/secure/attachment/12988868/test-documents.7z
> tar zxvf solr-9.0.0-SNAPSHOT.tgz
> cd solr-9.0.0-SNAPSHOT
> ./bin/solr start
> ./bin/solr create -c tika-integration-example -d
> ~/Downloads/tika-integration-example/conf
> curl
> 'http://localhost:8983/solr/tika-integration-example/update/extract?literal.id=doc1&commit=true'
> -F "myfile=@$HOME/Downloads/test-documents.7z"
> {code}
> You should see an error like:
> {code:java}
>
>
>
> Error 500 java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.commons.compress.archivers.sevenz.Coders
>
> HTTP ERROR 500 java.lang.NoClassDefFoundError: Could not initialize
> class org.apache.commons.compress.archivers.sevenz.Coders
>
> URI:/solr/tika-integration-example/update/extract
> STATUS:500
> MESSAGE:java.lang.NoClassDefFoundError: Could not initialize
> class org.apache.commons.compress.archivers.sevenz.Coders
> SERVLET:default
> CAUSED BY:java.lang.NoClassDefFoundError: Could not
> initialize class org.apache.commons.compress.archivers.sevenz.Coders
>
> Caused by:java.lang.NoClassDefFoundError: Could not initialize
> class org.apache.commons.compress.archivers.sevenz.Coders
> at
> org.apache.commons.compress.archivers.sevenz.SevenZFile.readEncodedHeader(SevenZFile.java:437)
> at
> org.apache.commons.compress.archivers.sevenz.SevenZFile.readHeaders(SevenZFile.java:355)
> at
> org.apache.commons.compress.archivers.sevenz.SevenZFile.(SevenZFile.java:241)
> at
> org.apache.commons.compress.archivers.sevenz.SevenZFile.(SevenZFile.java:108)
> at
> org.apache.commons.compress.archivers.sevenz.SevenZFile.(SevenZFile.java:262)
> at
> org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:257)
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at
> org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
> at
> org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:228)
> at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:208)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2582)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:799)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:578)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:419)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jett
[jira] [Commented] (SOLR-14069) Ref Guide: Overhaul resources, libs, plugins, configsets
[ https://issues.apache.org/jira/browse/SOLR-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997134#comment-16997134 ] ASF subversion and git services commented on SOLR-14069: Commit 7c048c5070988f35d38d5f592fad5d295ddb380a in lucene-solr's branch refs/heads/gradle-master from David Smiley [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7c048c5 ] SOLR-14069: Ref guide: overhaul: resources, libs, plugins, config-sets (#1077) * split "resource-and-plugin-loading.adoc" into "resource-loading.adoc" and "libs.adoc" then overhauled both. * enhanced "config-sets.adoc", moving some content in from elsewhere; bit of an overhaul. * solr-plugins.adoc is now top-level; overhauled content * Move resource-loading.adoc up a level in the TOC to underneath "The Well-Configured Solr Instance. * Separate out the leading sentence. > Ref Guide: Overhaul resources, libs, plugins, configsets > > > Key: SOLR-14069 > URL: https://issues.apache.org/jira/browse/SOLR-14069 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Fix For: 8.4 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > This is a bit of an overhaul of some topics. > See my final comment on SOLR-13662. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997139#comment-16997139 ] ASF subversion and git services commented on SOLR-14087: Commit d64c5c20b6de78f71ffb79f0249dd8a943d7c839 in lucene-solr's branch refs/heads/gradle-master from Noble Paul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d64c5c2 ] SOLR-14087: disable package store API if -Denable.packages not set to true > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14071) Untrusted configsets shouldn't be allowed to use directive
[ https://issues.apache.org/jira/browse/SOLR-14071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997136#comment-16997136 ] ASF subversion and git services commented on SOLR-14071: Commit be0b963a2280c3b79fab11ca9370a06e7c7beb39 in lucene-solr's branch refs/heads/gradle-master from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=be0b963 ] SOLR-14071: Updating upgrade notice > Untrusted configsets shouldn't be allowed to use directive > > > Key: SOLR-14071 > URL: https://issues.apache.org/jira/browse/SOLR-14071 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Fix For: 8.4 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > Allowing untrusted configsets, i.e. those have been uploaded using the > configset upload API without authx enabled, to use the directive can > open up possibilities for malicious users to include insecure contribs > libraries. > Whoever wants to use their own libraries can add them to the classpath of > Solr (i.e. place them wherever solr-core-*jar resides). For them, the > directive won't be necessary anyway. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997141#comment-16997141 ] ASF subversion and git services commented on SOLR-14087: Commit 479db61df82226e26ce68ce67e28fc51bce0b06a in lucene-solr's branch refs/heads/gradle-master from Noble Paul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=479db61 ] SOLR-14087: changed the filestore location to .filestore instead of $filestore > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14072) Deprecate plugin loading using runtimelib
[ https://issues.apache.org/jira/browse/SOLR-14072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997142#comment-16997142 ] ASF subversion and git services commented on SOLR-14072: Commit 2db48314f97a6017fe216f77c34cf5b261985cf4 in lucene-solr's branch refs/heads/gradle-master from David Smiley [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2db4831 ] SOLR-14072: Deprecate Blob API and runtimeLib (#1086) > Deprecate plugin loading using runtimelib > - > > Key: SOLR-14072 > URL: https://issues.apache.org/jira/browse/SOLR-14072 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Assignee: David Smiley >Priority: Major > Fix For: 8.4 > > Time Spent: 50m > Remaining Estimate: 0h > > Package management system supercedes the old way of loading jars from blob > store using runtimelib. Let us deprecate the old way. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9077) Gradle build
[
https://issues.apache.org/jira/browse/LUCENE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Weiss updated LUCENE-9077:
Description:
This task focuses on providing gradle-based build equivalent for Lucene and
Solr (on master branch). See notes below on why this respin is needed.
The code lives on *gradle-master* branch. It is kept with sync with *master*.
Try running the following to see an overview of helper guides concerning
typical workflow, testing and ant-migration helpers:
gradlew :help
A list of items that needs to be added or requires work. If you'd like to work
on any of these, please add your name to the list. Once you have a patch/ pull
request let me (dweiss) know - I'll try to coordinate the merges.
* (/) Apply forbiddenAPIs
* (/) Generate hardware-aware gradle defaults for parallelism (count of
workers and test JVMs).
* (/) Fail the build if --tests filter is applied and no tests execute during
the entire build (this allows for an empty set of filtered tests at single
project level).
* (/) Port other settings and randomizations from common-build.xml
* (/) Configure security policy/ sandboxing for tests.
* (/) test's console output on -Ptests.verbose=true
* (/) add a :helpDeps explanation to how the dependency system works (palantir
plugin, lockfile) and how to retrieve structured information about current
dependencies of a given module (in a tree-like output).
* (/) jar checksums, jar checksum computation and validation. This should be
done without intermediate folders (directly on dependency sets).
* verify min. JVM version and exact gradle version on build startup to minimize
odd build side-effects
* identify and list precommit tasks so that they can be ported one by one.
(Mark's branch has some of this stuff already implemented)
* Repro-line for failed tests/ runs.
Hard-to-implement stuff already investigated:
* (!) *Printing console output of failed tests.* There doesn't seem to be any
way to do this in a reasonably efficient way. There are onOutput listeners but
they're slow to operate and solr tests emit *tons* of output so it's an
overkill. The only solution I think would be feasible is to parse test report
XMLs after the tests are complete and collect the output of failed tests from
there. The downside is that XMLs contain separated sysout/syserr.
* (!) *Tests working with security-debug logs or other JVM-early log output*.
From what I can see in the code Gradle's test runner works by redirecting
Java's stdout/ syserr so this just won't work. Perhaps we can spin the
ant-based test runner for such corner-cases.
Of lesser importance:
* add rendering of javadocs (gradlew javadoc) and attaching them to maven
publications.
* Add test 'beasting' (rerunning the same suite multiple times). I'm afraid
it'll be difficult to run it sensibly because gradle doesn't offer cwd
separation for the forked test runners.
* if you diff solr packaged distribution against ant-created distribution
there are minor differences in library versions and some JARs are excluded/
moved around. I didn't try to force these as everything seems to work (tests,
etc.) – perhaps these differences should be fixed in the ant build instead.
* identify and port any other "check" utilities that may be called from ant.
(Mark's branch has some of this stuff already implemented)
* [EOE] identify and port various "regenerate" tasks from ant builds (javacc,
precompiled automata, etc.)
* fill in POM details in gradle/defaults-maven.gradle so that they reflect the
previous content better (dependencies aside).
* Add any IDE integration layers that should be added (I use IntelliJ and it
imports the project out of the box, without the need for any special tuning).
* *Clean up dependencies, especially for Solr*: any \{ transitive = false }
should just explicitly exclude whatever they don't need (and their dependencies
currently declared explicitly should be folded). Figure out which scope to
import a dependency to.
* Add Solr packaging for docs/* (see TODO in packaging/build.gradle; currently
XSLT...)
* I didn't bother adding Solr dist/test-framework to packaging (who'd use it
from a binary distribution?
*{color:#ff}Note:{color}* this builds on the work done by Mark Miller and
Cao Mạnh Đạt but also applies lessons learned from those two efforts:
* *Do not try to do too many things at once*. If we deviate too far from
master, the branch will be hard to merge.
* *Do everything in baby-steps* and add small, independent build fragments
replacing the old ant infrastructure.
* *Try to engage people to run, test and contribute early*. It can't be a
one-man effort. The more people understand and can contribute to the build, the
more healthy it will be.
was:
tThis task focuses on providing gradle-based build equivalent for Lucene and
Solr (on master branch). See notes below on why
[GitHub] [lucene-solr] sigram commented on a change in pull request #1071: SOLR-14003: Refactor code to avoid reading state from Replica/Slic
sigram commented on a change in pull request #1071: SOLR-14003: Refactor code
to avoid reading state from Replica/Slic
URL: https://github.com/apache/lucene-solr/pull/1071#discussion_r358170501
##
File path: solr/solrj/src/java/org/apache/solr/common/cloud/Replica.java
##
@@ -167,11 +167,15 @@ public String getNodeName() {
return nodeName;
}
- /** Returns the {@link State} of this replica. */
+ /** Returns the {@link State} of this replica. use {@link
org.apache.solr.client.solrj.cloud.ShardStateProvider#getState(Replica)}
instead*/
+ @Deprecated
public State getState() {
Review comment:
Then I think this refactoring falls short of its potential - because instead
of having a single uniform API we now end up with two APIs that claim to do the
same, and the old one still prevents us from moving the state reporting out of
the actual Replica object - ie. no matter how clever we are with the
`ShardStateProvider` we still end up having to cache the potentially stale
state inside `Replica`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14091) Remove deprecated soLingerTime when configuring Jetty connector
[
https://issues.apache.org/jira/browse/SOLR-14091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Krueger updated SOLR-14091:
Description:
Jetty has deprecated ServerConnector#soLingerTime since 9.4.12 as "linger
should not be set for NIO sockets" (see
[https://github.com/eclipse/jetty.project/issues/2468]). We should remove it
from configuration, too. Currently, there is a bunch of
{code:java}
[pool-1-thread-1] org.eclipse.jetty.server.AbstractConnector Ignoring
deprecated socket close linger time {code}
warnings spamming our logs.
was:
Jetty has deprecated ServerConnector#soLingerTime since 9.4.12 as "linger
should not be set for NIO sockets" (see
[https://github.com/eclipse/jetty.project/issues/2468]). We should remove it
from configuration, too. Currently, there is a bunch of
{code:java}
[pool-1-thread-1] org.eclipse.jetty.server.AbstractConnector
Ignoring deprecated socket close linger time {code}
warnings spamming our logs.
> Remove deprecated soLingerTime when configuring Jetty connector
> ---
>
> Key: SOLR-14091
> URL: https://issues.apache.org/jira/browse/SOLR-14091
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 7.7.2, 8.3.1
>Reporter: Matthias Krueger
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Jetty has deprecated ServerConnector#soLingerTime since 9.4.12 as "linger
> should not be set for NIO sockets" (see
> [https://github.com/eclipse/jetty.project/issues/2468]). We should remove it
> from configuration, too. Currently, there is a bunch of
> {code:java}
> [pool-1-thread-1] org.eclipse.jetty.server.AbstractConnector Ignoring
> deprecated socket close linger time {code}
> warnings spamming our logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] sigram commented on a change in pull request #1071: SOLR-14003: Refactor code to avoid reading state from Replica/Slic
sigram commented on a change in pull request #1071: SOLR-14003: Refactor code
to avoid reading state from Replica/Slic
URL: https://github.com/apache/lucene-solr/pull/1071#discussion_r358176763
##
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/cloud/DirectShardState.java
##
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.client.solrj.cloud;
+
+import java.util.function.Predicate;
+
+import org.apache.solr.common.cloud.Replica;
+import org.apache.solr.common.cloud.Slice;
+
+/**Reads the state from state.json
+ *
+ */
+public class DirectShardState implements ShardStateProvider {
Review comment:
I think we have to decide where we keep (and maintain) the actual state. The
API that you proposed provides two ways of getting this info, and two potential
locations to get it (which may get out of sync, or have different staleness),
which in my opinion is one too many :)
If the replica state resides in `Replica` and this is our source of truth
then having a `ShardStateProvider` seems pointless. What is the source of truth
then - the state that is cached in `Replica` or the state that we obtain from
`ShardStateProvider` implementation, which is guaranteed to the the most
up-to-date?
My understanding was that the purpose of introducing `ShardStateProvider`
was to provide The single API point that is the only source of truth about the
current shard/replica state (similar to other *StateProvider-s). In this model
`Replica` becomes a simple static descriptor and it's NOT responsible for
caching its state - that's the role of `ShardStateProvider`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9087) Should the BKD tree use a fixed maxPointsInLeafNode?
[ https://issues.apache.org/jira/browse/LUCENE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997217#comment-16997217 ] Ignacio Vera commented on LUCENE-9087: -- We are currently using one bitset when writing the tree and it is only used to count the number of documents we are storing in the tree which it can be different to the number of points if you have multi-value documents. I think we can detect that situation in the constructor (totalPointCount != maxDoc) and therefore we could easily spare that bitset for trees that only contain single-value documents. There used to be bitsets of size maxDoc for the handling of deleted documents before using the radix partitioning, this is not needed any more. > Should the BKD tree use a fixed maxPointsInLeafNode? > - > > Key: LUCENE-9087 > URL: https://issues.apache.org/jira/browse/LUCENE-9087 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Ignacio Vera >Priority: Major > > Currently the BKD tree uses a fixed maxPointsInLeafNode provided in the > constructor. For the current default codec the value is set to 1200. This is > a good compromise between memory usage and performance of the BKD tree. > Lowering this value can increase search performance but it has a penalty in > memory usage. Now that the BKD tree can be load off-heap, this can be less of > a concern. Note that lowering too much that value can hurt performance as > well as the tree becomes too deep and benefits are gone. > For data types that use the tree as an effective R-tree (ranges and shapes > datatypes) the benefits are larger as it can minimise the overlap between > leaf nodes. > Finally, creating too many leaf nodes can be dangerous at write time as > memory usage depends on the number of leaf nodes created. The writer creates > a long array of length = numberOfLeafNodes. > What I am wondering here is if we can improve this situation in order to > create the most efficient tree? My current ideas are: > > * We can adapt the points per leaf depending on that number so we create a > tree with the best depth and best points per leaf. Note that for the for 1D > case we have an upper estimation of the number of points that the tree will > be indexing. > * Add a mechanism so field types can easily define their best points per > leaf. In the case, field types like ranges or shapes can define its own value > to minimise overlap. > * Maybe the default is just too high now that we can load the tree off-heap. > > Any thoughts? > > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997227#comment-16997227 ] Ishan Chattopadhyaya commented on SOLR-14087: - bq. [~noble.paul] Why was the location of the filestore changed from ".filestore" to "filestore"? Sorry but I disagree with this. I don't recall us ever making a choice like this before – making a directory invisible to a typical directory listing. There was some discussion, https://the-asf.slack.com/archives/CNMTSU970/p1576386856009700 Problem with "filestore" was that in standalone mode, no one can create a core called "filestore". Problem with "$filestore" is that scripting will become a nightmare with need for escaping etc., esp. if someone wants to cold bootstrap some packages later on. +1 to ".filestore" because "." prefix is convention for system files and in our case too, our ".system" collection. No user can create a core named ".filestore", but it is understandable. > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14093) Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
Robert Muir created SOLR-14093:
--
Summary: Ban ObjectInputStream and ObjectOutputStream in
forbidden-apis
Key: SOLR-14093
URL: https://issues.apache.org/jira/browse/SOLR-14093
Project: Solr
Issue Type: Task
Security Level: Public (Default Security Level. Issues are Public)
Components: Build
Reporter: Robert Muir
Assignee: Robert Muir
suggested build failure message:
{quote}
[forbidden-apis] Forbidden class/interface use: java.io.ObjectInputStream [Java
deserialization is unsafe when the data is untrusted. The java developer is
powerless: no checks or casts help, exploitation can happen in places such as
clinit or finalize!]
{quote}
I will whitelist existing places doing this for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Moved] (LUCENE-9094) Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
[
https://issues.apache.org/jira/browse/LUCENE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir moved SOLR-14093 to LUCENE-9094:
Component/s: (was: Build)
general/build
Key: LUCENE-9094 (was: SOLR-14093)
Lucene Fields: New
Project: Lucene - Core (was: Solr)
Security: (was: Public)
> Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
> --
>
> Key: LUCENE-9094
> URL: https://issues.apache.org/jira/browse/LUCENE-9094
> Project: Lucene - Core
> Issue Type: Task
> Components: general/build
>Reporter: Robert Muir
>Assignee: Robert Muir
>Priority: Major
>
> suggested build failure message:
> {quote}
> [forbidden-apis] Forbidden class/interface use: java.io.ObjectInputStream
> [Java deserialization is unsafe when the data is untrusted. The java
> developer is powerless: no checks or casts help, exploitation can happen in
> places such as clinit or finalize!]
> {quote}
> I will whitelist existing places doing this for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997244#comment-16997244 ] David Smiley commented on SOLR-14087: - I was a part of that discussion. The problem of namespace collission with cores is not introduced with this issue; it already exists with "lib" and "configsets". This just adds another. I think the solution to _that_ problem is putting the cores in a directory like "cores" underneath solr home. That can wait till 9.0. Furthermore, configsets and the core root dir are both configurable so users can put them where they please. filestore ought to be similar but (a) it can wait till 8.5; really this stuff is experimental, and (b) it's an opt-in system at present. Please change the name back. > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14094) TestSolrCachePerf is flaky
Adrien Grand created SOLR-14094:
---
Summary: TestSolrCachePerf is flaky
Key: SOLR-14094
URL: https://issues.apache.org/jira/browse/SOLR-14094
Project: Solr
Issue Type: Test
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Adrien Grand
I hit the below failure while building the RC.
{noformat}
[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
[junit4] 2> 921086 INFO
(SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
o.a.s.SolrTestCaseJ4 Created dataDir:
/home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
[junit4] 2> 921086 WARN
(SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
[junit4] 2> 921086 INFO
(SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 921087 INFO
(SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 921087 INFO
(SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 921088 INFO
(TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
[junit4] 2> 927857 INFO
(TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
-Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
-Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
[junit4]>at
__randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
[junit4]>at
org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
[junit4]>at
org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
[junit4]>at
org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
[junit4]>at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at:
/home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
locale=ar-QA, timezone=AGT
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
IndexSizeTriggerMixedBoundsTest, NumberUtilsTest, TestZkChroot,
HdfsDirectoryTest, RollingRestartTest, TestSolrCLIRunExample, ClusterStateTest,
DocValuesMultiTest, TestLeaderElectionWithEmptyReplica, TestCustomSort,
TestSchemaManager, TestInPlaceUpdatesDistrib, TestReloadAndDeleteDocs,
HighlighterConfigTest, MBeansHandlerTest,
PeerSyncWithIndexFingerprintCachingTest, SubstringBytesRefFilterTest,
TestChildDocTransformer, IndexSizeTriggerTest, HealthCheckHandlerTest,
SuggesterFSTTest, TestLuceneIndexBackCompat, TestDelegationWithHadoopAuth,
TestSolrCoreProperties, DistanceFunctionTest,
SignatureUpdateProcessorFactoryTest, CdcrBootstrapTest, DebugComponentTest,
TriggerEventQueueTest, TestLegacyBM25SimilarityFactory, SolrMetricReporterTest,
TestCorePropertiesReload, TestPseudoReturnFields, DistributedVersionInfoTest,
CollectionPropsTest, SpatialFilterTest, CurrencyRangeFacetCloudTest,
SolrCoreCheckLockOnStartupTest, EchoParamsTest,
TestImpersonationWithHadoopAuth, CollectionsAPIDistributedZkTest,
TestPerFieldSimilarityWithDefaultOverride, QueryParsingTest,
TestDefaultStatsCache, DocExpirationUpdateProcessorFactoryTest,
TestComplexPhraseQParserPlugi
[jira] [Commented] (SOLR-14094) TestSolrCachePerf is flaky
[
https://issues.apache.org/jira/browse/SOLR-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997247#comment-16997247
]
ASF subversion and git services commented on SOLR-14094:
Commit ed97a36fb4d1c91cb4d17f6c988e9b5988190cbe in lucene-solr's branch
refs/heads/branch_8x from Adrien Grand
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ed97a36 ]
SOLR-14094: Bad-apple TestSolrCachePerf.
> TestSolrCachePerf is flaky
> --
>
> Key: SOLR-14094
> URL: https://issues.apache.org/jira/browse/SOLR-14094
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Adrien Grand
>Priority: Major
>
> I hit the below failure while building the RC.
> {noformat}
>[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Created dataDir:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
>[junit4] 2> 921086 WARN
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=false
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN,
> clientAuth=NaN)
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>[junit4] 2> 921088 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
>[junit4] 2> 927857 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
> -Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
> -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
> compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
>[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
> docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
> locale=ar-QA, timezone=AGT
>[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
> 1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
>[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
> AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
> TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
> IndexSizeTriggerMixedBoundsTest, NumberUtilsTest, TestZkChroot,
> HdfsDirectoryTest, RollingRestartTest, TestSolrCLIRunExample,
> ClusterStateTest, DocValuesMultiTest, TestLeaderElectionWithEmptyReplica,
> TestCustomSort, TestSchemaManager, TestInPlaceUpdatesDistrib,
> TestReloadAndDeleteDocs, HighlighterConfigTest, MBeansHandlerTest,
> PeerSyncWithIndexFingerprintCachingTest, SubstringBytesRefFilterTest,
> TestChildDocTransformer, IndexSizeTriggerTest, HealthCheckHandlerTest,
> SuggesterFSTTest, TestLuceneIndexBackCompat, TestDelegationWithHadoopAuth,
> Tes
[jira] [Commented] (SOLR-14094) TestSolrCachePerf is flaky
[
https://issues.apache.org/jira/browse/SOLR-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997248#comment-16997248
]
ASF subversion and git services commented on SOLR-14094:
Commit b660bcd0a2b1308cc94993239b86a71d6c0213b2 in lucene-solr's branch
refs/heads/master from Adrien Grand
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b660bcd ]
SOLR-14094: Bad-apple TestSolrCachePerf.
> TestSolrCachePerf is flaky
> --
>
> Key: SOLR-14094
> URL: https://issues.apache.org/jira/browse/SOLR-14094
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Adrien Grand
>Priority: Major
>
> I hit the below failure while building the RC.
> {noformat}
>[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Created dataDir:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
>[junit4] 2> 921086 WARN
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=false
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN,
> clientAuth=NaN)
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>[junit4] 2> 921088 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
>[junit4] 2> 927857 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
> -Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
> -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
> compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
>[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
> docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
> locale=ar-QA, timezone=AGT
>[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
> 1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
>[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
> AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
> TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
> IndexSizeTriggerMixedBoundsTest, NumberUtilsTest, TestZkChroot,
> HdfsDirectoryTest, RollingRestartTest, TestSolrCLIRunExample,
> ClusterStateTest, DocValuesMultiTest, TestLeaderElectionWithEmptyReplica,
> TestCustomSort, TestSchemaManager, TestInPlaceUpdatesDistrib,
> TestReloadAndDeleteDocs, HighlighterConfigTest, MBeansHandlerTest,
> PeerSyncWithIndexFingerprintCachingTest, SubstringBytesRefFilterTest,
> TestChildDocTransformer, IndexSizeTriggerTest, HealthCheckHandlerTest,
> SuggesterFSTTest, TestLuceneIndexBackCompat, TestDelegationWithHadoopAuth,
> TestSo
[jira] [Commented] (SOLR-14094) TestSolrCachePerf is flaky
[
https://issues.apache.org/jira/browse/SOLR-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997246#comment-16997246
]
ASF subversion and git services commented on SOLR-14094:
Commit 87d9793e659d34b3996c367e962fd376bd549193 in lucene-solr's branch
refs/heads/branch_8_4 from Adrien Grand
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=87d9793 ]
SOLR-14094: Bad-apple TestSolrCachePerf.
> TestSolrCachePerf is flaky
> --
>
> Key: SOLR-14094
> URL: https://issues.apache.org/jira/browse/SOLR-14094
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Adrien Grand
>Priority: Major
>
> I hit the below failure while building the RC.
> {noformat}
>[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Created dataDir:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
>[junit4] 2> 921086 WARN
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=false
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN,
> clientAuth=NaN)
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>[junit4] 2> 921088 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
>[junit4] 2> 927857 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
> -Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
> -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
> compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
>[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
> docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
> locale=ar-QA, timezone=AGT
>[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
> 1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
>[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
> AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
> TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
> IndexSizeTriggerMixedBoundsTest, NumberUtilsTest, TestZkChroot,
> HdfsDirectoryTest, RollingRestartTest, TestSolrCLIRunExample,
> ClusterStateTest, DocValuesMultiTest, TestLeaderElectionWithEmptyReplica,
> TestCustomSort, TestSchemaManager, TestInPlaceUpdatesDistrib,
> TestReloadAndDeleteDocs, HighlighterConfigTest, MBeansHandlerTest,
> PeerSyncWithIndexFingerprintCachingTest, SubstringBytesRefFilterTest,
> TestChildDocTransformer, IndexSizeTriggerTest, HealthCheckHandlerTest,
> SuggesterFSTTest, TestLuceneIndexBackCompat, TestDelegationWithHadoopAuth,
> Te
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997249#comment-16997249 ] ASF subversion and git services commented on SOLR-14087: Commit 7dfea5fe33fe0aea7eda001c046a3537402532f2 in lucene-solr's branch refs/heads/master from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7dfea5f ] SOLR-14087: Changing the filestore dir name back to filestore from .filestore > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997250#comment-16997250 ] ASF subversion and git services commented on SOLR-14087: Commit 8aa0103b148cda4be30f064a7fc4db52ce636d87 in lucene-solr's branch refs/heads/branch_8x from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8aa0103 ] SOLR-14087: Changing the filestore dir name back to filestore from .filestore > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997252#comment-16997252 ] ASF subversion and git services commented on SOLR-14087: Commit a4338769811ad645f488e3aeb15765e32910f3e2 in lucene-solr's branch refs/heads/branch_8_4 from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a433876 ] SOLR-14087: Changing the filestore dir name back to filestore from .filestore > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya resolved SOLR-14087. - Resolution: Fixed > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997255#comment-16997255 ] Ishan Chattopadhyaya commented on SOLR-14087: - bq. I think the solution to that problem is putting the cores in a directory like "cores" underneath solr home. +1. Can you please open a blocker issue for 9.0 to tackle this? > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14087) disable package store API if -Denable.packages not set to true
[ https://issues.apache.org/jira/browse/SOLR-14087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997257#comment-16997257 ] Ishan Chattopadhyaya commented on SOLR-14087: - bq. Please change the name back. Done. I suppose Adrien will be angry, as he should be. > disable package store API if -Denable.packages not set to true > -- > > Key: SOLR-14087 > URL: https://issues.apache.org/jira/browse/SOLR-14087 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Priority: Blocker > Fix For: 8.4 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Package store is only used by package manager. For better security, we can > disable it by default -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9087) Should the BKD tree use a fixed maxPointsInLeafNode?
[
https://issues.apache.org/jira/browse/LUCENE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997264#comment-16997264
]
Adrien Grand commented on LUCENE-9087:
--
I meant the bitsets that we use to represent deleted documents, as exposed by
{{LeafReader#getLiveDocs}}. It'd be nice if we could save some memory at build
time, but the point I was trying to make is that we shouldn't feel bad about
having transient memory usage that is the same order of magnitude as the
permanent memory usage that we require in case of deleted documents.
> Should the BKD tree use a fixed maxPointsInLeafNode?
> -
>
> Key: LUCENE-9087
> URL: https://issues.apache.org/jira/browse/LUCENE-9087
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ignacio Vera
>Priority: Major
>
> Currently the BKD tree uses a fixed maxPointsInLeafNode provided in the
> constructor. For the current default codec the value is set to 1200. This is
> a good compromise between memory usage and performance of the BKD tree.
> Lowering this value can increase search performance but it has a penalty in
> memory usage. Now that the BKD tree can be load off-heap, this can be less of
> a concern. Note that lowering too much that value can hurt performance as
> well as the tree becomes too deep and benefits are gone.
> For data types that use the tree as an effective R-tree (ranges and shapes
> datatypes) the benefits are larger as it can minimise the overlap between
> leaf nodes.
> Finally, creating too many leaf nodes can be dangerous at write time as
> memory usage depends on the number of leaf nodes created. The writer creates
> a long array of length = numberOfLeafNodes.
> What I am wondering here is if we can improve this situation in order to
> create the most efficient tree? My current ideas are:
>
> * We can adapt the points per leaf depending on that number so we create a
> tree with the best depth and best points per leaf. Note that for the for 1D
> case we have an upper estimation of the number of points that the tree will
> be indexing.
> * Add a mechanism so field types can easily define their best points per
> leaf. In the case, field types like ranges or shapes can define its own value
> to minimise overlap.
> * Maybe the default is just too high now that we can load the tree off-heap.
>
> Any thoughts?
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14066) Deprecate DIH
[
https://issues.apache.org/jira/browse/SOLR-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997269#comment-16997269
]
David Eric Pugh commented on SOLR-14066:
Thank you for taking a slightly slower pace with DIH. I want to point out
that while DIH may seem like an anti-pattern for those of you running huge Solr
setups with dedicated teams, there is huge user base out there that is just
looking for great search for their traditional database backed application, and
DIH has worked for many years just fine for them. This is based on my working
with *Many* different search teams.
Let's take advantage of the plugin framework to have a Solr that supports the
massive deploys and the small ones. Move DIH to a place where those who want
to use it can, and maybe open the door to a larger set of people supporting
these plugins.
Also, I want to point out that if we feel like DIH represents some sort of
anti-pattern, then what about the Streaming capability? I suspect that in the
future, I'll be writing a lot of
{code:java}
daemon(jdbc(update()))
{code}
style streaming expressions to move data around.
> Deprecate DIH
> -
>
> Key: SOLR-14066
> URL: https://issues.apache.org/jira/browse/SOLR-14066
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - DataImportHandler
>Reporter: Ishan Chattopadhyaya
>Priority: Major
> Attachments: image-2019-12-14-19-58-39-314.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> DataImportHandler has outlived its utility. DIH doesn't need to remain inside
> Solr anymore. Let us deprecate DIH in 8.4 (and remove it from the Solr distro
> in 9x or 10x).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14066) Deprecate DIH
[
https://issues.apache.org/jira/browse/SOLR-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997269#comment-16997269
]
David Eric Pugh edited comment on SOLR-14066 at 12/16/19 1:06 PM:
--
Thank you for taking a slightly slower pace with DIH. I want to point out
that while DIH may seem like an anti-pattern for those of you running huge Solr
setups with dedicated teams, there is huge user base out there that is just
looking for great search for their traditional database backed application, and
DIH has worked for many years just fine for them. This is based on my working
with *Many* different search teams.
Let's take advantage of the plugin framework to have a Solr that supports the
massive deploys and the small ones. Move DIH to a place where those who want
to use it can, and maybe open the door to a larger set of people supporting
these plugins.
Also, I want to point out that if we feel like DIH represents some sort of
anti-pattern, then what about the Streaming capability? I suspect that in the
future, I'll be writing a lot of
{code:java}
daemon(update(jdbc()))
{code}
style streaming expressions to move data around.
was (Author: epugh):
Thank you for taking a slightly slower pace with DIH. I want to point out
that while DIH may seem like an anti-pattern for those of you running huge Solr
setups with dedicated teams, there is huge user base out there that is just
looking for great search for their traditional database backed application, and
DIH has worked for many years just fine for them. This is based on my working
with *Many* different search teams.
Let's take advantage of the plugin framework to have a Solr that supports the
massive deploys and the small ones. Move DIH to a place where those who want
to use it can, and maybe open the door to a larger set of people supporting
these plugins.
Also, I want to point out that if we feel like DIH represents some sort of
anti-pattern, then what about the Streaming capability? I suspect that in the
future, I'll be writing a lot of
{code:java}
daemon(jdbc(update()))
{code}
style streaming expressions to move data around.
> Deprecate DIH
> -
>
> Key: SOLR-14066
> URL: https://issues.apache.org/jira/browse/SOLR-14066
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - DataImportHandler
>Reporter: Ishan Chattopadhyaya
>Priority: Major
> Attachments: image-2019-12-14-19-58-39-314.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> DataImportHandler has outlived its utility. DIH doesn't need to remain inside
> Solr anymore. Let us deprecate DIH in 8.4 (and remove it from the Solr distro
> in 9x or 10x).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9094) Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
[
https://issues.apache.org/jira/browse/LUCENE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir updated LUCENE-9094:
Attachment: LUCENE-9094.patch
> Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
> --
>
> Key: LUCENE-9094
> URL: https://issues.apache.org/jira/browse/LUCENE-9094
> Project: Lucene - Core
> Issue Type: Task
> Components: general/build
>Reporter: Robert Muir
>Assignee: Robert Muir
>Priority: Major
> Attachments: LUCENE-9094.patch
>
>
> suggested build failure message:
> {quote}
> [forbidden-apis] Forbidden class/interface use: java.io.ObjectInputStream
> [Java deserialization is unsafe when the data is untrusted. The java
> developer is powerless: no checks or casts help, exploitation can happen in
> places such as clinit or finalize!]
> {quote}
> I will whitelist existing places doing this for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14094) TestSolrCachePerf is flaky
[
https://issues.apache.org/jira/browse/SOLR-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997274#comment-16997274
]
Ishan Chattopadhyaya commented on SOLR-14094:
-
Could be due to SOLR-13898. FYI, [~ab].
> TestSolrCachePerf is flaky
> --
>
> Key: SOLR-14094
> URL: https://issues.apache.org/jira/browse/SOLR-14094
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Adrien Grand
>Priority: Major
>
> I hit the below failure while building the RC.
> {noformat}
>[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Created dataDir:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
>[junit4] 2> 921086 WARN
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=false
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN,
> clientAuth=NaN)
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>[junit4] 2> 921088 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
>[junit4] 2> 927857 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
> -Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
> -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
> compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
>[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
> docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
> locale=ar-QA, timezone=AGT
>[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
> 1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
>[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
> AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
> TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
> IndexSizeTriggerMixedBoundsTest, NumberUtilsTest, TestZkChroot,
> HdfsDirectoryTest, RollingRestartTest, TestSolrCLIRunExample,
> ClusterStateTest, DocValuesMultiTest, TestLeaderElectionWithEmptyReplica,
> TestCustomSort, TestSchemaManager, TestInPlaceUpdatesDistrib,
> TestReloadAndDeleteDocs, HighlighterConfigTest, MBeansHandlerTest,
> PeerSyncWithIndexFingerprintCachingTest, SubstringBytesRefFilterTest,
> TestChildDocTransformer, IndexSizeTriggerTest, HealthCheckHandlerTest,
> SuggesterFSTTest, TestLuceneIndexBackCompat, TestDelegationWithHadoopAuth,
> TestSolrCoreProperties, DistanceFunctionTest,
> SignatureUpdateProcessorFactoryTest, CdcrBootstrapTest, DebugComponentTest,
> TriggerEventQueueTest, TestLegacyBM25SimilarityFactory,
> SolrMetricReporterTes
[jira] [Created] (SOLR-14095) Remove serialization or support serialization filtering
Robert Muir created SOLR-14095: -- Summary: Remove serialization or support serialization filtering Key: SOLR-14095 URL: https://issues.apache.org/jira/browse/SOLR-14095 Project: Solr Issue Type: Task Security Level: Public (Default Security Level. Issues are Public) Reporter: Robert Muir Removing the use of serialization is greatly preferred. But if serialization over the wire must really happen, then we must use JDK's serialization filtering capability to prevent havoc. https://docs.oracle.com/javase/10/core/serialization-filtering1.htm#JSCOR-GUID-3ECB288D-E5BD-4412-892F-E9BB11D4C98A -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14094) TestSolrCachePerf is flaky
[
https://issues.apache.org/jira/browse/SOLR-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997280#comment-16997280
]
Andrzej Bialecki commented on SOLR-14094:
-
[~ichattopadhyaya] yes, this test was actually added in SOLR-13898.
{{FastLRUCache}} has been removed from 9.0. I'll look into this failure to see
if I didn't miss anything obvious but other than that I'm not going to spend a
lot of time on this.
> TestSolrCachePerf is flaky
> --
>
> Key: SOLR-14094
> URL: https://issues.apache.org/jira/browse/SOLR-14094
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Adrien Grand
>Priority: Major
>
> I hit the below failure while building the RC.
> {noformat}
>[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Created dataDir:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
>[junit4] 2> 921086 WARN
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=false
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN,
> clientAuth=NaN)
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>[junit4] 2> 921088 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
>[junit4] 2> 927857 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
> -Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
> -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
> compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
>[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
> docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
> locale=ar-QA, timezone=AGT
>[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
> 1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
>[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
> AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
> TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
> IndexSizeTriggerMixedBoundsTest, NumberUtilsTest, TestZkChroot,
> HdfsDirectoryTest, RollingRestartTest, TestSolrCLIRunExample,
> ClusterStateTest, DocValuesMultiTest, TestLeaderElectionWithEmptyReplica,
> TestCustomSort, TestSchemaManager, TestInPlaceUpdatesDistrib,
> TestReloadAndDeleteDocs, HighlighterConfigTest, MBeansHandlerTest,
> PeerSyncWithIndexFingerprintCachingTest, SubstringBytesRefFilterTest,
> TestChildDocTransformer, IndexSizeTriggerTest, HealthCheckHandlerTest,
> SuggesterFSTTest, TestLuceneIndexBackCompat, TestDelegationWithHadoopAuth,
> TestSo
[jira] [Assigned] (SOLR-14094) TestSolrCachePerf is flaky
[
https://issues.apache.org/jira/browse/SOLR-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki reassigned SOLR-14094:
---
Assignee: Andrzej Bialecki
> TestSolrCachePerf is flaky
> --
>
> Key: SOLR-14094
> URL: https://issues.apache.org/jira/browse/SOLR-14094
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Adrien Grand
>Assignee: Andrzej Bialecki
>Priority: Major
>
> I hit the below failure while building the RC.
> {noformat}
>[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Created dataDir:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
>[junit4] 2> 921086 WARN
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=false
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN,
> clientAuth=NaN)
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>[junit4] 2> 921088 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
>[junit4] 2> 927857 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
> -Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
> -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
> compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
>[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
> docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
> locale=ar-QA, timezone=AGT
>[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
> 1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
>[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
> AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
> TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
> IndexSizeTriggerMixedBoundsTest, NumberUtilsTest, TestZkChroot,
> HdfsDirectoryTest, RollingRestartTest, TestSolrCLIRunExample,
> ClusterStateTest, DocValuesMultiTest, TestLeaderElectionWithEmptyReplica,
> TestCustomSort, TestSchemaManager, TestInPlaceUpdatesDistrib,
> TestReloadAndDeleteDocs, HighlighterConfigTest, MBeansHandlerTest,
> PeerSyncWithIndexFingerprintCachingTest, SubstringBytesRefFilterTest,
> TestChildDocTransformer, IndexSizeTriggerTest, HealthCheckHandlerTest,
> SuggesterFSTTest, TestLuceneIndexBackCompat, TestDelegationWithHadoopAuth,
> TestSolrCoreProperties, DistanceFunctionTest,
> SignatureUpdateProcessorFactoryTest, CdcrBootstrapTest, DebugComponentTest,
> TriggerEventQueueTest, TestLegacyBM25SimilarityFactory,
> SolrMetricReporterTest, TestCorePropertiesReload,
[jira] [Comment Edited] (SOLR-14094) TestSolrCachePerf is flaky
[
https://issues.apache.org/jira/browse/SOLR-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997280#comment-16997280
]
Andrzej Bialecki edited comment on SOLR-14094 at 12/16/19 1:22 PM:
---
[~ichattopadhyaya] yes, this test was actually added in SOLR-13898.
{{FastLRUCache}} has been removed from 9.0. I'll look into this failure to see
if I didn't miss anything obvious but other than that I'm not going to spend a
lot of time on this.
[~jpountz] this failure is specific to {{FastLRUCache}}, this test on master
doesn't use this implementation (because it's no longer present there) so I'm
going to re-enable this test on master.
was (Author: ab):
[~ichattopadhyaya] yes, this test was actually added in SOLR-13898.
{{FastLRUCache}} has been removed from 9.0. I'll look into this failure to see
if I didn't miss anything obvious but other than that I'm not going to spend a
lot of time on this.
> TestSolrCachePerf is flaky
> --
>
> Key: SOLR-14094
> URL: https://issues.apache.org/jira/browse/SOLR-14094
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Adrien Grand
>Assignee: Andrzej Bialecki
>Priority: Major
>
> I hit the below failure while building the RC.
> {noformat}
>[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Created dataDir:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
>[junit4] 2> 921086 WARN
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=false
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN,
> clientAuth=NaN)
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>[junit4] 2> 921088 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
>[junit4] 2> 927857 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
> -Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
> -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
> compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
>[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
> docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
> locale=ar-QA, timezone=AGT
>[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
> 1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
>[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
> AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
> TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
> IndexSizeTrigger
[jira] [Commented] (LUCENE-9087) Should the BKD tree use a fixed maxPointsInLeafNode?
[ https://issues.apache.org/jira/browse/LUCENE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997283#comment-16997283 ] Ignacio Vera commented on LUCENE-9087: -- Thanks [~jpountz] for the explanation. > Should the BKD tree use a fixed maxPointsInLeafNode? > - > > Key: LUCENE-9087 > URL: https://issues.apache.org/jira/browse/LUCENE-9087 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Ignacio Vera >Priority: Major > > Currently the BKD tree uses a fixed maxPointsInLeafNode provided in the > constructor. For the current default codec the value is set to 1200. This is > a good compromise between memory usage and performance of the BKD tree. > Lowering this value can increase search performance but it has a penalty in > memory usage. Now that the BKD tree can be load off-heap, this can be less of > a concern. Note that lowering too much that value can hurt performance as > well as the tree becomes too deep and benefits are gone. > For data types that use the tree as an effective R-tree (ranges and shapes > datatypes) the benefits are larger as it can minimise the overlap between > leaf nodes. > Finally, creating too many leaf nodes can be dangerous at write time as > memory usage depends on the number of leaf nodes created. The writer creates > a long array of length = numberOfLeafNodes. > What I am wondering here is if we can improve this situation in order to > create the most efficient tree? My current ideas are: > > * We can adapt the points per leaf depending on that number so we create a > tree with the best depth and best points per leaf. Note that for the for 1D > case we have an upper estimation of the number of points that the tree will > be indexing. > * Add a mechanism so field types can easily define their best points per > leaf. In the case, field types like ranges or shapes can define its own value > to minimise overlap. > * Maybe the default is just too high now that we can load the tree off-heap. > > Any thoughts? > > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
Dawid Weiss created SOLR-14096:
--
Summary: DistribPackageStore attempts to create .filestore in
read-only directory
Key: SOLR-14096
URL: https://issues.apache.org/jira/browse/SOLR-14096
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Dawid Weiss
I see this in the logs when running with security manager:
{code}
1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
o.a.s.f.DistribPackageStore Unable to create
[/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
directory in SOLR_HOME
[/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
Features requiring this directory may fail.
=> java.security.AccessControlException: access denied
("java.io.FilePermission"
"/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
"write")
at
java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
java.security.AccessControlException: access denied ("java.io.FilePermission"
"/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
"write")
at
java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
~[?:?]
at
java.security.AccessController.checkPermission(AccessController.java:897) ~[?:?]
at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
~[?:?]
at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
at java.io.File.mkdir(File.java:1323) ~[?:?]
at java.io.File.mkdirs(File.java:1355) ~[?:?]
at
org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
~[main/:?]
at
org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
~[main/:?]
at
org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
~[main/:?]
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
~[main/:?]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9094) Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
[
https://issues.apache.org/jira/browse/LUCENE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997293#comment-16997293
]
Adrien Grand commented on LUCENE-9094:
--
+1
> Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
> --
>
> Key: LUCENE-9094
> URL: https://issues.apache.org/jira/browse/LUCENE-9094
> Project: Lucene - Core
> Issue Type: Task
> Components: general/build
>Reporter: Robert Muir
>Assignee: Robert Muir
>Priority: Major
> Attachments: LUCENE-9094.patch
>
>
> suggested build failure message:
> {quote}
> [forbidden-apis] Forbidden class/interface use: java.io.ObjectInputStream
> [Java deserialization is unsafe when the data is untrusted. The java
> developer is powerless: no checks or casts help, exploitation can happen in
> places such as clinit or finalize!]
> {quote}
> I will whitelist existing places doing this for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997295#comment-16997295
]
Adrien Grand commented on SOLR-14096:
-
This also caused a failure of org.apache.solr.handler.admin.LoggingHandlerTest
when building a RC.
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Major
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997301#comment-16997301
]
Dawid Weiss commented on SOLR-14096:
This test fails for me as well.
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Major
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9095) remove java serialization from lucene/replicator
Robert Muir created LUCENE-9095: --- Summary: remove java serialization from lucene/replicator Key: LUCENE-9095 URL: https://issues.apache.org/jira/browse/LUCENE-9095 Project: Lucene - Core Issue Type: Task Components: modules/replicator Environment: re Reporter: Robert Muir This seems to be deserializing java objects from http streams. In LUCENE-9094 I marked the usages with SuppressForbidden, just to try to fend off additional serialization from being added to the codebase. But we should really fix this one! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on issue #1082: SOLR-13984: add (experimental, disabled by default) security manager support
rmuir commented on issue #1082: SOLR-13984: add (experimental, disabled by default) security manager support URL: https://github.com/apache/lucene-solr/pull/1082#issuecomment-566074507 https://issues.apache.org/jira/browse/LUCENE-9094 https://issues.apache.org/jira/browse/LUCENE-9095 https://issues.apache.org/jira/browse/SOLR-14095 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997313#comment-16997313
]
Kevin Risden commented on SOLR-14096:
-
I already opened a Jira for this error. Looks like the folder recently changed
names though. I haven't seen this cause test failures unless it's the new name?
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Major
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997315#comment-16997315
]
Kevin Risden commented on SOLR-14096:
-
SOLR-14078
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Major
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9094) Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
[
https://issues.apache.org/jira/browse/LUCENE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997316#comment-16997316
]
Kevin Risden commented on LUCENE-9094:
--
+1
> Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
> --
>
> Key: LUCENE-9094
> URL: https://issues.apache.org/jira/browse/LUCENE-9094
> Project: Lucene - Core
> Issue Type: Task
> Components: general/build
>Reporter: Robert Muir
>Assignee: Robert Muir
>Priority: Major
> Attachments: LUCENE-9094.patch
>
>
> suggested build failure message:
> {quote}
> [forbidden-apis] Forbidden class/interface use: java.io.ObjectInputStream
> [Java deserialization is unsafe when the data is untrusted. The java
> developer is powerless: no checks or casts help, exploitation can happen in
> places such as clinit or finalize!]
> {quote}
> I will whitelist existing places doing this for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #587: LUCENE-8707: Add LatLonShape and XYShape distance query
jpountz commented on a change in pull request #587: LUCENE-8707: Add
LatLonShape and XYShape distance query
URL: https://github.com/apache/lucene-solr/pull/587#discussion_r358241048
##
File path:
lucene/sandbox/src/java/org/apache/lucene/document/LatLonShapeDistanceQuery.java
##
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.document;
+
+import org.apache.lucene.geo.Circle;
+import org.apache.lucene.geo.WGS84Circle2D;
+import org.apache.lucene.geo.Component2D;
+import org.apache.lucene.geo.GeoEncodingUtils;
+import org.apache.lucene.index.PointValues.Relation;
+import org.apache.lucene.util.NumericUtils;
+
+/**
+ * Finds all previously indexed shapes that intersect the specified distance
query.
+ *
+ * The field must be indexed using
+ * {@link LatLonShape#createIndexableFields} added per document.
+ *
+ * @lucene.experimental
+ **/
+final class LatLonShapeDistanceQuery extends ShapeQuery {
+ final Circle circle;
+ final Component2D circle2D;
+
+ public LatLonShapeDistanceQuery(String field, ShapeField.QueryRelation
queryRelation, Circle circle) {
+super(field, queryRelation);
+this.circle = circle;
+this.circle2D = WGS84Circle2D.create(circle);
+ }
+
+ @Override
+ protected Relation relateRangeBBoxToQuery(int minXOffset, int minYOffset,
byte[] minTriangle,
+int maxXOffset, int maxYOffset,
byte[] maxTriangle) {
+
+double minLat =
GeoEncodingUtils.decodeLatitude(NumericUtils.sortableBytesToInt(minTriangle,
minYOffset));
+double minLon =
GeoEncodingUtils.decodeLongitude(NumericUtils.sortableBytesToInt(minTriangle,
minXOffset));
+double maxLat =
GeoEncodingUtils.decodeLatitude(NumericUtils.sortableBytesToInt(maxTriangle,
maxYOffset));
+double maxLon =
GeoEncodingUtils.decodeLongitude(NumericUtils.sortableBytesToInt(maxTriangle,
maxXOffset));
+
+// check internal node against query
+return circle2D.relate(minLon, maxLon, minLat, maxLat);
+ }
+
+ @Override
+ protected boolean queryMatches(byte[] t, ShapeField.DecodedTriangle
scratchTriangle, ShapeField.QueryRelation queryRelation) {
+ShapeField.decodeTriangle(t, scratchTriangle);
+
+double alat = GeoEncodingUtils.decodeLatitude(scratchTriangle.aY);
+double alon = GeoEncodingUtils.decodeLongitude(scratchTriangle.aX);
+double blat = GeoEncodingUtils.decodeLatitude(scratchTriangle.bY);
+double blon = GeoEncodingUtils.decodeLongitude(scratchTriangle.bX);
+double clat = GeoEncodingUtils.decodeLatitude(scratchTriangle.cY);
+double clon = GeoEncodingUtils.decodeLongitude(scratchTriangle.cX);
+
+switch (queryRelation) {
+ case INTERSECTS: return circle2D.relateTriangle(alon, alat, blon, blat,
clon, clat) != Relation.CELL_OUTSIDE_QUERY;
+ case WITHIN: return circle2D.relateTriangle(alon, alat, blon, blat,
clon, clat) == Relation.CELL_INSIDE_QUERY;
+ case DISJOINT: return circle2D.relateTriangle(alon, alat, blon, blat,
clon, clat) == Relation.CELL_OUTSIDE_QUERY;
+ default: throw new IllegalArgumentException("Unsupported query type :["
+ queryRelation + "]");
Review comment:
Let's fail CONTAINS directly in the constructor? By the way, are we skipping
it because it's a lot of work to support or because it doesn't make much sense?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #587: LUCENE-8707: Add LatLonShape and XYShape distance query
jpountz commented on a change in pull request #587: LUCENE-8707: Add
LatLonShape and XYShape distance query
URL: https://github.com/apache/lucene-solr/pull/587#discussion_r358241577
##
File path:
lucene/sandbox/src/java/org/apache/lucene/document/XYShapeDistanceQuery.java
##
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.document;
+
+
+import org.apache.lucene.geo.Component2D;
+import org.apache.lucene.geo.XYCircle;
+import org.apache.lucene.geo.XYCircle2D;
+import org.apache.lucene.geo.XYEncodingUtils;
+import org.apache.lucene.index.PointValues.Relation;
+import org.apache.lucene.util.NumericUtils;
+
+/**
+ * Finds all previously indexed shapes that intersect the specified distance
query.
+ *
+ * The field must be indexed using
+ * {@link XYShape#createIndexableFields} added per document.
+ *
+ * @lucene.experimental
Review comment:
no need for this tag
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #587: LUCENE-8707: Add LatLonShape and XYShape distance query
jpountz commented on a change in pull request #587: LUCENE-8707: Add
LatLonShape and XYShape distance query
URL: https://github.com/apache/lucene-solr/pull/587#discussion_r358242820
##
File path: lucene/sandbox/src/java/org/apache/lucene/geo/Circle.java
##
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.geo;
+
+
+/**
+ * Represents a circle on the earth's surface.
+ *
+ * NOTES:
+ *
+ *Latitude/longitude values must be in decimal degrees.
+ *Radius must be in meters.
+ * For more advanced GeoSpatial indexing and query operations see the
{@code spatial-extras} module
+ *
+ * @lucene.experimental
+ */
+public class Circle {
+ /** Center latitude */
+ private final double lat;
+ /** Center longitude */
+ private final double lon;
+ /** radius in meters */
+ private final double distance;
+ /** Max radius allowed, half of the earth mean radius.*/
+ public static double MAXRADIUS = GeoUtils.EARTH_MEAN_RADIUS_METERS / 2.0;
+
+
+ /**
+ * Creates a new circle from the supplied latitude/longitude center and
distance in meters..
+ */
+ public Circle(double lat, double lon, double radiusMeters) {
+GeoUtils.checkLatitude(lat);
+GeoUtils.checkLongitude(lon);
+if (radiusMeters <= 0) {
+ throw new IllegalArgumentException("Radius must be bigger than 0, got "
+ radiusMeters);
+}
+if (radiusMeters >= MAXRADIUS) {
Review comment:
when validating, it's often beffer to do `radiusMeters < MAXRADIUS ==
false`, which has the nice property of also failing if `radiusMeters` in NaN
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #587: LUCENE-8707: Add LatLonShape and XYShape distance query
jpountz commented on a change in pull request #587: LUCENE-8707: Add
LatLonShape and XYShape distance query
URL: https://github.com/apache/lucene-solr/pull/587#discussion_r358249163
##
File path: lucene/sandbox/src/java/org/apache/lucene/geo/WGS84Circle2D.java
##
@@ -0,0 +1,285 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.geo;
+
+import org.apache.lucene.index.PointValues.Relation;
+import org.apache.lucene.util.SloppyMath;
+
+
+/**
+ * 2D circle implementation containing geo spatial logic.
+ *
+ * @lucene.internal
+ */
+public class WGS84Circle2D implements Component2D {
+ final Rectangle rectangle;
+ final boolean crossesDateline;
+ final double centerLat;
+ final double centerLon;
+ final double sortKey;
+ final double axisLat;
+
+ private WGS84Circle2D(double centerLon, double centerLat, double distance) {
+this.centerLat = centerLat;
+this.centerLon = centerLon;
+this.rectangle = Rectangle.fromPointDistance(centerLat, centerLon,
distance);
+this.sortKey = GeoUtils.distanceQuerySortKey(distance);
+this.axisLat = Rectangle.axisLat(centerLat, distance);
+this.crossesDateline = rectangle.minLon > rectangle.maxLon;
+ }
+
+ @Override
+ public double getMinX() {
+return rectangle.minLon;
+ }
+
+ @Override
+ public double getMaxX() {
+return rectangle.maxLon;
+ }
+
+ @Override
+ public double getMinY() {
+return rectangle.minLat;
+ }
+
+ @Override
+ public double getMaxY() {
+return rectangle.maxLat;
+ }
+
+ @Override
+ public boolean contains(double x, double y) {
+return SloppyMath.haversinSortKey(y, x, this.centerLat, this.centerLon) <=
sortKey;
+ }
+
+ @Override
+ public Relation relate(double minX, double maxX, double minY, double maxY) {
+if (disjoint(minX, maxX, minY, maxY)) {
+ return Relation.CELL_OUTSIDE_QUERY;
+}
+if (within(minX, maxX, minY, maxY)) {
+ return Relation.CELL_CROSSES_QUERY;
+}
+return GeoUtils.relate(minY, maxY, minX, maxX, centerLat, centerLon,
sortKey, axisLat);
+ }
+
+ @Override
+ public Relation relateTriangle(double minX, double maxX, double minY, double
maxY, double ax, double ay, double bx, double by, double cx, double cy) {
+if (disjoint(minX, maxX, minY, maxY)) {
+ return Relation.CELL_OUTSIDE_QUERY;
+}
+if (ax == bx && bx == cx && ay == by && by == cy) {
+ // indexed "triangle" is a point: shortcut by checking contains
+ return contains(ax, ay) ? Relation.CELL_INSIDE_QUERY :
Relation.CELL_OUTSIDE_QUERY;
+} else if (ax == cx && ay == cy) {
+ // indexed "triangle" is a line segment: shortcut by calling appropriate
method
+ return relateIndexedLineSegment(ax, ay, bx, by);
+} else if (ax == bx && ay == by) {
+ // indexed "triangle" is a line segment: shortcut by calling appropriate
method
+ return relateIndexedLineSegment(bx, by, cx, cy);
+} else if (bx == cx && by == cy) {
+ // indexed "triangle" is a line segment: shortcut by calling appropriate
method
+ return relateIndexedLineSegment(cx, cy, ax, ay);
+}
+// indexed "triangle" is a triangle:
+return relateIndexedTriangle(minX, maxX, minY, maxY, ax, ay, bx, by, cx,
cy);
+ }
+
+ @Override
+ public WithinRelation withinTriangle(double minX, double maxX, double minY,
double maxY, double ax, double ay, boolean ab, double bx, double by, boolean
bc, double cx, double cy, boolean ca) {
+// short cut, lines and points cannot contain this type of shape
+if ((ax == bx && ay == by) || (ax == cx && ay == cy) || (bx == cx && by ==
cy)) {
+ return WithinRelation.DISJOINT;
+}
+
+if (disjoint(minX, maxX, minY, maxY)) {
+ return WithinRelation.DISJOINT;
+}
+
+// if any of the points is inside the polygon, the polygon cannot be
within this indexed
+// shape because points belong to the original indexed shape.
+if (contains(ax, ay) || contains(bx, by) || contains(cx, cy)) {
+ return WithinRelation.NOTWITHIN;
+}
+
+WithinRelation relation = WithinRelation.DISJOINT;
+// if any of the edges intersects an the edge belongs to the shape then it
cannot be within.
+// if it only
[GitHub] [lucene-solr] jpountz commented on a change in pull request #587: LUCENE-8707: Add LatLonShape and XYShape distance query
jpountz commented on a change in pull request #587: LUCENE-8707: Add
LatLonShape and XYShape distance query
URL: https://github.com/apache/lucene-solr/pull/587#discussion_r358239995
##
File path:
lucene/sandbox/src/java/org/apache/lucene/document/LatLonShapeDistanceQuery.java
##
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.document;
+
+import org.apache.lucene.geo.Circle;
+import org.apache.lucene.geo.WGS84Circle2D;
+import org.apache.lucene.geo.Component2D;
+import org.apache.lucene.geo.GeoEncodingUtils;
+import org.apache.lucene.index.PointValues.Relation;
+import org.apache.lucene.util.NumericUtils;
+
+/**
+ * Finds all previously indexed shapes that intersect the specified distance
query.
+ *
+ * The field must be indexed using
+ * {@link LatLonShape#createIndexableFields} added per document.
+ *
+ * @lucene.experimental
Review comment:
no need for this javadoc tag on classes that are not public
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #587: LUCENE-8707: Add LatLonShape and XYShape distance query
jpountz commented on a change in pull request #587: LUCENE-8707: Add
LatLonShape and XYShape distance query
URL: https://github.com/apache/lucene-solr/pull/587#discussion_r358246538
##
File path: lucene/sandbox/src/java/org/apache/lucene/geo/WGS84Circle2D.java
##
@@ -0,0 +1,285 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.lucene.geo;
+
+import org.apache.lucene.index.PointValues.Relation;
+import org.apache.lucene.util.SloppyMath;
+
+
+/**
+ * 2D circle implementation containing geo spatial logic.
+ *
+ * @lucene.internal
+ */
+public class WGS84Circle2D implements Component2D {
+ final Rectangle rectangle;
+ final boolean crossesDateline;
+ final double centerLat;
+ final double centerLon;
+ final double sortKey;
+ final double axisLat;
+
+ private WGS84Circle2D(double centerLon, double centerLat, double distance) {
+this.centerLat = centerLat;
+this.centerLon = centerLon;
+this.rectangle = Rectangle.fromPointDistance(centerLat, centerLon,
distance);
+this.sortKey = GeoUtils.distanceQuerySortKey(distance);
+this.axisLat = Rectangle.axisLat(centerLat, distance);
+this.crossesDateline = rectangle.minLon > rectangle.maxLon;
+ }
+
+ @Override
+ public double getMinX() {
+return rectangle.minLon;
+ }
+
+ @Override
+ public double getMaxX() {
+return rectangle.maxLon;
+ }
+
+ @Override
+ public double getMinY() {
+return rectangle.minLat;
+ }
+
+ @Override
+ public double getMaxY() {
+return rectangle.maxLat;
+ }
+
+ @Override
+ public boolean contains(double x, double y) {
+return SloppyMath.haversinSortKey(y, x, this.centerLat, this.centerLon) <=
sortKey;
+ }
+
+ @Override
+ public Relation relate(double minX, double maxX, double minY, double maxY) {
+if (disjoint(minX, maxX, minY, maxY)) {
+ return Relation.CELL_OUTSIDE_QUERY;
+}
+if (within(minX, maxX, minY, maxY)) {
+ return Relation.CELL_CROSSES_QUERY;
+}
+return GeoUtils.relate(minY, maxY, minX, maxX, centerLat, centerLon,
sortKey, axisLat);
+ }
+
+ @Override
+ public Relation relateTriangle(double minX, double maxX, double minY, double
maxY, double ax, double ay, double bx, double by, double cx, double cy) {
+if (disjoint(minX, maxX, minY, maxY)) {
+ return Relation.CELL_OUTSIDE_QUERY;
+}
+if (ax == bx && bx == cx && ay == by && by == cy) {
+ // indexed "triangle" is a point: shortcut by checking contains
+ return contains(ax, ay) ? Relation.CELL_INSIDE_QUERY :
Relation.CELL_OUTSIDE_QUERY;
+} else if (ax == cx && ay == cy) {
+ // indexed "triangle" is a line segment: shortcut by calling appropriate
method
+ return relateIndexedLineSegment(ax, ay, bx, by);
+} else if (ax == bx && ay == by) {
+ // indexed "triangle" is a line segment: shortcut by calling appropriate
method
+ return relateIndexedLineSegment(bx, by, cx, cy);
+} else if (bx == cx && by == cy) {
+ // indexed "triangle" is a line segment: shortcut by calling appropriate
method
+ return relateIndexedLineSegment(cx, cy, ax, ay);
+}
+// indexed "triangle" is a triangle:
+return relateIndexedTriangle(minX, maxX, minY, maxY, ax, ay, bx, by, cx,
cy);
+ }
+
+ @Override
+ public WithinRelation withinTriangle(double minX, double maxX, double minY,
double maxY, double ax, double ay, boolean ab, double bx, double by, boolean
bc, double cx, double cy, boolean ca) {
+// short cut, lines and points cannot contain this type of shape
+if ((ax == bx && ay == by) || (ax == cx && ay == cy) || (bx == cx && by ==
cy)) {
+ return WithinRelation.DISJOINT;
+}
+
+if (disjoint(minX, maxX, minY, maxY)) {
+ return WithinRelation.DISJOINT;
+}
+
+// if any of the points is inside the polygon, the polygon cannot be
within this indexed
+// shape because points belong to the original indexed shape.
+if (contains(ax, ay) || contains(bx, by) || contains(cx, cy)) {
+ return WithinRelation.NOTWITHIN;
+}
+
+WithinRelation relation = WithinRelation.DISJOINT;
+// if any of the edges intersects an the edge belongs to the shape then it
cannot be within.
+// if it only
[GitHub] [lucene-solr] jpountz commented on a change in pull request #587: LUCENE-8707: Add LatLonShape and XYShape distance query
jpountz commented on a change in pull request #587: LUCENE-8707: Add
LatLonShape and XYShape distance query
URL: https://github.com/apache/lucene-solr/pull/587#discussion_r358242372
##
File path: lucene/sandbox/src/java/org/apache/lucene/geo/Circle.java
##
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.geo;
+
+
+/**
+ * Represents a circle on the earth's surface.
+ *
+ * NOTES:
+ *
+ *Latitude/longitude values must be in decimal degrees.
+ *Radius must be in meters.
+ * For more advanced GeoSpatial indexing and query operations see the
{@code spatial-extras} module
+ *
+ * @lucene.experimental
+ */
+public class Circle {
+ /** Center latitude */
+ private final double lat;
+ /** Center longitude */
+ private final double lon;
+ /** radius in meters */
+ private final double distance;
Review comment:
maybe call it `radius` like in the javadocs?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #587: LUCENE-8707: Add LatLonShape and XYShape distance query
jpountz commented on a change in pull request #587: LUCENE-8707: Add
LatLonShape and XYShape distance query
URL: https://github.com/apache/lucene-solr/pull/587#discussion_r358252975
##
File path:
lucene/sandbox/src/java/org/apache/lucene/document/LatLonShapeDistanceQuery.java
##
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.document;
+
+import org.apache.lucene.geo.Circle;
+import org.apache.lucene.geo.WGS84Circle2D;
+import org.apache.lucene.geo.Component2D;
+import org.apache.lucene.geo.GeoEncodingUtils;
+import org.apache.lucene.index.PointValues.Relation;
+import org.apache.lucene.util.NumericUtils;
+
+/**
+ * Finds all previously indexed shapes that intersect the specified distance
query.
+ *
+ * The field must be indexed using
+ * {@link LatLonShape#createIndexableFields} added per document.
+ *
+ * @lucene.experimental
+ **/
+final class LatLonShapeDistanceQuery extends ShapeQuery {
+ final Circle circle;
+ final Component2D circle2D;
+
+ public LatLonShapeDistanceQuery(String field, ShapeField.QueryRelation
queryRelation, Circle circle) {
+super(field, queryRelation);
+this.circle = circle;
+this.circle2D = WGS84Circle2D.create(circle);
+ }
+
+ @Override
+ protected Relation relateRangeBBoxToQuery(int minXOffset, int minYOffset,
byte[] minTriangle,
+int maxXOffset, int maxYOffset,
byte[] maxTriangle) {
+
+double minLat =
GeoEncodingUtils.decodeLatitude(NumericUtils.sortableBytesToInt(minTriangle,
minYOffset));
+double minLon =
GeoEncodingUtils.decodeLongitude(NumericUtils.sortableBytesToInt(minTriangle,
minXOffset));
+double maxLat =
GeoEncodingUtils.decodeLatitude(NumericUtils.sortableBytesToInt(maxTriangle,
maxYOffset));
+double maxLon =
GeoEncodingUtils.decodeLongitude(NumericUtils.sortableBytesToInt(maxTriangle,
maxXOffset));
+
+// check internal node against query
+return circle2D.relate(minLon, maxLon, minLat, maxLat);
+ }
+
+ @Override
+ protected boolean queryMatches(byte[] t, ShapeField.DecodedTriangle
scratchTriangle, ShapeField.QueryRelation queryRelation) {
+ShapeField.decodeTriangle(t, scratchTriangle);
+
+double alat = GeoEncodingUtils.decodeLatitude(scratchTriangle.aY);
+double alon = GeoEncodingUtils.decodeLongitude(scratchTriangle.aX);
+double blat = GeoEncodingUtils.decodeLatitude(scratchTriangle.bY);
+double blon = GeoEncodingUtils.decodeLongitude(scratchTriangle.bX);
+double clat = GeoEncodingUtils.decodeLatitude(scratchTriangle.cY);
+double clon = GeoEncodingUtils.decodeLongitude(scratchTriangle.cX);
+
+switch (queryRelation) {
+ case INTERSECTS: return circle2D.relateTriangle(alon, alat, blon, blat,
clon, clat) != Relation.CELL_OUTSIDE_QUERY;
Review comment:
Are we expecting INTERSECTS to be the common case? If yes it's a bit
disappointing that it it checks all corners in the common case, when it could
stop after one corner falls within the circle? In any case I don't think we
need to handle it now, but we should at least leave a TODO?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14066) Deprecate DIH
[ https://issues.apache.org/jira/browse/SOLR-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997318#comment-16997318 ] Jan Høydahl commented on SOLR-14066: Yea, what do you think Eric about the 6 steps above? Would you do it differently? I unassigned myself from this issue and closed the PR. So it's up for grabs. > Deprecate DIH > - > > Key: SOLR-14066 > URL: https://issues.apache.org/jira/browse/SOLR-14066 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - DataImportHandler >Reporter: Ishan Chattopadhyaya >Priority: Major > Attachments: image-2019-12-14-19-58-39-314.png > > Time Spent: 20m > Remaining Estimate: 0h > > DataImportHandler has outlived its utility. DIH doesn't need to remain inside > Solr anymore. Let us deprecate DIH in 8.4 (and remove it from the Solr distro > in 9x or 10x). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy closed pull request #1084: SOLR-14066 Deprecate DataImportHandler
janhoy closed pull request #1084: SOLR-14066 Deprecate DataImportHandler URL: https://github.com/apache/lucene-solr/pull/1084 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on issue #1084: SOLR-14066 Deprecate DataImportHandler
janhoy commented on issue #1084: SOLR-14066 Deprecate DataImportHandler URL: https://github.com/apache/lucene-solr/pull/1084#issuecomment-566079512 We'll not deprecate in 8.4 but take another approach. Closing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14066) Deprecate DIH
[ https://issues.apache.org/jira/browse/SOLR-14066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl resolved SOLR-14066. Resolution: Won't Do > Deprecate DIH > - > > Key: SOLR-14066 > URL: https://issues.apache.org/jira/browse/SOLR-14066 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: contrib - DataImportHandler >Reporter: Ishan Chattopadhyaya >Priority: Major > Attachments: image-2019-12-14-19-58-39-314.png > > Time Spent: 40m > Remaining Estimate: 0h > > DataImportHandler has outlived its utility. DIH doesn't need to remain inside > Solr anymore. Let us deprecate DIH in 8.4 (and remove it from the Solr distro > in 9x or 10x). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated SOLR-14096:
Priority: Blocker (was: Major)
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9077) Gradle build
[
https://issues.apache.org/jira/browse/LUCENE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Weiss updated LUCENE-9077:
Description:
This task focuses on providing gradle-based build equivalent for Lucene and
Solr (on master branch). See notes below on why this respin is needed.
The code lives on *gradle-master* branch. It is kept with sync with *master*.
Try running the following to see an overview of helper guides concerning
typical workflow, testing and ant-migration helpers:
gradlew :help
A list of items that needs to be added or requires work. If you'd like to work
on any of these, please add your name to the list. Once you have a patch/ pull
request let me (dweiss) know - I'll try to coordinate the merges.
* (/) Apply forbiddenAPIs
* (/) Generate hardware-aware gradle defaults for parallelism (count of
workers and test JVMs).
* (/) Fail the build if --tests filter is applied and no tests execute during
the entire build (this allows for an empty set of filtered tests at single
project level).
* (/) Port other settings and randomizations from common-build.xml
* (/) Configure security policy/ sandboxing for tests.
* (/) test's console output on -Ptests.verbose=true
* (/) add a :helpDeps explanation to how the dependency system works (palantir
plugin, lockfile) and how to retrieve structured information about current
dependencies of a given module (in a tree-like output).
* (/) jar checksums, jar checksum computation and validation. This should be
done without intermediate folders (directly on dependency sets).
* (/) verify min. JVM version and exact gradle version on build startup to
minimize odd build side-effects
* identify and list precommit tasks so that they can be ported one by one.
(Mark's branch has some of this stuff already implemented)
* Repro-line for failed tests/ runs.
Hard-to-implement stuff already investigated:
* (!) *Printing console output of failed tests.* There doesn't seem to be any
way to do this in a reasonably efficient way. There are onOutput listeners but
they're slow to operate and solr tests emit *tons* of output so it's an
overkill. The only solution I think would be feasible is to parse test report
XMLs after the tests are complete and collect the output of failed tests from
there. The downside is that XMLs contain separated sysout/syserr.
* (!) *Tests working with security-debug logs or other JVM-early log output*.
From what I can see in the code Gradle's test runner works by redirecting
Java's stdout/ syserr so this just won't work. Perhaps we can spin the
ant-based test runner for such corner-cases.
Of lesser importance:
* add rendering of javadocs (gradlew javadoc) and attaching them to maven
publications.
* Add test 'beasting' (rerunning the same suite multiple times). I'm afraid
it'll be difficult to run it sensibly because gradle doesn't offer cwd
separation for the forked test runners.
* if you diff solr packaged distribution against ant-created distribution
there are minor differences in library versions and some JARs are excluded/
moved around. I didn't try to force these as everything seems to work (tests,
etc.) – perhaps these differences should be fixed in the ant build instead.
* identify and port any other "check" utilities that may be called from ant.
(Mark's branch has some of this stuff already implemented)
* [EOE] identify and port various "regenerate" tasks from ant builds (javacc,
precompiled automata, etc.)
* fill in POM details in gradle/defaults-maven.gradle so that they reflect the
previous content better (dependencies aside).
* Add any IDE integration layers that should be added (I use IntelliJ and it
imports the project out of the box, without the need for any special tuning).
* *Clean up dependencies, especially for Solr*: any \{ transitive = false }
should just explicitly exclude whatever they don't need (and their dependencies
currently declared explicitly should be folded). Figure out which scope to
import a dependency to.
* Add Solr packaging for docs/* (see TODO in packaging/build.gradle; currently
XSLT...)
* I didn't bother adding Solr dist/test-framework to packaging (who'd use it
from a binary distribution?
*{color:#ff}Note:{color}* this builds on the work done by Mark Miller and
Cao Mạnh Đạt but also applies lessons learned from those two efforts:
* *Do not try to do too many things at once*. If we deviate too far from
master, the branch will be hard to merge.
* *Do everything in baby-steps* and add small, independent build fragments
replacing the old ant infrastructure.
* *Try to engage people to run, test and contribute early*. It can't be a
one-man effort. The more people understand and can contribute to the build, the
more healthy it will be.
was:
This task focuses on providing gradle-based build equivalent for Lucene and
Solr (on master branch). See notes below on w
[jira] [Commented] (SOLR-14095) Remove serialization or support serialization filtering
[
https://issues.apache.org/jira/browse/SOLR-14095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997325#comment-16997325
]
Robert Muir commented on SOLR-14095:
If we really have to keep this stuff, I think the preferred way is to use
{{jdk.serialFilter}} system property. This way, protection can be backported
back to 8.x without requiring java 9: if the user has java 9+ then they get the
benefit.
> Remove serialization or support serialization filtering
> ---
>
> Key: SOLR-14095
> URL: https://issues.apache.org/jira/browse/SOLR-14095
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Robert Muir
>Priority: Major
>
> Removing the use of serialization is greatly preferred.
> But if serialization over the wire must really happen, then we must use JDK's
> serialization filtering capability to prevent havoc.
> https://docs.oracle.com/javase/10/core/serialization-filtering1.htm#JSCOR-GUID-3ECB288D-E5BD-4412-892F-E9BB11D4C98A
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14084) Security manager access denied not causing tests to fail
[
https://issues.apache.org/jira/browse/SOLR-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997328#comment-16997328
]
Joel Bernstein commented on SOLR-14084:
---
Let's also discuss the "userfiles" directory which is appearing in the test
reports.
{code:java}
30 access denied ("java.io.FilePermission"
"/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/userfiles"
"write"){code}
This directory is created in the SolrResourceLoader as follows:
{code:java}
/**
* Solr allows users to store arbitrary files in a special directory located
directly under SOLR_HOME.
*
* This directory is generally created by each node on startup. Files located
in this directory can then be
* manipulated using select Solr features (e.g. streaming expressions).
*/
public static final String USER_FILES_DIRECTORY = "userfiles";
public static void ensureUserFilesDataDir(Path solrHome) {
final Path userFilesPath = getUserFilesPath(solrHome);
final File userFilesDirectory = new File(userFilesPath.toString());
if (!userFilesDirectory.exists()) {
try {
final boolean created = userFilesDirectory.mkdir();
if (!created) {
log.warn("Unable to create [{}] directory in SOLR_HOME [{}]. Features
requiring this directory may fail.", USER_FILES_DIRECTORY, solrHome);
}
} catch (Exception e) {
log.warn("Unable to create [" + USER_FILES_DIRECTORY + "] directory in
SOLR_HOME [" + solrHome + "]. Features requiring this directory may fail.", e);
}
}
}
public static Path getUserFilesPath(Path solrHome) {
return Paths.get(solrHome.toAbsolutePath().toString(),
USER_FILES_DIRECTORY).toAbsolutePath();
} {code}
Tests pass for me when they are run using the basic:
{code:java}
ant test {code}
There are specific tests that exercise this directory which pass, but that may
be because the tests also populate this directory with a file in a way that
would create the directory if it does not exist.
> Security manager access denied not causing tests to fail
>
>
> Key: SOLR-14084
> URL: https://issues.apache.org/jira/browse/SOLR-14084
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Kevin Risden
>Priority: Major
>
> FYI [~rcmuir] this is from a run where all the tests pass.
> Looking at the output in ./build/solr-core/test/tests-report.txt
> {code:java}
> # grep -F 'access denied' tests-report.txt | cut -d':' -f2 | sort | uniq -c
>1 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/build/solr-core/test/J2/temp/solr.util.TestSolrCLIRunExample_7960AD1EAA781935-001/tempDir-001/failExecuteScript"
> "execute")
> 952 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/filestore"
> "write")
> 30 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/userfiles"
> "write")
> 54 access denied ("java.net.SocketPermission" "--" "resolve")
>2 access denied ("java.net.SocketPermission" "thrasher-T100" "resolve")
>4 127.0.0.1
> {code}
> These didn't cause tests to fail but need to see why these are happening. I
> wouldn't expect we need most of these.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9094) Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
[
https://issues.apache.org/jira/browse/LUCENE-9094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997326#comment-16997326
]
Jan Høydahl commented on LUCENE-9094:
-
+1
> Ban ObjectInputStream and ObjectOutputStream in forbidden-apis
> --
>
> Key: LUCENE-9094
> URL: https://issues.apache.org/jira/browse/LUCENE-9094
> Project: Lucene - Core
> Issue Type: Task
> Components: general/build
>Reporter: Robert Muir
>Assignee: Robert Muir
>Priority: Major
> Attachments: LUCENE-9094.patch
>
>
> suggested build failure message:
> {quote}
> [forbidden-apis] Forbidden class/interface use: java.io.ObjectInputStream
> [Java deserialization is unsafe when the data is untrusted. The java
> developer is powerless: no checks or casts help, exploitation can happen in
> places such as clinit or finalize!]
> {quote}
> I will whitelist existing places doing this for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14095) Remove serialization or support serialization filtering
[ https://issues.apache.org/jira/browse/SOLR-14095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997329#comment-16997329 ] Robert Muir commented on SOLR-14095: In fact i think jdk.serialFilter may exist in java 8 too. Need to check. > Remove serialization or support serialization filtering > --- > > Key: SOLR-14095 > URL: https://issues.apache.org/jira/browse/SOLR-14095 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Robert Muir >Priority: Major > > Removing the use of serialization is greatly preferred. > But if serialization over the wire must really happen, then we must use JDK's > serialization filtering capability to prevent havoc. > https://docs.oracle.com/javase/10/core/serialization-filtering1.htm#JSCOR-GUID-3ECB288D-E5BD-4412-892F-E9BB11D4C98A -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997330#comment-16997330
]
Ishan Chattopadhyaya commented on SOLR-14096:
-
Noble might be able to comment on the failure. I switched it back to filestore
after it was briefly switched to .filestore. Kevin, IIUC, old name was fine
with that test?
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997334#comment-16997334
]
Ishan Chattopadhyaya commented on SOLR-14096:
-
I'm looking into it, as Noble will only be back online after 6-8 hours.
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14078) DistribPackageStore tries to write to source tree
[
https://issues.apache.org/jira/browse/SOLR-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997332#comment-16997332
]
Dawid Weiss commented on SOLR-14078:
I agree -- no test should be writing to the source tree. It's good that it's
throwing an exception. It's bad it's swallowed.
> DistribPackageStore tries to write to source tree
> -
>
> Key: SOLR-14078
> URL: https://issues.apache.org/jira/browse/SOLR-14078
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Kevin Risden
>Priority: Major
>
> Found while looking at SOLR-14064:
> This doesn't cause the test to fail, but it still is wrong to try to write to
> the source tree. There are junit temp dirs for the running test. It looks
> like SOLR_HOME is potentially set incorrectly?
> FWIW I don't think this is specific to TestRecoveryHdfs. It is just where I
> saw the error.
> {code:java}
> 2> 23044 WARN (SUITE-TestRecoveryHdfs-seed#[AC80F05AAAD3A298]-worker) [
> ] o.a.s.f.DistribPackageStore Unable to create
> [/Users/risdenk/repos/apache/lucene-solr/solr/core/src/test-files/solr/filestore]
> directory in SOLR_HOME
> [/Users/risdenk/repos/apache/lucene-solr/solr/core/src/test-files/solr].
> Features requiring this directory may fail.
> 2> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/Users/risdenk/repos/apache/lucene-solr/solr/core/src/test-files/solr/filestore"
> "write")
> 2> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> 2> java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/Users/risdenk/repos/apache/lucene-solr/solr/core/src/test-files/solr/filestore"
> "write")
> 2> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> 2> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> 2> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> 2> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> 2> at java.io.File.mkdir(File.java:1323) ~[?:?]
> 2> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> 2> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[java/:?]
> 2> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[java/:?]
> 2> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:77)
> ~[java/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997335#comment-16997335
]
Dawid Weiss commented on SOLR-14096:
This test is flaky (that's one thing). The exception is just sprinkles on top.
I don't think it's the cause of the failure. Async logging handlers in log4j
are probably responsible for test failures - on slower machines it'll break.
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997342#comment-16997342
]
Ishan Chattopadhyaya commented on SOLR-14096:
-
bq. This test is flaky (that's one thing). The exception is just sprinkles on
top. I don't think it's the cause of the failure. Async logging handlers in
log4j are probably responsible for test failures - on slower machines it'll
break.
David, you mean LoggingHandlerTest is flaky and unrelated to that security
exception with the filestore? (Sorry, I just saw both thse issues just now,
trying to parse the context.)
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14095) Remove serialization and/or support serialization filtering
[ https://issues.apache.org/jira/browse/SOLR-14095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated SOLR-14095: --- Summary: Remove serialization and/or support serialization filtering (was: Remove serialization or support serialization filtering) > Remove serialization and/or support serialization filtering > --- > > Key: SOLR-14095 > URL: https://issues.apache.org/jira/browse/SOLR-14095 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Robert Muir >Priority: Major > > Removing the use of serialization is greatly preferred. > But if serialization over the wire must really happen, then we must use JDK's > serialization filtering capability to prevent havoc. > https://docs.oracle.com/javase/10/core/serialization-filtering1.htm#JSCOR-GUID-3ECB288D-E5BD-4412-892F-E9BB11D4C98A -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8739) ZSTD Compressor support in Lucene
[ https://issues.apache.org/jira/browse/LUCENE-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997346#comment-16997346 ] Adrien Grand commented on LUCENE-8739: -- As I expected it needs quite a lot of code, compared to the 500 lines we have for LZ4. If you can run benchmarks, I'd be curious, but in general I suspect that the JDK implementation of DEFLATE is more appealing for the kind of trade-offs that zstd provides. > ZSTD Compressor support in Lucene > - > > Key: LUCENE-8739 > URL: https://issues.apache.org/jira/browse/LUCENE-8739 > Project: Lucene - Core > Issue Type: New Feature > Components: core/codecs >Reporter: Sean Torres >Priority: Minor > Labels: features > > ZStandard has a great speed and compression ratio tradeoff. > ZStandard is open source compression from Facebook. > More about ZSTD > [https://github.com/facebook/zstd] > [https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997348#comment-16997348
]
Ishan Chattopadhyaya commented on SOLR-14096:
-
I just beasted this test, and seems to pass. What could be happening is that
TestDistribPackageStore is setting -Denable.packages.true, but not unsetting
it. So, while that doesn't cause failure in TestDistribPackageStore, but is
causing failure in some other test sharing that JVM.
While it is bad that TestDistribPackageStore itself is not failing, I'm going
to unset that property to make sure other tests remain fine.
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997349#comment-16997349
]
Dawid Weiss commented on SOLR-14096:
Yes, I believe these are unrelated things. LoggingHandlerTest sometimes
succeeds even if the exception is still thrown/ reported inside.
{code}
./gradlew -p solr/core test --tests LoggingHandlerTest
-Ptests.useSecurityManager=true -Ptests.verbose=true
{code}
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997352#comment-16997352
]
ASF subversion and git services commented on SOLR-14096:
Commit 461317062c833b0b38e09c41d31bfb0b472df0ec in lucene-solr's branch
refs/heads/branch_8x from Ishan Chattopadhyaya
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4613170 ]
SOLR-14096: Stopping -Denable.packages=true from leaking to other tests
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997351#comment-16997351
]
ASF subversion and git services commented on SOLR-14096:
Commit ee0b066ab63ab21645b91c5a190b157fa56b6ca7 in lucene-solr's branch
refs/heads/master from Ishan Chattopadhyaya
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ee0b066 ]
SOLR-14096: Stopping -Denable.packages=true from leaking to other tests
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997354#comment-16997354
]
Dawid Weiss commented on SOLR-14096:
Ant version:
{code}
ant test -Dtestcase=LoggingHandlerTest
{code}
results in the same:
{code}
[junit4] 2> 2300 WARN
(SUITE-LoggingHandlerTest-seed#[2DC5A72073A6EBD]-worker) [ ]
o.a.s.f.DistribPackageStore Unable to create
[/home/dweiss/work/lucene-solr/solr/core/src/test-files/solr/.filestore]
directory in SOLR_HOME
[/home/dweiss/work/lucene-solr/solr/core/src/test-files/solr]. Features
requiring this directory may fail.
[junit4] 2> => java.security.AccessControlException: access
denied ("java.io.FilePermission"
"/home/dweiss/work/lucene-solr/solr/core/src/test-files/solr/.filestore"
"write")
{code}
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14094) TestSolrCachePerf is flaky
[
https://issues.apache.org/jira/browse/SOLR-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997356#comment-16997356
]
ASF subversion and git services commented on SOLR-14094:
Commit b5a2cfba4fd9d9443eaf07e4c93540aaed1c3634 in lucene-solr's branch
refs/heads/master from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b5a2cfb ]
SOLR-14094: Enable this test again in master.
> TestSolrCachePerf is flaky
> --
>
> Key: SOLR-14094
> URL: https://issues.apache.org/jira/browse/SOLR-14094
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Adrien Grand
>Assignee: Andrzej Bialecki
>Priority: Major
>
> I hit the below failure while building the RC.
> {noformat}
>[junit4] Suite: org.apache.solr.search.TestSolrCachePerf
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Created dataDir:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001/data-dir-97-001
>[junit4] 2> 921086 WARN
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
>[junit4] 2> 921086 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=false
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN,
> clientAuth=NaN)
>[junit4] 2> 921087 INFO
> (SUITE-TestSolrCachePerf-seed#[1D407DF9BFA38129]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
>[junit4] 2> 921088 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testGetPutCompute
>[junit4] 2> 927857 INFO
> (TEST-TestSolrCachePerf.testGetPutCompute-seed#[1D407DF9BFA38129]) [ ]
> o.a.s.SolrTestCaseJ4 ###Ending testGetPutCompute
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCachePerf
> -Dtests.method=testGetPutCompute -Dtests.seed=1D407DF9BFA38129
> -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=AGT
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 6.77s J2 | TestSolrCachePerf.testGetPutCompute <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Cache FastLRUCache:
> compute ratio should be higher or equal to get/put ratio: 0.9917 >= 0.9941
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([1D407DF9BFA38129:9DE8FAF1672A99B6]:0)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.assertGreaterThanOrEqual(TestSolrCachePerf.java:84)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.lambda$testGetPutCompute$0(TestSolrCachePerf.java:75)
>[junit4]>at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>at
> org.apache.solr.search.TestSolrCachePerf.testGetPutCompute(TestSolrCachePerf.java:73)
>[junit4]>at java.lang.Thread.run(Thread.java:748)
>[junit4] 2> NOTE: leaving temporary files on disk at:
> /home/jpountz/.lucene-releases/8.4.0/lucene-solr/solr/build/solr-core/test/J2/temp/solr.search.TestSolrCachePerf_1D407DF9BFA38129-001
>[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {},
> docValues:{}, maxPointsInLeafNode=825, maxMBSortInHeap=7.419122986645501,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@d7facc8),
> locale=ar-QA, timezone=AGT
>[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation
> 1.8.0_151 (64-bit)/cpus=12,threads=1,free=188791576,total=528482304
>[junit4] 2> NOTE: All tests run in this JVM: [TestCryptoKeys,
> AtomicUpdatesTest, TestNestedUpdateProcessor, SolrPluginUtilsTest,
> TestStressCloudBlindAtomicUpdates, DirectoryFactoryTest, TaggerTest,
> IndexSizeTriggerMixedBoundsTest, NumberUtilsTest, TestZkChroot,
> HdfsDirectoryTest, RollingRestartTest, TestSolrCLIRunExample,
> ClusterStateTest, DocValuesMultiTest, TestLeaderElectionWithEmptyReplica,
> TestCustomSort, TestSchemaManager, TestInPlaceUpdatesDistrib,
> TestReloadAndDeleteDocs, HighlighterConfigTest, MBeansHandlerTest,
> PeerSyncWithIndexFingerprintCachingTest, SubstringBytesRefFilterTest,
> TestChildDocTransformer, IndexSizeTriggerTest, HealthCheckHandlerTest,
> SuggesterFSTTest, TestLuceneIndexBa
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997357#comment-16997357
]
Joel Bernstein commented on SOLR-14096:
---
[~dweiss], [~krisden], I just commented on SOLR-14084, about another directory
that is being created. Tests are passing for me that deal with this directory.
Is there action now that needs to be taken about this or this something can be
cleaned up after the 8.4 release?
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14084) Security manager access denied not causing tests to fail
[
https://issues.apache.org/jira/browse/SOLR-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997358#comment-16997358
]
Kevin Risden commented on SOLR-14084:
-
I think userfiles falls into the same category of SOLR-14078 where the Solr
home isn't set correctly by SolrTestCaseJ4.
> Security manager access denied not causing tests to fail
>
>
> Key: SOLR-14084
> URL: https://issues.apache.org/jira/browse/SOLR-14084
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Kevin Risden
>Priority: Major
>
> FYI [~rcmuir] this is from a run where all the tests pass.
> Looking at the output in ./build/solr-core/test/tests-report.txt
> {code:java}
> # grep -F 'access denied' tests-report.txt | cut -d':' -f2 | sort | uniq -c
>1 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/build/solr-core/test/J2/temp/solr.util.TestSolrCLIRunExample_7960AD1EAA781935-001/tempDir-001/failExecuteScript"
> "execute")
> 952 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/filestore"
> "write")
> 30 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/userfiles"
> "write")
> 54 access denied ("java.net.SocketPermission" "--" "resolve")
>2 access denied ("java.net.SocketPermission" "thrasher-T100" "resolve")
>4 127.0.0.1
> {code}
> These didn't cause tests to fail but need to see why these are happening. I
> wouldn't expect we need most of these.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997357#comment-16997357
]
Joel Bernstein edited comment on SOLR-14096 at 12/16/19 2:55 PM:
-
[~dweiss], [~krisden], I just commented on SOLR-14084, about another directory
that is being created. Tests are passing for me that deal with this directory.
Is there action now that needs to be taken about this or is this something can
be cleaned up after the 8.4 release?
was (Author: joel.bernstein):
[~dweiss], [~krisden], I just commented on SOLR-14084, about another directory
that is being created. Tests are passing for me that deal with this directory.
Is there action now that needs to be taken about this or this something can be
cleaned up after the 8.4 release?
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14084) Security manager access denied not causing tests to fail
[
https://issues.apache.org/jira/browse/SOLR-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997360#comment-16997360
]
Joel Bernstein commented on SOLR-14084:
---
Yes, it's the same problem. So the action item here is to change SolrTestCaseJ4
it seems. Is this a blocker for 8.4?
> Security manager access denied not causing tests to fail
>
>
> Key: SOLR-14084
> URL: https://issues.apache.org/jira/browse/SOLR-14084
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Kevin Risden
>Priority: Major
>
> FYI [~rcmuir] this is from a run where all the tests pass.
> Looking at the output in ./build/solr-core/test/tests-report.txt
> {code:java}
> # grep -F 'access denied' tests-report.txt | cut -d':' -f2 | sort | uniq -c
>1 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/build/solr-core/test/J2/temp/solr.util.TestSolrCLIRunExample_7960AD1EAA781935-001/tempDir-001/failExecuteScript"
> "execute")
> 952 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/filestore"
> "write")
> 30 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/userfiles"
> "write")
> 54 access denied ("java.net.SocketPermission" "--" "resolve")
>2 access denied ("java.net.SocketPermission" "thrasher-T100" "resolve")
>4 127.0.0.1
> {code}
> These didn't cause tests to fail but need to see why these are happening. I
> wouldn't expect we need most of these.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14084) Security manager access denied not causing tests to fail
[
https://issues.apache.org/jira/browse/SOLR-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997363#comment-16997363
]
Kevin Risden commented on SOLR-14084:
-
I don't personally think its a blocker for 8.4. It hasn't caused any tests to
fail. It is isolated to tests only.
> Security manager access denied not causing tests to fail
>
>
> Key: SOLR-14084
> URL: https://issues.apache.org/jira/browse/SOLR-14084
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Kevin Risden
>Priority: Major
>
> FYI [~rcmuir] this is from a run where all the tests pass.
> Looking at the output in ./build/solr-core/test/tests-report.txt
> {code:java}
> # grep -F 'access denied' tests-report.txt | cut -d':' -f2 | sort | uniq -c
>1 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/build/solr-core/test/J2/temp/solr.util.TestSolrCLIRunExample_7960AD1EAA781935-001/tempDir-001/failExecuteScript"
> "execute")
> 952 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/filestore"
> "write")
> 30 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/userfiles"
> "write")
> 54 access denied ("java.net.SocketPermission" "--" "resolve")
>2 access denied ("java.net.SocketPermission" "thrasher-T100" "resolve")
>4 127.0.0.1
> {code}
> These didn't cause tests to fail but need to see why these are happening. I
> wouldn't expect we need most of these.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14096) DistribPackageStore attempts to create .filestore in read-only directory
[
https://issues.apache.org/jira/browse/SOLR-14096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997364#comment-16997364
]
Ishan Chattopadhyaya commented on SOLR-14096:
-
[~dweiss], are you using the latest commits? I see that your test reports
".filestore", which has now been changed to "filestore". Can you please do a
"git pull" to see if that still persists?
FYI, I've pushed a fix to stop proliferation of the "enable.packages" property
to other tests. I've pushed to master and branch_8x, and will push to
branch_8_4 after running the full tests.
> DistribPackageStore attempts to create .filestore in read-only directory
>
>
> Key: SOLR-14096
> URL: https://issues.apache.org/jira/browse/SOLR-14096
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Dawid Weiss
>Priority: Blocker
>
> I see this in the logs when running with security manager:
> {code}
> 1645805 WARN (SUITE-LoggingHandlerTest-seed#[DEADBEEF]-worker) [ ]
> o.a.s.f.DistribPackageStore Unable to create
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore]
> directory in SOLR_HOME
> [/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr].
> Features requiring this directory may fail.
> => java.security.AccessControlException: access denied
> ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> java.security.AccessControlException: access denied ("java.io.FilePermission"
> "/home/dweiss/work-ssd/lucene-solr/solr/core/build/resources/test/solr/.filestore"
> "write")
> at
> java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
> ~[?:?]
> at
> java.security.AccessController.checkPermission(AccessController.java:897)
> ~[?:?]
> at java.lang.SecurityManager.checkPermission(SecurityManager.java:322)
> ~[?:?]
> at java.lang.SecurityManager.checkWrite(SecurityManager.java:752) ~[?:?]
> at java.io.File.mkdir(File.java:1323) ~[?:?]
> at java.io.File.mkdirs(File.java:1355) ~[?:?]
> at
> org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:476)
> ~[main/:?]
> at
> org.apache.solr.filestore.DistribPackageStore.(DistribPackageStore.java:65)
> ~[main/:?]
> at
> org.apache.solr.filestore.PackageStoreAPI.(PackageStoreAPI.java:78)
> ~[main/:?]
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:619)
> ~[main/:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14084) Security manager access denied not causing tests to fail
[
https://issues.apache.org/jira/browse/SOLR-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997365#comment-16997365
]
Dawid Weiss commented on SOLR-14084:
I don't think it's a new regression. It's just general flakiness of those tests.
> Security manager access denied not causing tests to fail
>
>
> Key: SOLR-14084
> URL: https://issues.apache.org/jira/browse/SOLR-14084
> Project: Solr
> Issue Type: Test
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
>Reporter: Kevin Risden
>Priority: Major
>
> FYI [~rcmuir] this is from a run where all the tests pass.
> Looking at the output in ./build/solr-core/test/tests-report.txt
> {code:java}
> # grep -F 'access denied' tests-report.txt | cut -d':' -f2 | sort | uniq -c
>1 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/build/solr-core/test/J2/temp/solr.util.TestSolrCLIRunExample_7960AD1EAA781935-001/tempDir-001/failExecuteScript"
> "execute")
> 952 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/filestore"
> "write")
> 30 access denied ("java.io.FilePermission"
> "/Users/krisden/repos/apache/lucene-solr/solr/core/src/test-files/solr/userfiles"
> "write")
> 54 access denied ("java.net.SocketPermission" "--" "resolve")
>2 access denied ("java.net.SocketPermission" "thrasher-T100" "resolve")
>4 127.0.0.1
> {code}
> These didn't cause tests to fail but need to see why these are happening. I
> wouldn't expect we need most of these.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
