[jira] [Commented] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990802#comment-16990802 ] Jan Høydahl commented on SOLR-14013: +1 Perf improvements should be backed by benchmarks. > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13990) Switch out woodstox-core-asl with aalto-xml and upgrade woodstox stax-2 API
[ https://issues.apache.org/jira/browse/SOLR-13990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990818#comment-16990818 ] Anshum Gupta commented on SOLR-13990: - Sorry everyone about being MIA and not tracking this down. Also, thanks for reverting this. I'm still reading all of this conversation here so catching up on what exactly happened. > Switch out woodstox-core-asl with aalto-xml and upgrade woodstox stax-2 API > --- > > Key: SOLR-13990 > URL: https://issues.apache.org/jira/browse/SOLR-13990 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Anshum Gupta >Assignee: Anshum Gupta >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > Switched out woodstox-core-asl with aalto-xml and upgrade woodstax stax-2 > API. > About aalto-xml: > Aalto XML processor is an ultra-high performance next generation Stax XML > processor implementation, implementing both basic Stax API > ({{javax.xml.stream}}) and Stax2 API extension > ({{org.codehaus.woodstox.stax2}}). In addition, it also implements SAX2 API. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13818) Upgrade jackson to 2.10.0
[ https://issues.apache.org/jira/browse/SOLR-13818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990849#comment-16990849 ] Lucene/Solr QA commented on SOLR-13818: --- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color} | {color:red} SOLR-13818 does not apply to master. Rebase required? Wrong Branch? See https://wiki.apache.org/solr/HowToContribute#Creating_the_patch_file for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | SOLR-13818 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12987839/SOLR-13818.patch | | Console output | https://builds.apache.org/job/PreCommit-SOLR-Build/617/console | | Powered by | Apache Yetus 0.7.0 http://yetus.apache.org | This message was automatically generated. > Upgrade jackson to 2.10.0 > - > > Key: SOLR-13818 > URL: https://issues.apache.org/jira/browse/SOLR-13818 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Andras Salamon >Assignee: Kevin Risden >Priority: Major > Fix For: 8.4 > > Attachments: SOLR-13818-01.patch, SOLR-13818.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Jackson 2.10.0 has been released: > [https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10], they > promise to solve the "endless CVE patches" problem. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14033) Fix Hadoop tests with security manager
[ https://issues.apache.org/jira/browse/SOLR-14033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990854#comment-16990854 ] Robert Muir commented on SOLR-14033: +1 > Fix Hadoop tests with security manager > -- > > Key: SOLR-14033 > URL: https://issues.apache.org/jira/browse/SOLR-14033 > Project: Solr > Issue Type: Test > Security Level: Public(Default Security Level. Issues are Public) > Components: Hadoop Integration, hdfs, Tests >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Attachments: SOLR-14033.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SOLR-14000 and SOLR-14002 have been cleaning this up. Seeing some HDFS suite > failures due to missing nulling out static variables and Hadoop being stupid > with the security manager. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14033) Fix Hadoop tests with security manager
[ https://issues.apache.org/jira/browse/SOLR-14033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990856#comment-16990856 ] Robert Muir commented on SOLR-14033: {quote} Alright well Hadoop is definitely stupid when it comes to running w/ the security manager. {quote} One of those times where I really wish emojis worked correctly in JIRA :) > Fix Hadoop tests with security manager > -- > > Key: SOLR-14033 > URL: https://issues.apache.org/jira/browse/SOLR-14033 > Project: Solr > Issue Type: Test > Security Level: Public(Default Security Level. Issues are Public) > Components: Hadoop Integration, hdfs, Tests >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Attachments: SOLR-14033.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SOLR-14000 and SOLR-14002 have been cleaning this up. Seeing some HDFS suite > failures due to missing nulling out static variables and Hadoop being stupid > with the security manager. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990868#comment-16990868 ] Adrien Grand commented on SOLR-14013: - I'm also a bit disappointed that SOLR-12885 changed Field's constructor from String to CharSequence silently. > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on a change in pull request #1049: LUCENE-9074: Slice Allocation Circuit Breakers in IndexSearcher
jpountz commented on a change in pull request #1049: LUCENE-9074: Slice Allocation Circuit Breakers in IndexSearcher URL: https://github.com/apache/lucene-solr/pull/1049#discussion_r355191345 ## File path: lucene/core/src/java/org/apache/lucene/search/SliceExecutionControlPlane.java ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.lucene.search; + +import java.util.concurrent.Executor; + +/** + * Manages the end to end execution and management of slices for an IndexSearcher + * instance. This includes execution on the underlying Executor and managing + * thread allocations to ensure a consistent throughput under varying stress loads. + * This class can block allocation of new slices when executing a query + * Remaining segments will be allocated to a single slice + * NOTE: This can be degrading to query performance since one thread can + * be overloaded with multiple segments hence this is a tradeoff + * between query throughput and latency + * + * A typical implementation of this interface would include the execution framework + * along with a circuit breaker condition which will control whether new threads will be + * created or not. + */ +public interface SliceExecutionControlPlane { + + /** + * Return true if the circuit breaker condition has triggered, + * false otherwise + */ + boolean hasCircuitBreakerTriggered(); Review comment: Maybe we should move the logic of assigning tasks to threads to this class to give it more flexibility instead of just exposing whether the pool is running over capacity. I'm thinking of something that could look like `void invokeAll(Collection tasks)` (similar to ForkJoinPool), which could then decide to merge some runnables together, run some of them on the current thread, etc. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9078) Term vectors options should not be configurable per-doc
[ https://issues.apache.org/jira/browse/LUCENE-9078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990897#comment-16990897 ] Adrien Grand commented on LUCENE-9078: -- +1 I'm not sure this is the only blocker for the Fields removal though, we'd still need a class to hold term vectors for multiple fields. > Term vectors options should not be configurable per-doc > --- > > Key: LUCENE-9078 > URL: https://issues.apache.org/jira/browse/LUCENE-9078 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Bruno Roustant >Priority: Major > > Make term vectors constant across the index. Remove the user ability to > modify the term vector options per doc, IndexWriter allows this. > Once done, consider removing Fields, as the list of fields could be obtained > from FieldInfos. See the discussion in LUCENE-8041. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9078) Term vectors options should not be configurable per-doc
[ https://issues.apache.org/jira/browse/LUCENE-9078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990913#comment-16990913 ] David Smiley commented on LUCENE-9078: -- I think it's unfortunate that TVs are row-stored today. If the query/scenario only needs a sub-set of TV fields then there's plenty of waste in decoding the others. It's trappy to code against the current API wherein you may inadvertently re-load all TVs from disk when getting TVs of other fields without realizing the ramifications. For example in the UnifiedHighlighter there's a little cache mechanism to ensure TVs are only fetched once – see TermVectorReusingLeafReader. I know raising this is a distraction here; I could file an issue. It's tangentially related because the class that replaces Fields for TV use-case would be fundamentally different if we get column-stored TVs. > Term vectors options should not be configurable per-doc > --- > > Key: LUCENE-9078 > URL: https://issues.apache.org/jira/browse/LUCENE-9078 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Bruno Roustant >Priority: Major > > Make term vectors constant across the index. Remove the user ability to > modify the term vector options per doc, IndexWriter allows this. > Once done, consider removing Fields, as the list of fields could be obtained > from FieldInfos. See the discussion in LUCENE-8041. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13987) Admin UI should not rely on javascript eval()
[ https://issues.apache.org/jira/browse/SOLR-13987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990917#comment-16990917 ] David Smiley commented on SOLR-13987: - Thanks so much Kevin! > Admin UI should not rely on javascript eval() > - > > Key: SOLR-13987 > URL: https://issues.apache.org/jira/browse/SOLR-13987 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI >Reporter: Robert Muir >Assignee: Kevin Risden >Priority: Major > Fix For: 8.4 > > Attachments: SOLR-13987.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Followup from SOLR-13982: currently any CSP is weak because it must allow > this eval: means arbitrary javascript can still be executed. > Let's fix the admin UI to not require eval so it can be disabled by the > browser. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13936) Schema/Config endpoints to modify configset with no core/collection
[ https://issues.apache.org/jira/browse/SOLR-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990918#comment-16990918 ] David Smiley commented on SOLR-13936: - I'm super happy to see some configSet manipulation APIs that are decoupled from the collection! +1 to deprecate current configSet manipulation that are coupled to the collection. It's confusing/weird. > Schema/Config endpoints to modify configset with no core/collection > --- > > Key: SOLR-13936 > URL: https://issues.apache.org/jira/browse/SOLR-13936 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: config-api >Reporter: Apoorv Bhawsar >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > All schema/config configurations should work even in cases where a collection > is not associated with them > This jira will involve > 1. Refactoring existing handler/manager to work without {{SolrCore}} > 2. Adding {{/api/cluster}} endpoints to support such modifications > Endpoints - > * {{/api/cluster/configset/\{name}/schema}} > * {{/ap/cluster/configset/\{name}/config}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990919#comment-16990919 ] Ishan Chattopadhyaya commented on SOLR-14013: - bq. Perf improvements should be backed by benchmarks. FYI, https://issues.apache.org/jira/browse/SOLR-12885?focusedCommentId=16709641&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16709641 > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13936) Schema/Config endpoints to modify configset with no core/collection
[ https://issues.apache.org/jira/browse/SOLR-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990920#comment-16990920 ] Ishan Chattopadhyaya commented on SOLR-13936: - bq. Should it? Why? Clearly you're envisioning a usage pattern I'm not thinking of; I'm just trying to understand what that is. My vision is that there should be absolutely no manipulation that can be done to the configset by hand-editing but not possible via APIs. Hence, as I said, "need for hand editing configset should be eliminated." <-- Please keep in mind that I'm not talking about eliminating the ability for handediting etc; just saying the need for doing so should be eliminated. > Schema/Config endpoints to modify configset with no core/collection > --- > > Key: SOLR-13936 > URL: https://issues.apache.org/jira/browse/SOLR-13936 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: config-api >Reporter: Apoorv Bhawsar >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > All schema/config configurations should work even in cases where a collection > is not associated with them > This jira will involve > 1. Refactoring existing handler/manager to work without {{SolrCore}} > 2. Adding {{/api/cluster}} endpoints to support such modifications > Endpoints - > * {{/api/cluster/configset/\{name}/schema}} > * {{/ap/cluster/configset/\{name}/config}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13936) Schema/Config endpoints to modify configset with no core/collection
[ https://issues.apache.org/jira/browse/SOLR-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990921#comment-16990921 ] Ishan Chattopadhyaya commented on SOLR-13936: - bq. This workflow is already confusing enough Agree. I think in this workflow, some warnings etc. should be sent to the user that he/she has actually modified other collections sharing the configset. Also, if not already present, the ref guide should contain a big warning to this effect as well. However, I'm not losing my sleep over it because sharing configsets is a very niche feature and I don't think many users are affected at the moment; and whoever is doing so is already an expert user and understands the consequences. > Schema/Config endpoints to modify configset with no core/collection > --- > > Key: SOLR-13936 > URL: https://issues.apache.org/jira/browse/SOLR-13936 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: config-api >Reporter: Apoorv Bhawsar >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > All schema/config configurations should work even in cases where a collection > is not associated with them > This jira will involve > 1. Refactoring existing handler/manager to work without {{SolrCore}} > 2. Adding {{/api/cluster}} endpoints to support such modifications > Endpoints - > * {{/api/cluster/configset/\{name}/schema}} > * {{/ap/cluster/configset/\{name}/config}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13936) Schema/Config endpoints to modify configset with no core/collection
[ https://issues.apache.org/jira/browse/SOLR-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990924#comment-16990924 ] Ishan Chattopadhyaya commented on SOLR-13936: - bq. Should not the old config-api and schema-api be deprecated too if we move both of them to the cluster level? I think the old collection level config API should be moved to the /api/cluster level, but it should continue to work (maybe using same code paths under the covers). I think regular users will prefer to use it because no Solr user should need to be aware of the concept of the configsets. The default configset was a step in that direction. I'm against deprecating the collection level APIs for doing this. It would be *absolutely terrible* if a user created collection "mycollection", but had to invoke config API on `/api/cluster/configset/mycollection.AUTOCREATED`. (A user would be left wondering, what is a "configset" and why this "AUTOCREATED" suffix etc.). However, sorry [~apoorvprecisely], for taking this issue in a tangent. This issue shouldn't be about what we do with the collection level config APIs. > Schema/Config endpoints to modify configset with no core/collection > --- > > Key: SOLR-13936 > URL: https://issues.apache.org/jira/browse/SOLR-13936 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: config-api >Reporter: Apoorv Bhawsar >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > All schema/config configurations should work even in cases where a collection > is not associated with them > This jira will involve > 1. Refactoring existing handler/manager to work without {{SolrCore}} > 2. Adding {{/api/cluster}} endpoints to support such modifications > Endpoints - > * {{/api/cluster/configset/\{name}/schema}} > * {{/ap/cluster/configset/\{name}/config}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990926#comment-16990926 ] Yonik Seeley commented on SOLR-14013: - Those benchmarks look like they are testing different settings, not a before-vs-after patch scenario? > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990926#comment-16990926 ] Yonik Seeley edited comment on SOLR-14013 at 12/8/19 5:26 PM: -- Those benchmarks look like they are testing different settings, not a before-vs-after patch scenario? Also, a lot of the issues I saw would affect the indexing pipeline performance. was (Author: ysee...@gmail.com): Those benchmarks look like they are testing different settings, not a before-vs-after patch scenario? > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gunasekhardora commented on issue #1048: SOLR-13806: change type of QueryResponse._explainMap
gunasekhardora commented on issue #1048: SOLR-13806: change type of QueryResponse._explainMap URL: https://github.com/apache/lucene-solr/pull/1048#issuecomment-562974842 @sigram Added an unit test for the same. Please verify it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9078) Term vectors options should not be configurable per-doc
[ https://issues.apache.org/jira/browse/LUCENE-9078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990965#comment-16990965 ] Michael McCandless commented on LUCENE-9078: +1 > Term vectors options should not be configurable per-doc > --- > > Key: LUCENE-9078 > URL: https://issues.apache.org/jira/browse/LUCENE-9078 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Bruno Roustant >Priority: Major > > Make term vectors constant across the index. Remove the user ability to > modify the term vector options per doc, IndexWriter allows this. > Once done, consider removing Fields, as the list of fields could be obtained > from FieldInfos. See the discussion in LUCENE-8041. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13904) Make Analytics component aware of timeAllowed
[ https://issues.apache.org/jira/browse/SOLR-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated SOLR-13904: Attachment: SOLR-13904.patch Status: Patch Available (was: Patch Available) > Make Analytics component aware of timeAllowed > - > > Key: SOLR-13904 > URL: https://issues.apache.org/jira/browse/SOLR-13904 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Houston Putman >Priority: Major > Attachments: SOLR-13904.patch, SOLR-13904.patch, SOLR-13904.patch, > SOLR-13904.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13904) Make Analytics component aware of timeAllowed
[ https://issues.apache.org/jira/browse/SOLR-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990982#comment-16990982 ] Mikhail Khludnev commented on SOLR-13904: - Attached the patch. Now test reproduces time exceeded always. I keep repeat=10, to check whether assumption about time exceeding pass every time. After yetus confirm that there's no ignored tests, I remove repeat, and turn assumption to assert. Beside of that I use 524 as http code to signal from AnalyticsComponent to AnalyticsHandler about exceeding. It's the only suitable code I've found. Not sure how it impacts non-cloud, but I propose to keep it out of scope for a while. > Make Analytics component aware of timeAllowed > - > > Key: SOLR-13904 > URL: https://issues.apache.org/jira/browse/SOLR-13904 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Houston Putman >Priority: Major > Attachments: SOLR-13904.patch, SOLR-13904.patch, SOLR-13904.patch, > SOLR-13904.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9086) Benchmark new Graviton2 ARM EC2 instances
Michael McCandless created LUCENE-9086: -- Summary: Benchmark new Graviton2 ARM EC2 instances Key: LUCENE-9086 URL: https://issues.apache.org/jira/browse/LUCENE-9086 Project: Lucene - Core Issue Type: Task Reporter: Michael McCandless Assignee: Michael McCandless At [AWS re:Invent 2019|https://reinvent.awsevents.com/] last week, AWS announced new [EC2 instances based on the Graviton2 ARM processor|https://aws.amazon.com/blogs/aws/coming-soon-graviton2-powered-general-purpose-compute-optimized-memory-optimized-ec2-instances] which apparently can be much faster than the original A1 instances, at least according to internal benchmarks. I've been running Lucene's benchmarks ({{wikimediumall}}, indexing 33.3 M docs and running a diverse and repeatable set of search tasks) on these instances, comparing a {{c4.8xlarge}} (x86-64) instance against the new {{m6g.8xlarge}} (ARM), and I'll summarize the results here. Net/net ARM seems to be faster at raw indexing than x86-64, even though {{m6g.8xlarge}} has only 32 cores versus 36 cores, but a bit slower at merging, while searching seems to be faster for some queries and slower for others. I'll try to get the full results posted soon. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9084) circular synchronization wait (potential deadlock) in AnalyzingInfixSuggester
[ https://issues.apache.org/jira/browse/LUCENE-9084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990983#comment-16990983 ] Paul Ward commented on LUCENE-9084: --- bq. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. bq. This bug was found through code inspection. However, this is a 2-line simple patch and all the existing tests pass. Based on the issue description and the code referenced, it is clear that circular wait (deadlock) can happen. > circular synchronization wait (potential deadlock) in AnalyzingInfixSuggester > - > > Key: LUCENE-9084 > URL: https://issues.apache.org/jira/browse/LUCENE-9084 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Reporter: Paul Ward >Priority: Major > Attachments: LUCENE-9084.patch > > > h1. Github Pull Request > I created a pull request on github for this: > [https://github.com/apache/lucene-solr/pull/1064] > h1. Bug Description > Detailed code (snippets and links) are in the sections after this overview > (section **Detailed Code** and **This Patch's Code**). > Method {{ensureOpen()}} is {{synchronized}} (acquires {{this}}) and its body > contains a {{synchronized (searcherMgrLock)}} block (i.e., then acquires > {{searcherMgrLock}}). > Method {{ensureOpen()}} is called two times from public methods {{add()}} and > {{update()}}. > A thread calling public methods {{add()}} or {{update()}} will acquire locks > in order: > {code:java} > this -> searcherMgrLock > {code} > Public method {{build()}} has a {{synchronized (searcherMgrLock)}} block in > which it calls method {{add()}}. Method {{add()}}, as described above, calls > method {{ensureOpen()}}. > Therefore, a thread calling public method {{build()}} will acquire locks in > order: > {code:java} > searcherMgrLock -> this -> searcherMgrLock > {code} > 2 threads can acquire locks in different order which may cause a circular > wait (deadlock). > I do not know which threads call these methods, but there is a lot of > synchronization in these methods and in this file, so I think these methods > must be called concurrently. > One thread can acquire: > {{this -> searcherMgrLock}} (the first order above) > and the other thread can acquire: > {{searcherMgrLock -> this}} (the second order above). > Note how the above 2 orders lead to a circular wait. > h1. Detailed Code > Method {{ensureOpen()}} is {{synchronized}} and its body contains a > {{synchronized (searcherMgrLock)}}: > {code:java} > private synchronized void ensureOpen() throws IOException { <<< see > the synchronized keyword > if (writer == null) { > if (DirectoryReader.indexExists(dir)) { > // Already built; open it: > writer = new IndexWriter(dir, getIndexWriterConfig(getGramAnalyzer(), > IndexWriterConfig.OpenMode.APPEND)); > } else { > writer = new IndexWriter(dir, getIndexWriterConfig(getGramAnalyzer(), > IndexWriterConfig.OpenMode.CREATE)); > } > synchronized (searcherMgrLock) { <<< > {code} > [https://github.com/apache/lucene-solr/blob/master/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java#L371-L379] > Method {{ensureOpen()}} is called two times from public methods {{add()}} and > {{update()}}: > {code:java} > public void add(BytesRef text, Set contexts, long weight, > BytesRef payload) throws IOException { > ensureOpen(); << > {code} > [https://github.com/apache/lucene-solr/blob/master/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java#L394-L395] > {code:java} > public void update(BytesRef text, Set contexts, long weight, > BytesRef payload) throws IOException { > ensureOpen(); > {code} > [https://github.com/apache/lucene-solr/blob/master/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java#L406-L407] > Public method {{build()}} has a {{synchronized (searcherMgrLock)}} block in > which it calls method {{add()}}: > {code:java} > @Override > public void build(InputIterator iter) throws IOException { > > synchronized (searcherMgrLock) { > if (searcherMgr != null) { > searcherMgr.close(); > searcherMgr = null; > } > if (writer != null) { > writer.close(); > writer = null; > } > boolean success = false; > try { > // First pass: build a temporary normal Lucene index, > // just indexing the suggestions as they iterate: >
[jira] [Updated] (LUCENE-9077) Gradle build
[ https://issues.apache.org/jira/browse/LUCENE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-9077: Description: This task focuses on providing gradle-based build equivalent for Lucene and Solr (on master branch). See notes below on why this respin is needed. The code lives on *gradle-master* branch. It is kept with sync with *master*. Try running the following to see an overview of helper guides concerning typical workflow, testing and ant-migration helpers: gradlew :help A list of items that needs to be added or requires work. If you'd like to work on any of these, please add your name to the list. Once you have a patch/ pull request let me (dweiss) know - I'll try to coordinate the merges. * (/) Apply forbiddenAPIs * (/) Generate hardware-aware gradle defaults for parallelism (count of workers and test JVMs). * (/) Fail the build if --tests filter is applied and no tests execute during the entire build (this allows for an empty set of filtered tests at single project level). * (/) Port other settings and randomizations from common-build.xml * (/) Configure security policy/ sandboxing for tests. * (/) test's console output on -Ptests.verbose=true * (/) add a :helpDeps explanation to how the dependency system works (palantir plugin, lockfile) and how to retrieve structured information about current dependencies of a given module (in a tree-like output). * test's console output on failure? * jar checksums, jar checksum computation and validation. This should be done without intermediate folders (directly on dependency sets). * identify and list precommit tasks so that they can be ported one by one. (Mark's branch has some of this stuff already implemented) Of lesser importance: * add rendering of javadocs (gradlew javadoc) and attaching them to maven publications. * Repro-line for failed tests/ runs. * Add test 'beasting' (rerunning the same suite multiple times). I'm afraid it'll be difficult to run it sensibly because gradle doesn't offer cwd separation for the forked test runners. * if you diff solr packaged distribution against ant-created distribution there are minor differences in library versions and some JARs are excluded/ moved around. I didn't try to force these as everything seems to work (tests, etc.) – perhaps these differences should be fixed in the ant build instead. * identify and port any other "check" utilities that may be called from ant. (Mark's branch has some of this stuff already implemented) * [EOE] identify and port various "regenerate" tasks from ant builds (javacc, precompiled automata, etc.) * fill in POM details in gradle/defaults-maven.gradle so that they reflect the previous content better (dependencies aside). * Add any IDE integration layers that should be added (I use IntelliJ and it imports the project out of the box, without the need for any special tuning). * *Clean up dependencies, especially for Solr*: any \{ transitive = false } should just explicitly exclude whatever they don't need (and their dependencies currently declared explicitly should be folded). Figure out which scope to import a dependency to. * Add Solr packaging for docs/* (see TODO in packaging/build.gradle; currently XSLT...) * I didn't bother adding Solr dist/test-framework to packaging (who'd use it from a binary distribution? *{color:#ff}Note:{color}* this builds on the work done by Mark Miller and Cao Mạnh Đạt but also applies lessons learned from those two efforts: * *Do not try to do too many things at once*. If we deviate too far from master, the branch will be hard to merge. * *Do everything in baby-steps* and add small, independent build fragments replacing the old ant infrastructure. * *Try to engage people to run, test and contribute early*. It can't be a one-man effort. The more people understand and can contribute to the build, the more healthy it will be. was: This task focuses on providing gradle-based build equivalent for Lucene and Solr (on master branch). See notes below on why this respin is needed. The code lives on *gradle-master* branch. It is kept with sync with *master*. Try running the following to see an overview of helper guides concerning typical workflow, testing and ant-migration helpers: gradlew :help A list of items that needs to be added or requires work. If you'd like to work on any of these, please add your name to the list. Once you have a patch/ pull request let me (dweiss) know - I'll try to coordinate the merges. * (/) Apply forbiddenAPIs * (/) Generate hardware-aware gradle defaults for parallelism (count of workers and test JVMs). * (/) Fail the build if --tests filter is applied and no tests execute during the entire build (this allows for an empty set of filtered tests at single project level). * (/) Port other settings and randomizations from common-build.xml * (/) Conf
[jira] [Commented] (SOLR-13936) Schema/Config endpoints to modify configset with no core/collection
[ https://issues.apache.org/jira/browse/SOLR-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990999#comment-16990999 ] Noble Paul commented on SOLR-13936: --- bq. I think the old collection level config API should be moved to the /api/cluster level, I'm not sure if it is intuitive. As Ishan said, it is not intuitive for most users if we eliminate the config APIs from collection level. Ideally, a casual user should do the following # create a collection (without specifying a configset). Solr automatically creates a configset # add schema fields or manipulate config through {{/config}} and {{/schema}} This class of users will not or should not know about a configset There are extra functionalities for READ apis done through collection as a live collection is available. For instance, if a plugin is loaded from a package, the API tells you if it is indeed loaded from a package > Schema/Config endpoints to modify configset with no core/collection > --- > > Key: SOLR-13936 > URL: https://issues.apache.org/jira/browse/SOLR-13936 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: config-api >Reporter: Apoorv Bhawsar >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > All schema/config configurations should work even in cases where a collection > is not associated with them > This jira will involve > 1. Refactoring existing handler/manager to work without {{SolrCore}} > 2. Adding {{/api/cluster}} endpoints to support such modifications > Endpoints - > * {{/api/cluster/configset/\{name}/schema}} > * {{/ap/cluster/configset/\{name}/config}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13966) LatLonDocValuesField returns unparseable data
[ https://issues.apache.org/jira/browse/SOLR-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991006#comment-16991006 ] Thomas Wöckinger commented on SOLR-13966: - [~dsmiley] Anything new on this one? > LatLonDocValuesField returns unparseable data > - > > Key: SOLR-13966 > URL: https://issues.apache.org/jira/browse/SOLR-13966 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Schema and Analysis, UpdateRequestProcessors >Affects Versions: master (9.0), 8.3, 8.3.1 >Reporter: Thomas Wöckinger >Assignee: David Smiley >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > LatLonDocValuesField is used by LatLonPointSpatialField field type to store > doc values. It does this by encoding the two double values into a long. If > the field is part of a document which takes place within a atomic update > operation RealTimeGetComponent is used to load the field values from the > index. > If the request is a nested request (line number 654: isNestedRequest is set > by DistributedUpdateProcessor to Resolution.ROOT_WITH_CHILDREN) all field > values from the schema are copied into a SolrInputDocument (line 678). > The copy is implemented by toSolrInputDocument method (line 728 of > RealTimeGetComponent). The method retrieves the FieldType of the SchemaField > and calls the toObject method (line 740). In case of LatLonPointSpatialField > type, when docValues is set to true and stored is set to false, the fieldData > is stored by a LatLonDocValuesField instance. > The toObject method is implemented by the base class FieldType toExternal > with the LatLonDocValuesField instance (line 373). In line 359 the method > stringValue of the Field is called. The default implementation (line 259) > checks if the fieldsData is either CharSequence or a Number and calls > fieldsData.toString. > Because fieldsData is of type Long in LatLonDocValuesField class the value is > not decoded correctly. > From my opinion LatLonDocValuesField must implement (override) the > stringValue method and return the decoded value. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] stevetemple opened a new pull request #1069: Include the closing tags in documentation for lucene.search.join
stevetemple opened a new pull request #1069: Include the closing tags in documentation for lucene.search.join URL: https://github.com/apache/lucene-solr/pull/1069 # Description Include closing tags in package-info.java for Join # Solution Please provide a short description of the approach taken to implement your solution. # Tests Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I am authorized to contribute this code to the ASF and have removed any code I do not have a license to distribute. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [ ] I have developed this patch against the `master` branch. - [ ] I have run `ant precommit` and the appropriate test suite. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13936) Schema/Config endpoints to modify configset with no core/collection
[ https://issues.apache.org/jira/browse/SOLR-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991010#comment-16991010 ] Jan Høydahl commented on SOLR-13936: I'm thinking perhaps this issue right here is a good candidate as SIP-1 over at https://cwiki.apache.org/confluence/display/SOLR/Solr+Improvement+Proposals ? Then we could end up with a clear design proposal for fixing this mess, what should be deprecated and what APIs to support. I'm not happy with just simply adding more API surface without taking anything away. If I could choose I'd go in the direction of treating configsets as config *templates*, and once you create a collection using a certain configset (template), that config is *copied* into a collection specific config with the same name as the collection, which is edited through *today's* config API. That's more or less how it works in master/slave mode already. That would maintain todays flexibility ut remove much of the confusion. In such a world it could make sense to add a configset edit API and keep today's collection level APIs. Collection specific configs would live in zk {{/configs}} with same name as collection, while configsets perhaps should get their own znode? Such a breaking change should of course wait until a major release. > Schema/Config endpoints to modify configset with no core/collection > --- > > Key: SOLR-13936 > URL: https://issues.apache.org/jira/browse/SOLR-13936 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: config-api >Reporter: Apoorv Bhawsar >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > All schema/config configurations should work even in cases where a collection > is not associated with them > This jira will involve > 1. Refactoring existing handler/manager to work without {{SolrCore}} > 2. Adding {{/api/cluster}} endpoints to support such modifications > Endpoints - > * {{/api/cluster/configset/\{name}/schema}} > * {{/ap/cluster/configset/\{name}/config}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991014#comment-16991014 ] Yonik Seeley commented on SOLR-14013: - I worked up a quick-n-dirty patch to disable the charseq optimization stuff to test my hypothesis on slower indexing speed: {code} git diff diff --git a/solr/core/src/java/org/apache/solr/handler/component/HttpShardHandler.java b/solr/core/src/java/org/apache/solr/handler/component/HttpShardHandler.java index 69da3948fe9..620fffb1303 100644 --- a/solr/core/src/java/org/apache/solr/handler/component/HttpShardHandler.java +++ b/solr/core/src/java/org/apache/solr/handler/component/HttpShardHandler.java @@ -146,7 +146,7 @@ public class HttpShardHandler extends ShardHandler { private static final BinaryResponseParser READ_STR_AS_CHARSEQ_PARSER = new BinaryResponseParser() { @Override protected JavaBinCodec createCodec() { - return new JavaBinCodec(null, stringCache).setReadStringAsCharSeq(true); + return new JavaBinCodec(null, stringCache).setReadStringAsCharSeq(false); } }; diff --git a/solr/core/src/java/org/apache/solr/response/DocsStreamer.java b/solr/core/src/java/org/apache/solr/response/DocsStreamer.java index 3d1976e143c..056dc08d963 100644 --- a/solr/core/src/java/org/apache/solr/response/DocsStreamer.java +++ b/solr/core/src/java/org/apache/solr/response/DocsStreamer.java @@ -148,9 +148,7 @@ public class DocsStreamer implements Iterator { // because that doesn't include extra fields needed by transformers final Set fieldNamesNeeded = fields.getLuceneFieldNames(); -final SolrDocument out = ResultContext.READASBYTES.get() == null ? -new SolrDocument() : -new BinaryResponseWriter.MaskCharSeqSolrDocument(); +final SolrDocument out = new SolrDocument(); // NOTE: it would be tempting to try and optimize this to loop over fieldNamesNeeded // when it's smaller then the IndexableField[] in the Document -- but that's actually *less* effecient diff --git a/solr/solrj/src/java/org/apache/solr/common/util/ByteArrayUtf8CharSequence.java b/solr/solrj/src/java/org/apache/solr/common/util/ByteArrayUtf8CharSequence.java index 7a4abe2c303..53cfbee320f 100644 --- a/solr/solrj/src/java/org/apache/solr/common/util/ByteArrayUtf8CharSequence.java +++ b/solr/solrj/src/java/org/apache/solr/common/util/ByteArrayUtf8CharSequence.java @@ -209,8 +209,11 @@ public class ByteArrayUtf8CharSequence implements Utf8CharSequence { } return vals; } - public static Object convertCharSeq(Object o) { +return o; // nocommit + } + + public static Object _convertCharSeq(Object o) { if (o == null) return null; if (o instanceof Utf8CharSequence) return ((Utf8CharSequence) o).toString(); if (o instanceof Collection) return convertCharSeq((Collection) o); {code} I also hacked up the unit test I used to find the N^2 issue... it's obviously not good for benchmarking (being a unit test, etc), but good enough to detect anything major. I tested with a single value per string field (and many fields per doc).. it would be worse for multiple values per field. Results: = master, single valued string fields [junit4] 2> INDEX TIME=10293 [junit4] 2> QUERY TIME=891 xml [junit4] 2> QUERY TIME=415 javabin [junit4] 2> QUERY TIME=600 json [junit4] 2> INDEX TIME=10313 [junit4] 2> QUERY TIME=872 xml [junit4] 2> QUERY TIME=389 javabin [junit4] 2> QUERY TIME=579 json [junit4] 2> INDEX TIME=10307 [junit4] 2> QUERY TIME=858 xml [junit4] 2> QUERY TIME=410 javabin [junit4] 2> QUERY TIME=570 json [junit4] 2> INDEX TIME=10318 [junit4] 2> QUERY TIME=915 xml [junit4] 2> QUERY TIME=382 javabin [junit4] 2> QUERY TIME=600 json [junit4] 2> INDEX TIME=10579 [junit4] 2> QUERY TIME=843 xml [junit4] 2> QUERY TIME=386 javabin [junit4] 2> QUERY TIME=570 json = patch disabling charseq stuff, single valued string fields [junit4] 2> INDEX TIME=8547 [junit4] 2> QUERY TIME=881 xml [junit4] 2> QUERY TIME=396 javabin [junit4] 2> QUERY TIME=576 json [junit4] 2> INDEX TIME=9428 [junit4] 2> QUERY TIME=821 xml [junit4] 2> QUERY TIME=374 javabin [junit4] 2> QUERY TIME=543 json [junit4] 2> INDEX TIME=9181 [junit4] 2> QUERY TIME=812 xml [junit4] 2> QUERY TIME=382 javabin [junit4] 2> QUERY TIME=533 json [junit4] 2> INDEX TIME=9455 [junit4] 2> QUERY TIME=863 xml [junit4] 2> QUERY TIME=395 javabin [junit4] 2> QUERY TIME=613 json [junit4] 2> INDEX TIME=9530 [junit4] 2> QUERY TIME=863 xml [junit4] 2> QUERY TIME=385 javabin [junit4] 2> QUERY TIME=559 json So the charseq stuff (or rather probably the extra work to auto-convert-to-string) did cause slower indexing speed. There is enough noise that I don't think one can draw any conclusions about query speed.
[jira] [Updated] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-14013: Status: Open (was: Open) > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-14013: Attachment: TestQuerySpeed.java Status: Open (was: Open) Here's the "benchmark" program I used. I'm not recommending this... a real benchmark program shouldn't be run as a unit test. But if one is looking to debug performance issues, then running it as a unit test can be useful. > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: TestQuerySpeed.java, test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13936) Schema/Config endpoints to modify configset with no core/collection
[ https://issues.apache.org/jira/browse/SOLR-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991027#comment-16991027 ] Noble Paul commented on SOLR-13936: --- [~janhoy] There are 2 different types of users. * Users who just create collections without bothering about the configset * Users who have multiple collections with shared configset. These are power users who know very well about a configset Both are valid usecases. I do not wish to sacrifice one over the other. bq. I'm not happy with just simply adding more API surface without taking anything away. The APIs are same. But they apply differently. If you look at the PR, both the features are using the same code. Yes, there is a lot of cruft I would like to get rid of. I disagree with the idea that we can't add something unless we remove something. We should go thorough Solr APIs through a fine toothed comb and ask these questions * Is this useful? Is it worth keeping it around? * Is it logical? does it need change > Schema/Config endpoints to modify configset with no core/collection > --- > > Key: SOLR-13936 > URL: https://issues.apache.org/jira/browse/SOLR-13936 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: config-api >Reporter: Apoorv Bhawsar >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > All schema/config configurations should work even in cases where a collection > is not associated with them > This jira will involve > 1. Refactoring existing handler/manager to work without {{SolrCore}} > 2. Adding {{/api/cluster}} endpoints to support such modifications > Endpoints - > * {{/api/cluster/configset/\{name}/schema}} > * {{/ap/cluster/configset/\{name}/config}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-14013: -- Attachment: SOLR-14013.patch > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: SOLR-14013.patch, TestQuerySpeed.java, test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991031#comment-16991031 ] Noble Paul commented on SOLR-14013: --- I have submitted a bug fix for the perf degradation There are 3 places where the optimizations are done # Writing out responses # Indexing # deserializing responses during inter-node communications The changes are minimal for #1 and #2 and #3 are complex I would recommend reverting #2 and #3 and let #1 continue to be there with the bug fix (and more auditing) I have just submitted. I'll go with the decision of the community and do the necessary work > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: SOLR-14013.patch, TestQuerySpeed.java, test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991041#comment-16991041 ] Yonik Seeley commented on SOLR-14013: - Please don't commit that huge JSON file... a doc matching that can be created with a few lines of java in the test. I'm not sure the test belongs as a unit test anyway as it's more of a performance benchmark, but I don't care much either way as long as it's quick to run. In general, what I think should be done is: - the auto-convert changes should be removed (in SolrDocument, SolrInputField, MaskCharSeqSolrDocument) - if there are parts of the code base that can't handle CharSequence, then disable reading Strings as CharSequence and look at if those other pieces of code can be fixed to handle CharSequence. > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: SOLR-14013.patch, TestQuerySpeed.java, test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14013) javabin performance regressions
[ https://issues.apache.org/jira/browse/SOLR-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991031#comment-16991031 ] Noble Paul edited comment on SOLR-14013 at 12/9/19 2:50 AM: I have submitted a bug fix for the perf degradation There are 3 places where the optimizations are done # Writing out responses # Indexing # deserializing responses during inter-node communications The changes are minimal for #1 and #2 and #3 are complex I would recommend reverting #2 and #3 and let #1 continue to be there with the bug fix (and more auditing) I have just submitted. I'll go with the decision of the community and do the necessary work was (Author: noble.paul): I have submitted a bug fix for the perf degradation There are 3 places where the optimizations are done # Writing out responses # Indexing # deserializing responses during inter-node communications The changes are minimal for #1 and #2 and #3 are complex I would recommend reverting #2 and #3 and let #1 continue to be there with the bug fix (and more auditing) I have just submitted. I'll go with the decision of the community and do the necessary work > javabin performance regressions > --- > > Key: SOLR-14013 > URL: https://issues.apache.org/jira/browse/SOLR-14013 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 7.7 >Reporter: Yonik Seeley >Assignee: Yonik Seeley >Priority: Major > Attachments: SOLR-14013.patch, TestQuerySpeed.java, test.json > > > As noted by [~rrockenbaugh] in SOLR-13963, javabin also recently became > orders of magnitude slower in certain cases since v7.7. The cases identified > so far include large numbers of values in a field. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13904) Make Analytics component aware of timeAllowed
[ https://issues.apache.org/jira/browse/SOLR-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991129#comment-16991129 ] Lucene/Solr QA commented on SOLR-13904: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Release audit (RAT) {color} | {color:green} 3m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Check forbidden APIs {color} | {color:green} 3m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Validate source patterns {color} | {color:green} 3m 18s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m 45s{color} | {color:green} analytics in the patch passed. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 25m 42s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | SOLR-13904 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12988272/SOLR-13904.patch | | Optional Tests | compile javac unit ratsources checkforbiddenapis validatesourcepatterns | | uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | ant | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh | | git revision | master / df508ff | | ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 | | Default Java | LTS | | Test Results | https://builds.apache.org/job/PreCommit-SOLR-Build/619/testReport/ | | modules | C: solr/contrib/analytics U: solr/contrib/analytics | | Console output | https://builds.apache.org/job/PreCommit-SOLR-Build/619/console | | Powered by | Apache Yetus 0.7.0 http://yetus.apache.org | This message was automatically generated. > Make Analytics component aware of timeAllowed > - > > Key: SOLR-13904 > URL: https://issues.apache.org/jira/browse/SOLR-13904 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mikhail Khludnev >Assignee: Houston Putman >Priority: Major > Attachments: SOLR-13904.patch, SOLR-13904.patch, SOLR-13904.patch, > SOLR-13904.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1035: SOLR-13966: Decode lat lon bevor used by RealTimeGetComponent
dsmiley commented on a change in pull request #1035: SOLR-13966: Decode lat lon bevor used by RealTimeGetComponent URL: https://github.com/apache/lucene-solr/pull/1035#discussion_r355268146 ## File path: solr/core/src/test/org/apache/solr/update/processor/NestedAtomicUpdateTest.java ## @@ -709,6 +709,67 @@ private void testBlockAtomicSetToNullOrEmpty(boolean empty) throws Exception { "/response/docs/[0]/cat_ss/[1]==\"ccc\""); } + @Test + public void testBlockAtomicUpdateWithoutTouchingLatLonPointSpartialField() throws Exception { Review comment: This test appears to be a copy-paste from the one above, but simply adding a latLon field to it. Nothing more. Right? Then lets not approach it this way; we only need one method here. Comments inside can mention that latLon is here due to this issue (reference JIRA issue). If you can make this change quickly then it can get into 8.4. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13966) LatLonDocValuesField returns unparseable data
[ https://issues.apache.org/jira/browse/SOLR-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991158#comment-16991158 ] David Smiley commented on SOLR-13966: - Definitely don't have LLDVF override stringValue; I'm not surprised that didn't work. Your chosen path of checking instanceof in RTG is a code smell; I didn't love it in SolrDocumentFetcher either. Enhancing the appropriate FieldType is the best place. In LLPSF override Solr's FieldType.toExternal. If f.stringValue() is non-null then return that, else assume numeric and call decodeDocValueToString. In my code review on GH I have one comment there too concerning the test. > LatLonDocValuesField returns unparseable data > - > > Key: SOLR-13966 > URL: https://issues.apache.org/jira/browse/SOLR-13966 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Schema and Analysis, UpdateRequestProcessors >Affects Versions: master (9.0), 8.3, 8.3.1 >Reporter: Thomas Wöckinger >Assignee: David Smiley >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > LatLonDocValuesField is used by LatLonPointSpatialField field type to store > doc values. It does this by encoding the two double values into a long. If > the field is part of a document which takes place within a atomic update > operation RealTimeGetComponent is used to load the field values from the > index. > If the request is a nested request (line number 654: isNestedRequest is set > by DistributedUpdateProcessor to Resolution.ROOT_WITH_CHILDREN) all field > values from the schema are copied into a SolrInputDocument (line 678). > The copy is implemented by toSolrInputDocument method (line 728 of > RealTimeGetComponent). The method retrieves the FieldType of the SchemaField > and calls the toObject method (line 740). In case of LatLonPointSpatialField > type, when docValues is set to true and stored is set to false, the fieldData > is stored by a LatLonDocValuesField instance. > The toObject method is implemented by the base class FieldType toExternal > with the LatLonDocValuesField instance (line 373). In line 359 the method > stringValue of the Field is called. The default implementation (line 259) > checks if the fieldsData is either CharSequence or a Number and calls > fieldsData.toString. > Because fieldsData is of type Long in LatLonDocValuesField class the value is > not decoded correctly. > From my opinion LatLonDocValuesField must implement (override) the > stringValue method and return the decoded value. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14030) fix 2 javac warnings: [dep-ann] deprecated item is not annotated with @Deprecated
[ https://issues.apache.org/jira/browse/SOLR-14030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991175#comment-16991175 ] David Smiley commented on SOLR-14030: - BTW I think you should feel free to commit right away without a review and without posting a patch. The description is sufficient and it's inherently trivial in nature. Even a JIRA issue is not needed but I see you are using it as one item in a larger issue. Thanks for cleaning up the noise. > fix 2 javac warnings: [dep-ann] deprecated item is not annotated with > @Deprecated > - > > Key: SOLR-14030 > URL: https://issues.apache.org/jira/browse/SOLR-14030 > Project: Solr > Issue Type: Sub-task >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-14030.patch > > > {code} > [javac] > /Users/cpoerschke/git/solr/solrj/src/java/org/apache/solr/client/solrj/request/UpdateRequest.java:63: > warning: [dep-ann] deprecated item is not annotated with @Deprecated > [javac] public static final String MIN_REPFACT = "min_rf"; > [javac] ^ > [javac] > /Users/cpoerschke/git/solr/solrj/src/java/org/apache/solr/common/params/CommonParams.java:276: > warning: [dep-ann] deprecated item is not annotated with @Deprecated > [javac] String PREFER_LOCAL_SHARDS = "preferLocalShards"; > [javac] ^ > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz merged pull request #1067: LUCENE-9085: fix assertion in CharacterUtils
jpountz merged pull request #1067: LUCENE-9085: fix assertion in CharacterUtils URL: https://github.com/apache/lucene-solr/pull/1067 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9085) Fix the assertion in CharacterUtils
[ https://issues.apache.org/jira/browse/LUCENE-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991207#comment-16991207 ] ASF subversion and git services commented on LUCENE-9085: - Commit 79007128e3f1d5dfe51a9575473a4ab757c557f9 in lucene-solr's branch refs/heads/master from Daiki Tsuzuku [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7900712 ] LUCENE-9085: Fix assertion in CharacterUtils (#1067) > Fix the assertion in CharacterUtils > --- > > Key: LUCENE-9085 > URL: https://issues.apache.org/jira/browse/LUCENE-9085 > Project: Lucene - Core > Issue Type: Bug >Reporter: Daiki Tsuzuku >Priority: Major > Labels: easyfix > Attachments: LUCENE-9085.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Assertions in CharacterUtils's toLowerCase and toUpperCase is not correct. > Current assertion only allow offsets to be 0, but that is not consistent with > javadocs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org