[jira] [Commented] (LUCENE-9165) change generate-defaults.gradle not to cap testsJvms at 4
[
https://issues.apache.org/jira/browse/LUCENE-9165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022754#comment-17022754
]
Dawid Weiss commented on LUCENE-9165:
-
LGTM, thanks for testing, Robert. I remember ES had some fancy way of telling
hyperthreaded cores from physical CPUs - don't know if it's worth complicating
the script though (those who have such fancy machines can always override in
~/gradle.properties?).
https://github.com/elastic/elasticsearch/blob/d7ce4f203d0685f83e3480a63f264f73f0171d4e/buildSrc/src/main/java/org/elasticsearch/gradle/info/GlobalBuildInfoPlugin.java#L188-L228
> change generate-defaults.gradle not to cap testsJvms at 4
> -
>
> Key: LUCENE-9165
> URL: https://issues.apache.org/jira/browse/LUCENE-9165
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
> Attachments: LUCENE-9165.patch
>
>
> {code}
> def cpus = Runtime.runtime.availableProcessors()
> def testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 4))
> {code}
> Dividing by 2 is still good (hyperthreading is still a thing), but the
> artificial cap of 4 is a bottleneck if the hardware has more available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
mkhludnev commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370547212
##
File path:
solr/core/src/test/org/apache/solr/search/TestJoinQueryPerformance.java
##
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.BufferedReader;
+import java.io.FileReader;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.UUID;
+
+import com.google.common.collect.Lists;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.SolrQuery;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.HttpSolrClient;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.common.SolrInputDocument;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+
+public class TestJoinQueryPerformance extends SolrTestCaseJ4 {
Review comment:
How long does it take to run? Souldn't it be @Slow?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
mkhludnev commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370548790
##
File path:
solr/core/src/test/org/apache/solr/search/TestJoinQueryPerformance.java
##
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.BufferedReader;
+import java.io.FileReader;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.UUID;
+
+import com.google.common.collect.Lists;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.SolrQuery;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.HttpSolrClient;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.common.SolrInputDocument;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+
+public class TestJoinQueryPerformance extends SolrTestCaseJ4 {
+ // Dictionary used to load String data
+ private static final String DICT_PATH = "/usr/share/dict/words";
+ private static final int NUM_DICT_TERMS = 235886;
+ private static final String[] LOADED_DICTIONARY = new String[NUM_DICT_TERMS];
+
+ // Performance run parameters: Indexing
+ private static final String FROM_COLLECTION_NAME = "user_acls";
+ private static final int NUM_FROM_DOCS = 5050; // 1 + 2 + 3 + 4 + ... + 100
+ private static final String TO_COLLECTION_NAME = "products";
+ private static final int NUM_TO_DOCS = 50;
+ private static final int PERMISSION_CARDINALITY = 5; // 50K unique
groups/roles/whatever
+ private static int BATCH_SIZE = 500;
+ private static int NUM_COMMITS = 500;
+ private static final int VAL_MAX = 1000;
+ private static final int USER_MAX = 100;
+
+ private static String COLLECTION_NAME= "foo";
+
+ /*
+ * As I start out here, I think I'll want a few different axes.
+ * - "from" collection matches (with "to" matches held constant)
+ * - "to" collection matches (with "from" matches held constant)
+ *
+ * So I think I should index a finite number of docs
+ */
+
+ @BeforeClass
+ public static void setUpCluster() throws Exception {
+loadDictionary();
+//loadCollectionData(DType.USER);
Review comment:
Shouldn't it load at least anything?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
mkhludnev commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370547761
##
File path:
solr/core/src/test/org/apache/solr/search/TestJoinQueryPerformance.java
##
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.BufferedReader;
+import java.io.FileReader;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.UUID;
+
+import com.google.common.collect.Lists;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.SolrQuery;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.HttpSolrClient;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.common.SolrInputDocument;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+
+public class TestJoinQueryPerformance extends SolrTestCaseJ4 {
+ // Dictionary used to load String data
+ private static final String DICT_PATH = "/usr/share/dict/words";
+ private static final int NUM_DICT_TERMS = 235886;
+ private static final String[] LOADED_DICTIONARY = new String[NUM_DICT_TERMS];
+
+ // Performance run parameters: Indexing
+ private static final String FROM_COLLECTION_NAME = "user_acls";
+ private static final int NUM_FROM_DOCS = 5050; // 1 + 2 + 3 + 4 + ... + 100
+ private static final String TO_COLLECTION_NAME = "products";
+ private static final int NUM_TO_DOCS = 50;
+ private static final int PERMISSION_CARDINALITY = 5; // 50K unique
groups/roles/whatever
+ private static int BATCH_SIZE = 500;
+ private static int NUM_COMMITS = 500;
+ private static final int VAL_MAX = 1000;
+ private static final int USER_MAX = 100;
+
+ private static String COLLECTION_NAME= "foo";
+
+ /*
+ * As I start out here, I think I'll want a few different axes.
+ * - "from" collection matches (with "to" matches held constant)
+ * - "to" collection matches (with "from" matches held constant)
+ *
+ * So I think I should index a finite number of docs
+ */
+
+ @BeforeClass
+ public static void setUpCluster() throws Exception {
+loadDictionary();
+//loadCollectionData(DType.USER);
+//loadCollectionData(DType.DATA);
+ }
+
+ private static void loadDictionary() throws Exception {
+try (BufferedReader reader = new BufferedReader(new
FileReader(DICT_PATH))) {
+ for (int i = 0; i < NUM_DICT_TERMS; i++) {
+LOADED_DICTIONARY[i] = reader.readLine();
+ }
+}
+ }
+
+ public enum DType {
+USER(NUM_FROM_DOCS, FROM_COLLECTION_NAME) {
+ // id - unique string
+ // userid_s - username (user# from 1-100)...each user appears in #
entries
+ // permissions_ss - set of 300 string permissions (cardinality 50K)
+ @Override
+ SolrInputDocument buildDoc() {
+if (userRecordCounts[currentUser - 1] == currentUser) {
+ currentUser++;
+} else {
+ userRecordCounts[currentUser -1]++;
+}
+
+final SolrInputDocument newDoc = new SolrInputDocument("id",
UUID.randomUUID().toString());
+final String userString = "user" + currentUser;
+final String[] permissions = getAFewDictionaryWords(300,
PERMISSION_CARDINALITY);
+
+newDoc.addField("userid_s", userString);
+newDoc.addField("permissions_ss", permissions);
+
+return newDoc;
+ }
+},
+DATA(NUM_TO_DOCS, TO_COLLECTION_NAME) {
+ // id - unique string
+ // val_i - random int between 1-1000
+ // cost_d - random cost between 1-1000
+ // body_txt - random text string between 100 - 1 words
+ // acl_ss - set of 100-3000 string permissions (cardinality 50K)
+ @Override
+ SolrInputDocument buildDoc() {
+final SolrInputDocument newDoc = new SolrInputDocument("id",
UUID.randomUUID().toString());
+final int val = random().nextInt(1000) + 1;
+final double cost = random().nextDouble() * 1000d;
+final String body = String.join(" ",
getAFewDictionaryWords(random().nextInt(9900) + 100));
+final String[] ac
[jira] [Commented] (LUCENE-9165) change generate-defaults.gradle not to cap testsJvms at 4
[
https://issues.apache.org/jira/browse/LUCENE-9165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022827#comment-17022827
]
Robert Muir commented on LUCENE-9165:
-
{quote}
don't know if it's worth complicating the script though (those who have such
fancy machines can always override in ~/gradle.properties?).
{quote}
Yes, let users with fancy machines simply do this. Let's not have such
craziness in our build to try to be magical. Look at how complicated it can
get: https://bugs.openjdk.java.net/browse/JDK-6515172
And besides, even if we do it "by the book", we would get it wrong and guess 16
for David's case in LUCENE-9160. Looks/sounds like kvm is not configured
properly and has 16 "vcpu" mapped to only 8 real cores on the host, which we
can't see. So we could have all that complicated logic only to get it wrong
anyway.
It is java's problem to deal with: let's keep it as simple as possible.
> change generate-defaults.gradle not to cap testsJvms at 4
> -
>
> Key: LUCENE-9165
> URL: https://issues.apache.org/jira/browse/LUCENE-9165
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
> Attachments: LUCENE-9165.patch
>
>
> {code}
> def cpus = Runtime.runtime.availableProcessors()
> def testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 4))
> {code}
> Dividing by 2 is still good (hyperthreading is still a thing), but the
> artificial cap of 4 is a bottleneck if the hardware has more available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12930) Add developer documentation to source repo
[ https://issues.apache.org/jira/browse/SOLR-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022836#comment-17022836 ] Dawid Weiss commented on SOLR-12930: [~ctargett] the existence of help/ files is verified in a gradle check -- you've moved them under dev-docs and gradle build no longer passes. Can we leave it at top level? While these files are "dev-docs" they are also self-contained and helpful if available right there at the top-level of the checkout. If we bury them under a pile of other dev-docs I'm afraid nobody will care? > Add developer documentation to source repo > -- > > Key: SOLR-12930 > URL: https://issues.apache.org/jira/browse/SOLR-12930 > Project: Solr > Issue Type: Improvement > Components: Tests >Reporter: Mark Miller >Priority: Major > Attachments: solr-dev-docs.zip > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9123) JapaneseTokenizer with search mode doesn't work with SynonymGraphFilter
[
https://issues.apache.org/jira/browse/LUCENE-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022865#comment-17022865
]
Alan Woodward commented on LUCENE-9123:
---
> It's just a rather tricky fix, though I think we added a base class to make
>working with incoming graph tokens easier, which could make that fix also
>easier.
This is GraphTokenStream, which does indeed make consuming graphs easier. The
problem isn't going to be working with the input, though, it's going to be
dealing with output. Currently there is no good way to insert arbitrary graphs
into a token stream that already contains multiple paths without breaking those
existing paths.
Let's say we have an incoming stream with a token ABC that has been
decompounded into 'A B C':
ABC(1, 3) A(0, 1) B(1, 1), C(1, 1) D(1, 1)
Here ABC is the term, and the numbers in the brackets are the position
increment and position length of the token. Note that 'ABC' has a position
length of three, which if followed points to the token 'D'
We now insert a multiterm synonym for B = B Q Q
ABC(1, 3) A(0, 1) B(1, 3) B(0, 1) Q(1, 1) Q(1, 1) C(1, 1) D(1, 1)
Because of the extra tokens inserted in the 'A B C' branch of the existing
graph, the position length of the 'ABC' term now points to 'C' instead of 'D'.
But by the time we've reached 'B' in the tokenstream, 'ABC' has already been
emitted so we can't go back and adjust it's position length.
I think the solution is going to have to involve changes to the TokenStream API
at query time; the current posinc+poslen encoding isn't flexible enough, and
breaks in confusing ways when you try and modify things mid-stream. What we do
an index time is even trickier, because we can't encode graphs in the index
without making things like phrase queries or interval queries much more complex
(and probably a lot slower). We could enforce the graph token filters to only
be applied at query time, but that leaves the question of how to deal with
decompounding filters.
> JapaneseTokenizer with search mode doesn't work with SynonymGraphFilter
> ---
>
> Key: LUCENE-9123
> URL: https://issues.apache.org/jira/browse/LUCENE-9123
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
>Affects Versions: 8.4
>Reporter: Kazuaki Hiraga
>Assignee: Tomoko Uchida
>Priority: Major
> Attachments: LUCENE-9123.patch, LUCENE-9123_8x.patch
>
>
> JapaneseTokenizer with `mode=search` or `mode=extended` doesn't work with
> both of SynonymGraphFilter and SynonymFilter when JT generates multiple
> tokens as an output. If we use `mode=normal`, it should be fine. However, we
> would like to use decomposed tokens that can maximize to chance to increase
> recall.
> Snippet of schema:
> {code:xml}
> positionIncrementGap="100" autoGeneratePhraseQueries="false">
>
>
> synonyms="lang/synonyms_ja.txt"
> tokenizerFactory="solr.JapaneseTokenizerFactory"/>
>
>
> tags="lang/stoptags_ja.txt" />
>
>
>
>
>
> minimumLength="4"/>
>
>
>
>
> {code}
> An synonym entry that generates error:
> {noformat}
> 株式会社,コーポレーション
> {noformat}
> The following is an output on console:
> {noformat}
> $ ./bin/solr create_core -c jp_test -d ../config/solrconfs
> ERROR: Error CREATEing SolrCore 'jp_test': Unable to create core [jp_test3]
> Caused by: term: 株式会社 analyzed to a token (株式会社) with position increment != 1
> (got: 0)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mocobeta merged pull request #1169: LUCENE-9004: A minor feature and patch -- support deleting vector values and fix segments merging
mocobeta merged pull request #1169: LUCENE-9004: A minor feature and patch -- support deleting vector values and fix segments merging URL: https://github.com/apache/lucene-solr/pull/1169 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9168) Test failure after new randomized values in LuceneTestCase
Ignacio Vera created LUCENE-9168:
Summary: Test failure after new randomized values in LuceneTestCase
Key: LUCENE-9168
URL: https://issues.apache.org/jira/browse/LUCENE-9168
Project: Lucene - Core
Issue Type: Test
Reporter: Ignacio Vera
The error can be reproduced in branch_8x:
{code:java}
ant test -Dtestcase=TestPointValues -Dtests.method=testTieBreakByDocID
-Dtests.seed=A7EAA3BF5766C7C0 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=my -Dtests.timezone=Singapore -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1 {code}
And the error looks like:
{code:java}
07:08:28[junit4]> Caused by:
org.apache.lucene.index.MergePolicy$MergeException:
java.nio.file.FileSystemException:
/var/lib/jenkins/workspace/apache+lucene-solr+branch_8x/lucene/build/core/test/J1/temp/lucene.index.TestPointValues_A7EAA3BF5766C7C0-001/tempDir-002/_bd.tip:
Too many open files{code}
I had a look into the issue and the indexWriter in the test is configured to
buffer very few docs:
c.setMaxBufferedDocs(TestUtil.nextInt(r, 2, 15));
But the merging factor (LogMergePolicy) is too high:
logmp.setMergeFactor(TestUtil.nextInt(r, 100, 500));
The test is creating too many files and running over the limit. The last
statement was changed in LUCENE-9157 from:
logmp.setMergeFactor(TestUtil.nextInt(r, 10, 50));
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14215) Improve performance of SolrFeature when there are no filter queries
Victor Chenal created SOLR-14215:
Summary: Improve performance of SolrFeature when there are no
filter queries
Key: SOLR-14215
URL: https://issues.apache.org/jira/browse/SOLR-14215
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Victor Chenal
In cases where a SolrFeature only contains a main query {{q}} and no filter
queries {{fq}}, it is possible to traverse documents using the doc iterator
from {{q}} instead of constructing one from a processed filter that combines
all of the queries.
In our test datasets this made retrieving the feature *10 times faster*.
Specifically, the call to
[{{getProcessedFilter}}|https://github.com/apache/lucene-solr/blob/16e537db7efe37eb7519b07dc96c5026c0a1d193/solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java#L242-L243]
inside {{getDocIdSetIteratorFromQueries}} is expensive. That method aims at
returning a doc iterator that traverses over the documents that match all of
the criteria of {{q}} and {{fq}}.
When there are no filter queries (i.e. {{fq}} is null or empty), this just
returns a doc iterator for {{q}}. Using the iterator from that query directly
is a lot faster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14210) Introduce Node-level status handler for replicas
[
https://issues.apache.org/jira/browse/SOLR-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022893#comment-17022893
]
Jan Høydahl commented on SOLR-14210:
bq. But I'm not sure if I understand how "the deep check assures that requests
will likely succeed" would work.
I guess your check for all cores on same node is in OK state qualifies as
"deep". The node health must be node level. Not sure about readiness - some
collections may be ready while other collections are recovering, would you then
tell k8s to send traffic to the pod or not? And what is the exact check k8s
does before restarting a node in a rolling restart? Is there a separate probe
for when a certain pod/node can be restarted? For Solr it could be that a node
hosts the only live replica for a shard since the other replica on another
pod/node is currently recovering, but how would k8s know that? If we had a
probe /admin/info/health?safeToRestart then that could check clusterstate first?
> Introduce Node-level status handler for replicas
>
>
> Key: SOLR-14210
> URL: https://issues.apache.org/jira/browse/SOLR-14210
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: master (9.0), 8.5
>Reporter: Houston Putman
>Priority: Major
>
> h2. Background
> As was brought up in SOLR-13055, in order to run Solr in a more cloud-native
> way, we need some additional features around node-level healthchecks.
> {quote}Like in Kubernetes we need 'liveliness' and 'readiness' probe
> explained in
> [https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/n]
> determine if a node is live and ready to serve live traffic.
> {quote}
>
> However there are issues around kubernetes managing it's own rolling
> restarts. With the current healthcheck setup, it's easy to envision a
> scenario in which Solr reports itself as "healthy" when all of its replicas
> are actually recovering. Therefore kubernetes, seeing a healthy pod would
> then go and restart the next Solr node. This can happen until all replicas
> are "recovering" and none are healthy. (maybe the last one restarted will be
> "down", but still there are no "active" replicas)
> h2. Proposal
> I propose we make an additional healthcheck handler that returns whether all
> replicas hosted by that Solr node are healthy and "active". That way we will
> be able to use the [default kubernetes rolling restart
> logic|https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies]
> with Solr.
> To add on to [Jan's point
> here|https://issues.apache.org/jira/browse/SOLR-13055?focusedCommentId=16716559&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16716559],
> this handler should be more friendly for other Content-Types and should use
> bettter HTTP response statuses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9168) Test failure after new randomized values in LuceneTestCase
[
https://issues.apache.org/jira/browse/LUCENE-9168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022905#comment-17022905
]
Ignacio Vera commented on LUCENE-9168:
--
cc [~rmuir]
> Test failure after new randomized values in LuceneTestCase
> --
>
> Key: LUCENE-9168
> URL: https://issues.apache.org/jira/browse/LUCENE-9168
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Ignacio Vera
>Priority: Major
>
> The error can be reproduced in branch_8x:
> {code:java}
> ant test -Dtestcase=TestPointValues -Dtests.method=testTieBreakByDocID
> -Dtests.seed=A7EAA3BF5766C7C0 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=my -Dtests.timezone=Singapore -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1 {code}
>
> And the error looks like:
> {code:java}
> 07:08:28[junit4]> Caused by:
> org.apache.lucene.index.MergePolicy$MergeException:
> java.nio.file.FileSystemException:
> /var/lib/jenkins/workspace/apache+lucene-solr+branch_8x/lucene/build/core/test/J1/temp/lucene.index.TestPointValues_A7EAA3BF5766C7C0-001/tempDir-002/_bd.tip:
> Too many open files{code}
>
> I had a look into the issue and the indexWriter in the test is configured to
> buffer very few docs:
> c.setMaxBufferedDocs(TestUtil.nextInt(r, 2, 15));
>
> But the merging factor (LogMergePolicy) is too high:
> logmp.setMergeFactor(TestUtil.nextInt(r, 100, 500));
>
> The test is creating too many files and running over the limit. The last
> statement was changed in LUCENE-9157 from:
>
> logmp.setMergeFactor(TestUtil.nextInt(r, 10, 50));
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9168) Test failure after new randomized values in LuceneTestCase
[
https://issues.apache.org/jira/browse/LUCENE-9168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022926#comment-17022926
]
Robert Muir commented on LUCENE-9168:
-
oops i will fix. looks like i changed the wrong line. the issue is that it
indexes thousands of docs.
> Test failure after new randomized values in LuceneTestCase
> --
>
> Key: LUCENE-9168
> URL: https://issues.apache.org/jira/browse/LUCENE-9168
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Ignacio Vera
>Priority: Major
>
> The error can be reproduced in branch_8x:
> {code:java}
> ant test -Dtestcase=TestPointValues -Dtests.method=testTieBreakByDocID
> -Dtests.seed=A7EAA3BF5766C7C0 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=my -Dtests.timezone=Singapore -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1 {code}
>
> And the error looks like:
> {code:java}
> 07:08:28[junit4]> Caused by:
> org.apache.lucene.index.MergePolicy$MergeException:
> java.nio.file.FileSystemException:
> /var/lib/jenkins/workspace/apache+lucene-solr+branch_8x/lucene/build/core/test/J1/temp/lucene.index.TestPointValues_A7EAA3BF5766C7C0-001/tempDir-002/_bd.tip:
> Too many open files{code}
>
> I had a look into the issue and the indexWriter in the test is configured to
> buffer very few docs:
> c.setMaxBufferedDocs(TestUtil.nextInt(r, 2, 15));
>
> But the merging factor (LogMergePolicy) is too high:
> logmp.setMergeFactor(TestUtil.nextInt(r, 100, 500));
>
> The test is creating too many files and running over the limit. The last
> statement was changed in LUCENE-9157 from:
>
> logmp.setMergeFactor(TestUtil.nextInt(r, 10, 50));
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] victorchenal opened a new pull request #1206: SOLR-14215: Improve SolrFeature performance when there are no filter queries
victorchenal opened a new pull request #1206: SOLR-14215: Improve SolrFeature performance when there are no filter queries URL: https://github.com/apache/lucene-solr/pull/1206 # Description In cases where a SolrFeature only contains a main query `q` and no filter queries `fq`, it is possible to traverse documents using the doc iterator from `q` instead of constructing one from a processed filter that combines all of the queries. In our test datasets this made retrieving the feature *10 times faster*. Specifically, the call to `getProcessedFilter` inside `getDocIdSetIteratorFromQueries` is expensive: https://github.com/apache/lucene-solr/blob/16e537db7efe37eb7519b07dc96c5026c0a1d193/solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java#L242-L243 That method aims at returning a doc iterator that traverses over the documents that match all of the criteria of `q` and `fq`. When there are no filter queries, this just returns a doc iterator for `q`. # Solution When `fq` is null or empty, use the iterator from `q` directly as it is a lot faster. # Tests Existing tests for `SolrFeature` already cover this use case, in particular `TestUserTermScoreWithQ`. # Checklist - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [x] I have developed this patch against the `master` branch. - [x] I have run `ant precommit` and the appropriate test suite. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz merged pull request #1126: LUCENE-4702: Terms dictionary compression.
jpountz merged pull request #1126: LUCENE-4702: Terms dictionary compression. URL: https://github.com/apache/lucene-solr/pull/1126 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9168) Test failure after new randomized values in LuceneTestCase
[
https://issues.apache.org/jira/browse/LUCENE-9168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022945#comment-17022945
]
ASF subversion and git services commented on LUCENE-9168:
-
Commit a29a4f4aa5224fc9a24ab247f0b028994025d820 in lucene-solr's branch
refs/heads/master from Robert Muir
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a29a4f4 ]
LUCENE-9168: don't let crazy tests run us out of open files with these params
> Test failure after new randomized values in LuceneTestCase
> --
>
> Key: LUCENE-9168
> URL: https://issues.apache.org/jira/browse/LUCENE-9168
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Ignacio Vera
>Priority: Major
>
> The error can be reproduced in branch_8x:
> {code:java}
> ant test -Dtestcase=TestPointValues -Dtests.method=testTieBreakByDocID
> -Dtests.seed=A7EAA3BF5766C7C0 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=my -Dtests.timezone=Singapore -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1 {code}
>
> And the error looks like:
> {code:java}
> 07:08:28[junit4]> Caused by:
> org.apache.lucene.index.MergePolicy$MergeException:
> java.nio.file.FileSystemException:
> /var/lib/jenkins/workspace/apache+lucene-solr+branch_8x/lucene/build/core/test/J1/temp/lucene.index.TestPointValues_A7EAA3BF5766C7C0-001/tempDir-002/_bd.tip:
> Too many open files{code}
>
> I had a look into the issue and the indexWriter in the test is configured to
> buffer very few docs:
> c.setMaxBufferedDocs(TestUtil.nextInt(r, 2, 15));
>
> But the merging factor (LogMergePolicy) is too high:
> logmp.setMergeFactor(TestUtil.nextInt(r, 100, 500));
>
> The test is creating too many files and running over the limit. The last
> statement was changed in LUCENE-9157 from:
>
> logmp.setMergeFactor(TestUtil.nextInt(r, 10, 50));
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] cpoerschke commented on issue #1172: SOLR-14189 switch from String.trim() to StringUtils.isWhitespace()
cpoerschke commented on issue #1172: SOLR-14189 switch from String.trim() to
StringUtils.isWhitespace()
URL: https://github.com/apache/lucene-solr/pull/1172#issuecomment-578136051
> ... String.trim() ... StringUtils.stripToNull() ...
StringUtils.isWhitespace() ...
I wonder if `StringUtils.isBlank()` might be another option:
```
- if (val == null || StringUtils.isWhitespace(val) ) {
+ if (StringUtils.isBlank(val)) {
```
Not sure about `StringUtils.isWhitespace` vs. `StringUtils.isBlank`
efficiency considerations though e.g.
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.4.1/solr/core/src/java/org/apache/solr/search/SwitchQParserPlugin.java#L166
uses it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4702) Terms dictionary compression
[
https://issues.apache.org/jira/browse/LUCENE-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022946#comment-17022946
]
ASF subversion and git services commented on LUCENE-4702:
-
Commit b283b8df628dc9bbdbbb994b4a3653b7eecd7fd9 in lucene-solr's branch
refs/heads/master from Adrien Grand
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b283b8d ]
LUCENE-4702: Terms dictionary compression. (#1126)
Compress blocks of suffixes in order to make the terms dictionary more
space-efficient. Two compression algorithms are used depending on which one is
more space-efficient:
- LowercaseAsciiCompression, which applies when all bytes are in the
`[0x1F,0x3F)` or `[0x5F,0x7F)` ranges, which notably include all digits,
lowercase ASCII characters, '.', '-' and '_', and encodes 4 chars on 3 bytes.
It is very often applicable on analyzed content and decompresses very quickly
thanks to auto-vectorization support in the JVM.
- LZ4, when the compression ratio is less than 0.75.
I was a bit unhappy with the complexity of the high-compression LZ4 option, so
I simplified it in order to only keep the logic that detects duplicate strings.
The logic about what to do in case overlapping matches are found, which was
responsible for most of the complexity while only yielding tiny benefits, has
been removed.
> Terms dictionary compression
>
>
> Key: LUCENE-4702
> URL: https://issues.apache.org/jira/browse/LUCENE-4702
> Project: Lucene - Core
> Issue Type: Wish
>Reporter: Adrien Grand
>Assignee: Adrien Grand
>Priority: Trivial
> Attachments: LUCENE-4702.patch, LUCENE-4702.patch
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> I've done a quick test with the block tree terms dictionary by replacing a
> call to IndexOutput.writeBytes to write suffix bytes with a call to
> LZ4.compressHC to test the peformance hit. Interestingly, search performance
> was very good (see comparison table below) and the tim files were 14% smaller
> (from 150432 bytes overall to 129516).
> {noformat}
> TaskQPS baseline StdDevQPS compressed StdDev
> Pct diff
> Fuzzy1 111.50 (2.0%) 78.78 (1.5%)
> -29.4% ( -32% - -26%)
> Fuzzy2 36.99 (2.7%) 28.59 (1.5%)
> -22.7% ( -26% - -18%)
> Respell 122.86 (2.1%) 103.89 (1.7%)
> -15.4% ( -18% - -11%)
> Wildcard 100.58 (4.3%) 94.42 (3.2%)
> -6.1% ( -13% -1%)
> Prefix3 124.90 (5.7%) 122.67 (4.7%)
> -1.8% ( -11% -9%)
>OrHighLow 169.87 (6.8%) 167.77 (8.0%)
> -1.2% ( -15% - 14%)
> LowTerm 1949.85 (4.5%) 1929.02 (3.4%)
> -1.1% ( -8% -7%)
> AndHighLow 2011.95 (3.5%) 1991.85 (3.3%)
> -1.0% ( -7% -5%)
> OrHighHigh 155.63 (6.7%) 154.12 (7.9%)
> -1.0% ( -14% - 14%)
> AndHighHigh 341.82 (1.2%) 339.49 (1.7%)
> -0.7% ( -3% -2%)
>OrHighMed 217.55 (6.3%) 216.16 (7.1%)
> -0.6% ( -13% - 13%)
> IntNRQ 53.10 (10.9%) 52.90 (8.6%)
> -0.4% ( -17% - 21%)
> MedTerm 998.11 (3.8%) 994.82 (5.6%)
> -0.3% ( -9% -9%)
> MedSpanNear 60.50 (3.7%) 60.36 (4.8%)
> -0.2% ( -8% -8%)
> HighSpanNear 19.74 (4.5%) 19.72 (5.1%)
> -0.1% ( -9% -9%)
> LowSpanNear 101.93 (3.2%) 101.82 (4.4%)
> -0.1% ( -7% -7%)
> AndHighMed 366.18 (1.7%) 366.93 (1.7%)
> 0.2% ( -3% -3%)
> PKLookup 237.28 (4.0%) 237.96 (4.2%)
> 0.3% ( -7% -8%)
>MedPhrase 173.17 (4.7%) 174.69 (4.7%)
> 0.9% ( -8% - 10%)
> LowSloppyPhrase 180.91 (2.6%) 182.79 (2.7%)
> 1.0% ( -4% -6%)
>LowPhrase 374.64 (5.5%) 379.11 (5.8%)
> 1.2% ( -9% - 13%)
> HighTerm 253.14 (7.9%) 256.97 (11.4%)
> 1.5% ( -16% - 22%)
> HighPhrase 19.52 (10.6%) 19.83 (11.0%)
> 1.6% ( -18% - 25%)
> MedSloppyPhrase 141.90 (2.6%) 144.11 (2.5%)
> 1.6% ( -3% -6%)
> HighSloppyPhrase 25.26 (4.8%) 25.97 (5.0%)
> 2.8% ( -6% - 13%)
> {noformat}
> Only queries which are very terms-dictionary-intensive got a performa
[jira] [Commented] (LUCENE-9168) Test failure after new randomized values in LuceneTestCase
[
https://issues.apache.org/jira/browse/LUCENE-9168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022947#comment-17022947
]
ASF subversion and git services commented on LUCENE-9168:
-
Commit 5070ee8bbe6347e345a516625d2e834c2c08577d in lucene-solr's branch
refs/heads/branch_8x from Robert Muir
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5070ee8 ]
LUCENE-9168: don't let crazy tests run us out of open files with these params
> Test failure after new randomized values in LuceneTestCase
> --
>
> Key: LUCENE-9168
> URL: https://issues.apache.org/jira/browse/LUCENE-9168
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Ignacio Vera
>Priority: Major
>
> The error can be reproduced in branch_8x:
> {code:java}
> ant test -Dtestcase=TestPointValues -Dtests.method=testTieBreakByDocID
> -Dtests.seed=A7EAA3BF5766C7C0 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=my -Dtests.timezone=Singapore -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1 {code}
>
> And the error looks like:
> {code:java}
> 07:08:28[junit4]> Caused by:
> org.apache.lucene.index.MergePolicy$MergeException:
> java.nio.file.FileSystemException:
> /var/lib/jenkins/workspace/apache+lucene-solr+branch_8x/lucene/build/core/test/J1/temp/lucene.index.TestPointValues_A7EAA3BF5766C7C0-001/tempDir-002/_bd.tip:
> Too many open files{code}
>
> I had a look into the issue and the indexWriter in the test is configured to
> buffer very few docs:
> c.setMaxBufferedDocs(TestUtil.nextInt(r, 2, 15));
>
> But the merging factor (LogMergePolicy) is too high:
> logmp.setMergeFactor(TestUtil.nextInt(r, 100, 500));
>
> The test is creating too many files and running over the limit. The last
> statement was changed in LUCENE-9157 from:
>
> logmp.setMergeFactor(TestUtil.nextInt(r, 10, 50));
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9168) Test failure after new randomized values in LuceneTestCase
[
https://issues.apache.org/jira/browse/LUCENE-9168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-9168.
-
Fix Version/s: 8.5
Resolution: Fixed
Thanks [~ivera], sorry for the failure.
> Test failure after new randomized values in LuceneTestCase
> --
>
> Key: LUCENE-9168
> URL: https://issues.apache.org/jira/browse/LUCENE-9168
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Ignacio Vera
>Priority: Major
> Fix For: 8.5
>
>
> The error can be reproduced in branch_8x:
> {code:java}
> ant test -Dtestcase=TestPointValues -Dtests.method=testTieBreakByDocID
> -Dtests.seed=A7EAA3BF5766C7C0 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=my -Dtests.timezone=Singapore -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1 {code}
>
> And the error looks like:
> {code:java}
> 07:08:28[junit4]> Caused by:
> org.apache.lucene.index.MergePolicy$MergeException:
> java.nio.file.FileSystemException:
> /var/lib/jenkins/workspace/apache+lucene-solr+branch_8x/lucene/build/core/test/J1/temp/lucene.index.TestPointValues_A7EAA3BF5766C7C0-001/tempDir-002/_bd.tip:
> Too many open files{code}
>
> I had a look into the issue and the indexWriter in the test is configured to
> buffer very few docs:
> c.setMaxBufferedDocs(TestUtil.nextInt(r, 2, 15));
>
> But the merging factor (LogMergePolicy) is too high:
> logmp.setMergeFactor(TestUtil.nextInt(r, 100, 500));
>
> The test is creating too many files and running over the limit. The last
> statement was changed in LUCENE-9157 from:
>
> logmp.setMergeFactor(TestUtil.nextInt(r, 10, 50));
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9165) change generate-defaults.gradle not to cap testsJvms at 4
[
https://issues.apache.org/jira/browse/LUCENE-9165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-9165.
-
Fix Version/s: master (9.0)
Resolution: Fixed
> change generate-defaults.gradle not to cap testsJvms at 4
> -
>
> Key: LUCENE-9165
> URL: https://issues.apache.org/jira/browse/LUCENE-9165
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: LUCENE-9165.patch
>
>
> {code}
> def cpus = Runtime.runtime.availableProcessors()
> def testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 4))
> {code}
> Dividing by 2 is still good (hyperthreading is still a thing), but the
> artificial cap of 4 is a bottleneck if the hardware has more available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9167) test speedup for slowest/pathological tests (round 3)
[ https://issues.apache.org/jira/browse/LUCENE-9167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022961#comment-17022961 ] ASF subversion and git services commented on LUCENE-9167: - Commit c53cc3edafbb00c9b1f411af7c2abf48e9b75288 in lucene-solr's branch refs/heads/master from Robert Muir [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c53cc3e ] LUCENE-9167: test speedup for slowest/pathological tests (round 3) > test speedup for slowest/pathological tests (round 3) > - > > Key: LUCENE-9167 > URL: https://issues.apache.org/jira/browse/LUCENE-9167 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Attachments: LUCENE-9167.patch > > > More fixing of tests, following the same formula as LUCENE-9163. > I tried to focus again on tests that can randomly "explode", applied a few > more techniques to sort them out, for example running all tests with > SimpleText codec and suppressing that codec for the ones that really struggle > with it, running with different Directory implementations, etc. > These kinds of outliers will randomly send test times sky high: so while this > patch may only save a minute or so for the best case, I think it really helps > reduce the stddev and worst case. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9167) test speedup for slowest/pathological tests (round 3)
[ https://issues.apache.org/jira/browse/LUCENE-9167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-9167. - Fix Version/s: master (9.0) Resolution: Fixed > test speedup for slowest/pathological tests (round 3) > - > > Key: LUCENE-9167 > URL: https://issues.apache.org/jira/browse/LUCENE-9167 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Fix For: master (9.0) > > Attachments: LUCENE-9167.patch > > > More fixing of tests, following the same formula as LUCENE-9163. > I tried to focus again on tests that can randomly "explode", applied a few > more techniques to sort them out, for example running all tests with > SimpleText codec and suppressing that codec for the ones that really struggle > with it, running with different Directory implementations, etc. > These kinds of outliers will randomly send test times sky high: so while this > patch may only save a minute or so for the best case, I think it really helps > reduce the stddev and worst case. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9168) Test failure after new randomized values in LuceneTestCase
[
https://issues.apache.org/jira/browse/LUCENE-9168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022964#comment-17022964
]
Ignacio Vera commented on LUCENE-9168:
--
Thanks t you [~rcmuir], the effort in speeding up the test is awesome :)
> Test failure after new randomized values in LuceneTestCase
> --
>
> Key: LUCENE-9168
> URL: https://issues.apache.org/jira/browse/LUCENE-9168
> Project: Lucene - Core
> Issue Type: Test
>Reporter: Ignacio Vera
>Priority: Major
> Fix For: 8.5
>
>
> The error can be reproduced in branch_8x:
> {code:java}
> ant test -Dtestcase=TestPointValues -Dtests.method=testTieBreakByDocID
> -Dtests.seed=A7EAA3BF5766C7C0 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=my -Dtests.timezone=Singapore -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1 {code}
>
> And the error looks like:
> {code:java}
> 07:08:28[junit4]> Caused by:
> org.apache.lucene.index.MergePolicy$MergeException:
> java.nio.file.FileSystemException:
> /var/lib/jenkins/workspace/apache+lucene-solr+branch_8x/lucene/build/core/test/J1/temp/lucene.index.TestPointValues_A7EAA3BF5766C7C0-001/tempDir-002/_bd.tip:
> Too many open files{code}
>
> I had a look into the issue and the indexWriter in the test is configured to
> buffer very few docs:
> c.setMaxBufferedDocs(TestUtil.nextInt(r, 2, 15));
>
> But the merging factor (LogMergePolicy) is too high:
> logmp.setMergeFactor(TestUtil.nextInt(r, 100, 500));
>
> The test is creating too many files and running over the limit. The last
> statement was changed in LUCENE-9157 from:
>
> logmp.setMergeFactor(TestUtil.nextInt(r, 10, 50));
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12930) Add developer documentation to source repo
[
https://issues.apache.org/jira/browse/SOLR-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022967#comment-17022967
]
Robert Muir commented on SOLR-12930:
Also the file move did not fix the gradle in-line help itself (I'm relying on
it a lot, since I'm not used to the commands). Now it fails with missing files:
{noformat}
$ ./gradlew helpWorkFlow
To honour the JVM settings for this build a new JVM will be forked. Please
consider using the daemon:
https://docs.gradle.org/6.0.1/userguide/gradle_daemon.html.
Daemon will be stopped at the end of the build stopping after processing
> Task :buildSrc:compileJava UP-TO-DATE
> Task :buildSrc:compileGroovy NO-SOURCE
> Task :buildSrc:processResources NO-SOURCE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
> Task :helpWorkflow FAILED
1 actionable task: 1 executed
<-> 0% WAITING
> IDLE
FAILURE: Build failed with an exception.
* Where:
Script '/home/rmuir/workspace/lucene-solr/gradle/help.gradle' line: 19
* What went wrong:
Execution failed for task ':helpWorkflow'.
> java.io.FileNotFoundException:
> /home/rmuir/workspace/lucene-solr/help/workflow.txt (No such file or
> directory)
{noformat}
> Add developer documentation to source repo
> --
>
> Key: SOLR-12930
> URL: https://issues.apache.org/jira/browse/SOLR-12930
> Project: Solr
> Issue Type: Improvement
> Components: Tests
>Reporter: Mark Miller
>Priority: Major
> Attachments: solr-dev-docs.zip
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14194) Allow Highlighting to work for indexes with uniqueKey that is not stored
[ https://issues.apache.org/jira/browse/SOLR-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022737#comment-17022737 ] Andrzej Wislowski edited comment on SOLR-14194 at 1/24/20 2:15 PM: --- I'll check new highlighter and fix tests was (Author: awislowski): I'll check news highlighter and fix tests pt., 24 sty 2020, 05:10 użytkownik David Smiley (Jira) > Allow Highlighting to work for indexes with uniqueKey that is not stored > > > Key: SOLR-14194 > URL: https://issues.apache.org/jira/browse/SOLR-14194 > Project: Solr > Issue Type: Improvement > Components: highlighter >Affects Versions: master (9.0) >Reporter: Andrzej Wislowski >Assignee: David Smiley >Priority: Minor > Labels: highlighter > Fix For: master (9.0) > > Attachments: SOLR-14194.patch > > > Highlighting requires uniqueKey to be a stored field. I have changed > Highlighter allow returning results on indexes with uniqueKey that is a not > stored field, but saved as a docvalue type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on issue #1202: SOLR-14149: CHANGES.txt Remove off-topic stuff
janhoy commented on issue #1202: SOLR-14149: CHANGES.txt Remove off-topic stuff URL: https://github.com/apache/lucene-solr/pull/1202#issuecomment-578152267 > Or maybe we don't and this is a difference between both branches for the asciidoc file similarly to how CHANGES.txt is necessarily different? I believe this would work. The master branch should contain what would go in a 9.0 release if we released today, so why not maintain the Upgrade Notes there from the start? Since it is easier to forget to touch two files, perhaps this should be a new checkbox in GH PR template "[ ] I have added an Upgrade Note to RefGuide if there are breaking changes". This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14193) Update tutorial.adoc(line no:664) so that command executes in windows enviroment
[
https://issues.apache.org/jira/browse/SOLR-14193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022991#comment-17022991
]
balaji sundaram commented on SOLR-14193:
[~ctargett]
Apologies, Without reading how to commit to Apache Solr i have raised the pull
request. After reading the documentation, i came to know that i have to create
a Jira issue and commit it to github from my fork.
That is why i created a Jira issue, if it is ok with you, i will try to raise
a PR next week.
> Update tutorial.adoc(line no:664) so that command executes in windows
> enviroment
>
>
> Key: SOLR-14193
> URL: https://issues.apache.org/jira/browse/SOLR-14193
> Project: Solr
> Issue Type: Bug
> Components: documentation
>Affects Versions: 8.4
>Reporter: balaji sundaram
>Priority: Minor
>
>
> {{When executing the following command in windows 10 "java -jar -Dc=films
> -Dparams=f.genre.split=true&f.directed_by.split=true&f.genre.separator=|&f.directed_by.separator=|
> -Dauto example\exampledocs\post.jar example\films\*.csv", it throws error "&
> was unexpected at this time."}}
> Fix: the command should escape "&" and "|" symbol{{}}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14193) Update tutorial.adoc(line no:664) so that command executes in windows enviroment
[
https://issues.apache.org/jira/browse/SOLR-14193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022991#comment-17022991
]
balaji sundaram edited comment on SOLR-14193 at 1/24/20 2:53 PM:
-
[~ctargett]
Apologies, Without reading how to commit to Apache Solr i have raised the pull
request. After reading the documentation, i came to know that i have to create
a Jira issue and commit it to github from my fork. So I closed the PR, since
it doesn't match guidelines set by Apache
That is why i created a Jira issue for trackin purposes. If it is ok with you,
i will try to raise a PR next week for this mentioned issue.
was (Author: balaji-s):
[~ctargett]
Apologies, Without reading how to commit to Apache Solr i have raised the pull
request. After reading the documentation, i came to know that i have to create
a Jira issue and commit it to github from my fork.
That is why i created a Jira issue, if it is ok with you, i will try to raise
a PR next week.
> Update tutorial.adoc(line no:664) so that command executes in windows
> enviroment
>
>
> Key: SOLR-14193
> URL: https://issues.apache.org/jira/browse/SOLR-14193
> Project: Solr
> Issue Type: Bug
> Components: documentation
>Affects Versions: 8.4
>Reporter: balaji sundaram
>Priority: Minor
>
>
> {{When executing the following command in windows 10 "java -jar -Dc=films
> -Dparams=f.genre.split=true&f.directed_by.split=true&f.genre.separator=|&f.directed_by.separator=|
> -Dauto example\exampledocs\post.jar example\films\*.csv", it throws error "&
> was unexpected at this time."}}
> Fix: the command should escape "&" and "|" symbol{{}}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9165) change generate-defaults.gradle not to cap testsJvms at 4
[
https://issues.apache.org/jira/browse/LUCENE-9165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022996#comment-17022996
]
ASF subversion and git services commented on LUCENE-9165:
-
Commit f5e9bb9493345284cc8ab54e33c94e64c53d65eb in lucene-solr's branch
refs/heads/master from Robert Muir
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f5e9bb9 ]
LUCENE-9165: explicitly cast with the horrible groovy language so that numbers
above 9 don't fail
> change generate-defaults.gradle not to cap testsJvms at 4
> -
>
> Key: LUCENE-9165
> URL: https://issues.apache.org/jira/browse/LUCENE-9165
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Robert Muir
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: LUCENE-9165.patch
>
>
> {code}
> def cpus = Runtime.runtime.availableProcessors()
> def testsJvms = (int) Math.max(1d, Math.min(cpus * 0.5d, 4))
> {code}
> Dividing by 2 is still good (hyperthreading is still a thing), but the
> artificial cap of 4 is a bottleneck if the hardware has more available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on issue #1202: SOLR-14149: CHANGES.txt Remove off-topic stuff
dsmiley commented on issue #1202: SOLR-14149: CHANGES.txt Remove off-topic stuff URL: https://github.com/apache/lucene-solr/pull/1202#issuecomment-578176920 > this is a difference between both branches Good; we agree. I'll update the PR. > perhaps this should be a new checkbox in GH PR template I disagree on that because I view the audience of that template to be our contributors, not us committers. I delete it as step 1 of a PR. Notice this comment at the very top of the template that I added: _(If you are a project committer then you may remove some/all of the following template.)_ I think it's enough for the CHANGES.txt header to refer to solr-upgrade-notes (in some fashion) as that is an inherent reminder that there are other upgrade notes. Besides, this is really for us committers to do and we will form new habits. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14194) Allow Highlighting to work for indexes with uniqueKey that is not stored
[ https://issues.apache.org/jira/browse/SOLR-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Wislowski updated SOLR-14194: - Attachment: SOLR-14194.patch Status: Patch Available (was: Patch Available) > Allow Highlighting to work for indexes with uniqueKey that is not stored > > > Key: SOLR-14194 > URL: https://issues.apache.org/jira/browse/SOLR-14194 > Project: Solr > Issue Type: Improvement > Components: highlighter >Affects Versions: master (9.0) >Reporter: Andrzej Wislowski >Assignee: David Smiley >Priority: Minor > Labels: highlighter > Fix For: master (9.0) > > Attachments: SOLR-14194.patch, SOLR-14194.patch > > > Highlighting requires uniqueKey to be a stored field. I have changed > Highlighter allow returning results on indexes with uniqueKey that is a not > stored field, but saved as a docvalue type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023027#comment-17023027
]
David Smiley commented on SOLR-13749:
-
Why was this added under "Improvements" and not "New Features"? Surely it's
the latter? Make the category if need be.
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be returned.
> This query parser will do an intersection based on join keys between 2
> collections.
> The local collection is the collection that you are searching against.
> The remote collection is the collection that contains the join keys that you
> want to use as a filter.
> Each shard participating in the distributed request will execute a query
> against the remote collection. If the local collection is setup with the
> compositeId router to be routed on the join key field, a hash range query is
> applied to the remote collection query to only match the documents that
> contain a potential match for the documents that are in the local shard/core.
>
>
> Here's some vocab to help with the descriptions of the various parameters.
> ||Term||Description||
> |Local Collection|This is the main collection that is being queried.|
> |Remote Collection|This is the collection that the XCJFQuery will query to
> resolve the join keys.|
> |XCJFQuery|The lucene query that executes a search to get back a set of join
> keys from a remote collection|
> |HashRangeQuery|The lucene query that matches only the documents whose hash
> code on a field falls within a specified range.|
>
>
> ||Param ||Required ||Description||
> |collection|Required|The name of the external Solr collection to be queried
> to retrieve the set of join key values ( required )|
> |zkHost|Optional|The connection string to be used to connect to Zookeeper.
> zkHost and solrUrl are both optional parameters, and at most one of them
> should be specified.
> If neither of zkHost or solrUrl are specified, the local Zookeeper cluster
> will be used. ( optional )|
> |solrUrl|Optional|The URL of the external Solr node to be queried ( optional
> )|
> |from|Required|The join key field name in the external collection ( required
> )|
> |to|Required|The join key field name in the local collection|
> |v|See Note|The query to be executed against the external Solr collection to
> retrieve the set of join key values.
> Note: The original query can be passed at the end of the string or as the
> "v" parameter.
> It's recommended to use query parameter substitution with the "v" parameter
> to ensure no issues arise with the default query parsers.|
> |routed| |true / false. If true, the XCJF query will use each shard's hash
> range to determine the set of join keys to retrieve for that shard.
> This parameter improves the performance of the cross-collection join, but
> it depends on the local collection being routed by the toField. If this
> parameter is not specified,
> the XCJF query will try to determine the correct value automatically.|
> |ttl| |The length of time that an XCJF query in the cache will be considered
> valid, in seconds. Defaults to 3600 (one hour).
> The XCJF query will not be aware of changes to the remote collection, so
> if the remote collection is updated, cached XCJF queries may give inaccurate
> results.
> After the ttl period has expired, the XCJF query will re-execute the join
> against the remote collection.|
> |_All others_| |Any normal Solr parameter can also be specified as a local
> param.|
>
> Example Solr Config.xml changes:
>
> {{<}}{{cache}} {{name}}{{=}}{{"hash_vin"}}
> {{ }}{{class}}{{=}}{{"solr.LRUCache"}}
> {{ }}{{size}}{{=}}{{"128"}}
> {{ }}{{initialSize}}{{=}}{{"0"}}
> {{ }}{{regenerator}}{{=}}{{"solr.NoOpRegenerator"}}{{/>}}
>
> {{<}}{{queryParser}} {{name}}{{=}}{{"xcjf"}}
> {{class}}{{=}}{{"org.apache.solr.search.join.XCJFQueryParserPlugin"}}{{>}}
> {{ }}{{<}}{{str}} {{name}}{{=
[jira] [Commented] (SOLR-14209) Upgrade JQuery to 3.4.1
[ https://issues.apache.org/jira/browse/SOLR-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023029#comment-17023029 ] Kevin Risden commented on SOLR-14209: - I made good progress on this last night. There are some changes to how the data gets loaded into jstree, but overall better than before. The only outstanding item is fixing the styling of the logging tab since new version of jstree has new CSS. > Upgrade JQuery to 3.4.1 > --- > > Key: SOLR-14209 > URL: https://issues.apache.org/jira/browse/SOLR-14209 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Admin UI, contrib - Velocity >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Major > Attachments: Screen Shot 2020-01-23 at 3.17.07 PM.png, Screen Shot > 2020-01-23 at 3.28.47 PM.png > > > Currently JQuery is on 2.1.3. It would be good to upgrade to the latest > version if possible. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14194) Allow Highlighting to work for indexes with uniqueKey that is not stored
[ https://issues.apache.org/jira/browse/SOLR-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Wislowski updated SOLR-14194: - Status: Open (was: Patch Available) > Allow Highlighting to work for indexes with uniqueKey that is not stored > > > Key: SOLR-14194 > URL: https://issues.apache.org/jira/browse/SOLR-14194 > Project: Solr > Issue Type: Improvement > Components: highlighter >Affects Versions: master (9.0) >Reporter: Andrzej Wislowski >Assignee: David Smiley >Priority: Minor > Labels: highlighter > Fix For: master (9.0) > > Attachments: SOLR-14194.patch, SOLR-14194.patch > > > Highlighting requires uniqueKey to be a stored field. I have changed > Highlighter allow returning results on indexes with uniqueKey that is a not > stored field, but saved as a docvalue type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14194) Allow Highlighting to work for indexes with uniqueKey that is not stored
[ https://issues.apache.org/jira/browse/SOLR-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Wislowski updated SOLR-14194: - Status: Patch Available (was: Open) > Allow Highlighting to work for indexes with uniqueKey that is not stored > > > Key: SOLR-14194 > URL: https://issues.apache.org/jira/browse/SOLR-14194 > Project: Solr > Issue Type: Improvement > Components: highlighter >Affects Versions: master (9.0) >Reporter: Andrzej Wislowski >Assignee: David Smiley >Priority: Minor > Labels: highlighter > Fix For: master (9.0) > > Attachments: SOLR-14194.patch, SOLR-14194.patch > > > Highlighting requires uniqueKey to be a stored field. I have changed > Highlighter allow returning results on indexes with uniqueKey that is a not > stored field, but saved as a docvalue type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14194) Allow Highlighting to work for indexes with uniqueKey that is not stored
[ https://issues.apache.org/jira/browse/SOLR-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023031#comment-17023031 ] Andrzej Wislowski commented on SOLR-14194: -- I have attached a patch, that also include change in the UnifiedSolrHighlighter > Allow Highlighting to work for indexes with uniqueKey that is not stored > > > Key: SOLR-14194 > URL: https://issues.apache.org/jira/browse/SOLR-14194 > Project: Solr > Issue Type: Improvement > Components: highlighter >Affects Versions: master (9.0) >Reporter: Andrzej Wislowski >Assignee: David Smiley >Priority: Minor > Labels: highlighter > Fix For: master (9.0) > > Attachments: SOLR-14194.patch, SOLR-14194.patch > > > Highlighting requires uniqueKey to be a stored field. I have changed > Highlighter allow returning results on indexes with uniqueKey that is a not > stored field, but saved as a docvalue type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14210) Introduce Node-level status handler for replicas
[
https://issues.apache.org/jira/browse/SOLR-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023039#comment-17023039
]
Houston Putman commented on SOLR-14210:
---
[~shalin], that's a good idea. No need for more handlers. One handler that
checks health at different levels is exactly what we want.
[~janhoy] , unfortunately k8s does not separate "is this ready for traffic?"
and "is this ready for the next pod to be restarted?". It would be wonderful if
there was a restartableProbe, but it will probably be hard to get that into k8s
anytime soon. Luckily there is a parameter for Services
"{{publishNotReadyAddresses: true"}} that will allow your Service objects
exposing your Solr clouds to send requests to pods that aren't ready. So this
lets us treat the "readyProbe" much more like a "restartableProbe". It's kind
of a mis-use of kubernetes, but you do what you have to do.
As per restarting multiple nodes at the same time that don't host the same
shards, that would be awesome. I did some research into it and I don't think we
would be able to handle it with just StatefulSets. We could probably build
something into the [solr operator|https://github.com/bloomberg/solr-operator],
but k8s doesn't let you really control parallel statefulSet pod upgrades very
well. I think the only way it could be done is using the [on-delete update
strategy|https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#on-delete]
and have the solr-operator (or any controller for a solr CRD) smartly pick
which pods it can delete in parallel without taking down multiple replicas of
any shard. I do agree it would be awesome to add functionality in this handler
that could eventually make those parallel operations easier.
> Introduce Node-level status handler for replicas
>
>
> Key: SOLR-14210
> URL: https://issues.apache.org/jira/browse/SOLR-14210
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: master (9.0), 8.5
>Reporter: Houston Putman
>Priority: Major
>
> h2. Background
> As was brought up in SOLR-13055, in order to run Solr in a more cloud-native
> way, we need some additional features around node-level healthchecks.
> {quote}Like in Kubernetes we need 'liveliness' and 'readiness' probe
> explained in
> [https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/n]
> determine if a node is live and ready to serve live traffic.
> {quote}
>
> However there are issues around kubernetes managing it's own rolling
> restarts. With the current healthcheck setup, it's easy to envision a
> scenario in which Solr reports itself as "healthy" when all of its replicas
> are actually recovering. Therefore kubernetes, seeing a healthy pod would
> then go and restart the next Solr node. This can happen until all replicas
> are "recovering" and none are healthy. (maybe the last one restarted will be
> "down", but still there are no "active" replicas)
> h2. Proposal
> I propose we make an additional healthcheck handler that returns whether all
> replicas hosted by that Solr node are healthy and "active". That way we will
> be able to use the [default kubernetes rolling restart
> logic|https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies]
> with Solr.
> To add on to [Jan's point
> here|https://issues.apache.org/jira/browse/SOLR-13055?focusedCommentId=16716559&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16716559],
> this handler should be more friendly for other Content-Types and should use
> bettter HTTP response statuses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on issue #1202: SOLR-14149: CHANGES.txt Remove off-topic stuff
janhoy commented on issue #1202: SOLR-14149: CHANGES.txt Remove off-topic stuff URL: https://github.com/apache/lucene-solr/pull/1202#issuecomment-578188915 Agree on the PR template. An even more “live” page for master upgrade notes would be https://github.com/apache/lucene-solr/blob/master/solr/solr-ref-guide/src/solr-upgrade-notes.adoc This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on issue #1202: SOLR-14149: CHANGES.txt Remove off-topic stuff
dsmiley commented on issue #1202: SOLR-14149: CHANGES.txt Remove off-topic stuff URL: https://github.com/apache/lucene-solr/pull/1202#issuecomment-578189983 LOL wow I'm ashamed for not knowing the ref guide looks pretty good served from GitHub. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov commented on a change in pull request #1169: LUCENE-9004: A minor feature and patch -- support deleting vector values and fix segments merging
msokolov commented on a change in pull request #1169: LUCENE-9004: A minor
feature and patch -- support deleting vector values and fix segments merging
URL: https://github.com/apache/lucene-solr/pull/1169#discussion_r370709036
##
File path: lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java
##
@@ -92,7 +108,277 @@ public void testSingleDocRecall() throws Exception {
iw.commit();
assertConsistentGraph(iw, values);
- assertRecall(dir, 0, values[0]);
+ assertRecall(dir, 1, values[0]);
+}
+ }
+
+ public void testDocsDeletionAndRecall() throws Exception {
+/**
+ * {@code KnnExactVectorValueWeight} applies in-set (i.e. the query vector
is exactly in the index)
+ * deletion strategy to filter all unmatched results searched by {@link
org.apache.lucene.search.KnnGraphQuery.KnnExactVectorValueQuery},
+ * and deletes at most ef*segmentCnt vectors that are the same to the
specified queryVector.
+ */
+final class KnnExactVectorValueWeight extends ConstantScoreWeight {
+ private final String field;
+ private final ScoreMode scoreMode;
+ private final float[] queryVector;
+ private final int ef;
+
+ KnnExactVectorValueWeight(Query query, float score, ScoreMode scoreMode,
String field, float[] queryVector, int ef) {
+super(query, score);
+this.field = field;
+this.scoreMode = scoreMode;
+this.queryVector = queryVector;
+this.ef = ef;
+ }
+
+ /**
+ * Returns a {@link Scorer} which can iterate in order over all matching
+ * documents and assign them a score.
+ *
+ * NOTE: null can be returned if no documents will be scored by
this
+ * query.
+ *
+ * NOTE: The returned {@link Scorer} does not have
+ * {@link LeafReader#getLiveDocs()} applied, they need to be checked on
top.
+ *
+ * @param context the {@link LeafReaderContext} for which to return the
{@link Scorer}.
+ * @return a {@link Scorer} which scores documents in/out-of order.
+ * @throws IOException if there is a low-level I/O error
+ */
+ @Override
+ public Scorer scorer(LeafReaderContext context) throws IOException {
+ScorerSupplier supplier = scorerSupplier(context);
+if (supplier == null) {
+ return null;
+}
+return supplier.get(Long.MAX_VALUE);
+ }
+
+ @Override
+ public ScorerSupplier scorerSupplier(LeafReaderContext context) throws
IOException {
+FieldInfo fi = context.reader().getFieldInfos().fieldInfo(field);
+int numDimensions = fi.getVectorNumDimensions();
+if (numDimensions != queryVector.length) {
+ throw new IllegalArgumentException("field=\"" + field + "\" was
indexed with dimensions=" + numDimensions +
+ "; this is incompatible with query dimensions=" +
queryVector.length);
+}
+
+final HNSWGraphReader hnswReader = new HNSWGraphReader(field, context);
+final VectorValues vectorValues =
context.reader().getVectorValues(field);
+if (vectorValues == null) {
+ // No docs in this segment/field indexed any vector values
+ return null;
+}
+
+final Weight weight = this;
+return new ScorerSupplier() {
+ @Override
+ public Scorer get(long leadCost) throws IOException {
+final Neighbors neighbors =
hnswReader.searchNeighbors(queryVector, ef, vectorValues);
+
+if (neighbors.size() > 0) {
+ Neighbor top = neighbors.top();
+ if (top.distance() > 0) {
+neighbors.clear();
+ } else {
+final List toDeleteNeighbors = new
ArrayList<>(neighbors.size());
Review comment:
I think I'm having trouble seeing where this is being used: maybe I just
don't understand how to do PRs on github and I'm missing it, but have you
posted a test that uses this query?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-5146) Figure out what it would take for lazily-loaded cores to play nice with SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023078#comment-17023078 ] David Smiley commented on SOLR-5146: I'm going to start work on this as it's a critical feature to support thousands of cores per node. Shawn rightfully points out it's necessary to also load cores super-fast, though I think separate issues should exist. SOLR-14040 for schema sharing is one, and there are smaller ones that have been contributed. In a SolrCloud world I think we'll identify the need to do more. Also out of scope on this specific Jira issue is directly addressing SolrCloud scale of # of collections/shards or whatever. The title says "lazy cores" which suggests only load-on-startup=false and is a rather easy case but I think the scope is actually full "transient" core compatibility, thus on-demand load/*unload* and implies a cache. I want to edit the title to not not have the word "lazy". I expect I'll use the new "Resource Management API" SOLR-13579 for the node-level cache, though I haven't yet dug into the details of that. I tried out the transient core cache in SolrCloud for the heck of it (not expecting it to work) and sure enough, it led to an error. Debugging that further might be a good early step to get a sense of the challenges ahead. Like Erick, I'm hopeful that there's not much technical barrier preventing this feature from "just working" in spite of SolrCloud. SolrCloud will think all the cores on the node are live; the trick is that some are asleep and can be awoken easily, but that distinction needs to be invisible to SolrCloud. I sympathize with Scott's theory that maybe we don't do transient cores and instead flush cashes and perhaps other things. I've thought of that a great deal. It's promising, but you won't quite save as much memory as actually closing the core. If I get stuck with SolrCloud transient core difficulties, I may look at such an alternative. And ultimately we can do both; there aren't mutually exclusive! I'm rather unsatisfied with the implementation of the existing transient core cache. It's weird to me that it's pluggable and has a rather large API surface area for something conceptually straight-forward. I suppose the details are complicated, and I'll have to dig to appreciate those complexities. > Figure out what it would take for lazily-loaded cores to play nice with > SolrCloud > - > > Key: SOLR-5146 > URL: https://issues.apache.org/jira/browse/SOLR-5146 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 4.5, 6.0 >Reporter: Erick Erickson >Assignee: David Smiley >Priority: Major > > The whole lazy-load core thing was implemented with non-SolrCloud use-cases > in mind. There are several user-list threads that ask about using lazy cores > with SolrCloud, especially in multi-tenant use-cases. > This is a marker JIRA to investigate what it would take to make lazy-load > cores play nice with SolrCloud. It's especially interesting how this all > works with shards, replicas, leader election, recovery, etc. > NOTE: This is pretty much totally unexplored territory. It may be that a few > trivial modifications are all that's needed. OTOH, It may be that we'd have > to rip apart SolrCloud to handle this case. Until someone dives into the > code, we don't know. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023134#comment-17023134
]
Dan Fox commented on SOLR-13749:
Good point - I just moved it to the New Features section in the 8.x pull
request.
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be returned.
> This query parser will do an intersection based on join keys between 2
> collections.
> The local collection is the collection that you are searching against.
> The remote collection is the collection that contains the join keys that you
> want to use as a filter.
> Each shard participating in the distributed request will execute a query
> against the remote collection. If the local collection is setup with the
> compositeId router to be routed on the join key field, a hash range query is
> applied to the remote collection query to only match the documents that
> contain a potential match for the documents that are in the local shard/core.
>
>
> Here's some vocab to help with the descriptions of the various parameters.
> ||Term||Description||
> |Local Collection|This is the main collection that is being queried.|
> |Remote Collection|This is the collection that the XCJFQuery will query to
> resolve the join keys.|
> |XCJFQuery|The lucene query that executes a search to get back a set of join
> keys from a remote collection|
> |HashRangeQuery|The lucene query that matches only the documents whose hash
> code on a field falls within a specified range.|
>
>
> ||Param ||Required ||Description||
> |collection|Required|The name of the external Solr collection to be queried
> to retrieve the set of join key values ( required )|
> |zkHost|Optional|The connection string to be used to connect to Zookeeper.
> zkHost and solrUrl are both optional parameters, and at most one of them
> should be specified.
> If neither of zkHost or solrUrl are specified, the local Zookeeper cluster
> will be used. ( optional )|
> |solrUrl|Optional|The URL of the external Solr node to be queried ( optional
> )|
> |from|Required|The join key field name in the external collection ( required
> )|
> |to|Required|The join key field name in the local collection|
> |v|See Note|The query to be executed against the external Solr collection to
> retrieve the set of join key values.
> Note: The original query can be passed at the end of the string or as the
> "v" parameter.
> It's recommended to use query parameter substitution with the "v" parameter
> to ensure no issues arise with the default query parsers.|
> |routed| |true / false. If true, the XCJF query will use each shard's hash
> range to determine the set of join keys to retrieve for that shard.
> This parameter improves the performance of the cross-collection join, but
> it depends on the local collection being routed by the toField. If this
> parameter is not specified,
> the XCJF query will try to determine the correct value automatically.|
> |ttl| |The length of time that an XCJF query in the cache will be considered
> valid, in seconds. Defaults to 3600 (one hour).
> The XCJF query will not be aware of changes to the remote collection, so
> if the remote collection is updated, cached XCJF queries may give inaccurate
> results.
> After the ttl period has expired, the XCJF query will re-execute the join
> against the remote collection.|
> |_All others_| |Any normal Solr parameter can also be specified as a local
> param.|
>
> Example Solr Config.xml changes:
>
> {{<}}{{cache}} {{name}}{{=}}{{"hash_vin"}}
> {{ }}{{class}}{{=}}{{"solr.LRUCache"}}
> {{ }}{{size}}{{=}}{{"128"}}
> {{ }}{{initialSize}}{{=}}{{"0"}}
> {{ }}{{regenerator}}{{=}}{{"solr.NoOpRegenerator"}}{{/>}}
>
> {{<}}{{queryParser}} {{name}}{{=}}{{"xcjf"}}
> {{class}}{{=}}{{"org.apache.solr.search.join.XCJFQueryParserPlugin"}}{{>}}
> {{ }}{{<}}{{str}} {{name}}{{=}}{{"routerField"}}{{>vin}}
> {{}}
>
> {{<}
[GitHub] [lucene-solr] eskatos opened a new pull request #1207: LUCENE-9169: Add Github Workflow for Gradle Wrapper Validation
eskatos opened a new pull request #1207: LUCENE-9169: Add Github Workflow for Gradle Wrapper Validation URL: https://github.com/apache/lucene-solr/pull/1207 # Description This workflow validates the checksums of Gradle Wrapper JAR files present in the source tree and fails if unknown Gradle Wrapper JAR files are found. # Solution See https://github.com/marketplace/actions/gradle-wrapper-validation # Tests N/A # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [x] I have developed this patch against the `master` branch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9169) Gradle Wrapper JAR files checksum should be validated
Paul Merlin created LUCENE-9169: --- Summary: Gradle Wrapper JAR files checksum should be validated Key: LUCENE-9169 URL: https://issues.apache.org/jira/browse/LUCENE-9169 Project: Lucene - Core Issue Type: Task Reporter: Paul Merlin See [https://github.com/marketplace/actions/gradle-wrapper-validation] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9169) Gradle Wrapper JAR files checksum should be validated
[ https://issues.apache.org/jira/browse/LUCENE-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023138#comment-17023138 ] Paul Merlin commented on LUCENE-9169: - Pull request at [https://github.com/apache/lucene-solr/pull/1207] > Gradle Wrapper JAR files checksum should be validated > - > > Key: LUCENE-9169 > URL: https://issues.apache.org/jira/browse/LUCENE-9169 > Project: Lucene - Core > Issue Type: Task >Reporter: Paul Merlin >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > See [https://github.com/marketplace/actions/gradle-wrapper-validation] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12325) introduce uniqueBlockQuery(parent:true) aggregation for JSON Facet
[
https://issues.apache.org/jira/browse/SOLR-12325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023184#comment-17023184
]
Lucene/Solr QA commented on SOLR-12325:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 53m
13s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-12325 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12991504/SOLR-12325.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / f5e9bb94933 |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/662/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/662/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> introduce uniqueBlockQuery(parent:true) aggregation for JSON Facet
> --
>
> Key: SOLR-12325
> URL: https://issues.apache.org/jira/browse/SOLR-12325
> Project: Solr
> Issue Type: New Feature
> Components: Facet Module
>Reporter: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-12325.patch, SOLR-12325.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> It might be faster twin for {{uniqueBlock(\_root_)}}. Please utilise buildin
> query parsing method, don't invent your own.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9170) wagon-ssh Maven HTTPS issue
Ishan Chattopadhyaya created LUCENE-9170: Summary: wagon-ssh Maven HTTPS issue Key: LUCENE-9170 URL: https://issues.apache.org/jira/browse/LUCENE-9170 Project: Lucene - Core Issue Type: Bug Reporter: Ishan Chattopadhyaya When I do, from lucene/ in branch_8_4: ant -Dversion=8.4.2 generate-maven-artifacts I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 instead of https equivalent. This is surprising to me, since I can't find the http URL anywhere. Here's my log: https://paste.centos.org/view/be2d3f3f This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated LUCENE-9170: - Attachment: LUCENE-9170.patch Assignee: Ishan Chattopadhyaya Status: Open (was: Open) Here's a patch for this. Would appreciate if someone can please review. > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Attachments: LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss merged pull request #1207: LUCENE-9169: Add Github Workflow for Gradle Wrapper Validation
dweiss merged pull request #1207: LUCENE-9169: Add Github Workflow for Gradle Wrapper Validation URL: https://github.com/apache/lucene-solr/pull/1207 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob merged pull request #1137: SOLR-14162 TestInjection can leak Timer objects
madrob merged pull request #1137: SOLR-14162 TestInjection can leak Timer objects URL: https://github.com/apache/lucene-solr/pull/1137 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14162) TestInjection can leak Timer objects
[ https://issues.apache.org/jira/browse/SOLR-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023232#comment-17023232 ] ASF subversion and git services commented on SOLR-14162: Commit eaa3dbe4408790feaf4ee7104422b2fa2111e2d5 in lucene-solr's branch refs/heads/master from Mike [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=eaa3dbe ] SOLR-14162 TestInjection can leak Timer objects (#1137) > TestInjection can leak Timer objects > > > Key: SOLR-14162 > URL: https://issues.apache.org/jira/browse/SOLR-14162 > Project: Solr > Issue Type: Bug > Components: Tests >Reporter: Mike Drob >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > In TestInjection we track all of the outstanding timers for shutdown but try > to clean up based on the TimerTask instead of the Timer itself. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated LUCENE-9170: - Attachment: LUCENE-9170.patch Status: Open (was: Open) > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Attachments: LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated LUCENE-9170: - Priority: Blocker (was: Major) > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Attachments: LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated LUCENE-9170: - Fix Version/s: 8.5 > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Fix For: 8.5 > > Attachments: LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12930) Add developer documentation to source repo
[ https://issues.apache.org/jira/browse/SOLR-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023282#comment-17023282 ] Cassandra Targett commented on SOLR-12930: -- OK, I'm not sure how being in a different directory than the one they were in is "burying" them - there's literally one other file there right now. If I approached the project fresh I don't know I would get that the "help" directory had build information I need, but I won't assume my experience is indicative of anyone else's. I made the suggestion over a week ago to move those files as I proposed and no one said there was any issue with doing it. You asked, Dawid, in your email to me that I revert the commit that moved them, so I'll do that, but I'll confess to not really understanding the why two top-level documentation directories is better than one. > Add developer documentation to source repo > -- > > Key: SOLR-12930 > URL: https://issues.apache.org/jira/browse/SOLR-12930 > Project: Solr > Issue Type: Improvement > Components: Tests >Reporter: Mark Miller >Priority: Major > Attachments: solr-dev-docs.zip > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12930) Add developer documentation to source repo
[ https://issues.apache.org/jira/browse/SOLR-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023282#comment-17023282 ] Cassandra Targett edited comment on SOLR-12930 at 1/24/20 9:51 PM: --- OK, I'm not sure how being in a different directory than the one they were in is "burying" them - there's literally one other file there right now. If I approached the project fresh I don't know I would get that the "help" directory had build information I need, but I won't assume my experience is indicative of anyone else's. I made the suggestion over a week ago to move those files as I proposed and no one said there was any issue with doing it. You asked, Dawid, in your email to me that I revert the commit that moved them, so I'll do that, but I'll confess to not really understanding why two top-level documentation directories is better than one. was (Author: ctargett): OK, I'm not sure how being in a different directory than the one they were in is "burying" them - there's literally one other file there right now. If I approached the project fresh I don't know I would get that the "help" directory had build information I need, but I won't assume my experience is indicative of anyone else's. I made the suggestion over a week ago to move those files as I proposed and no one said there was any issue with doing it. You asked, Dawid, in your email to me that I revert the commit that moved them, so I'll do that, but I'll confess to not really understanding the why two top-level documentation directories is better than one. > Add developer documentation to source repo > -- > > Key: SOLR-12930 > URL: https://issues.apache.org/jira/browse/SOLR-12930 > Project: Solr > Issue Type: Improvement > Components: Tests >Reporter: Mark Miller >Priority: Major > Attachments: solr-dev-docs.zip > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12930) Add developer documentation to source repo
[ https://issues.apache.org/jira/browse/SOLR-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023282#comment-17023282 ] Cassandra Targett edited comment on SOLR-12930 at 1/24/20 9:53 PM: --- OK, I'm not sure how being in a different directory than the one they were in is "burying" them - there's literally one other file there right now. If I approached the project fresh I don't know I would get that the "help" directory had build information I need, but I won't assume my experience is indicative of anyone else's. I made the suggestion over a week ago to move those files as I proposed and no one said there was any issue with doing it. You asked, Dawid, in your email to me that I revert the commit that moved them, so I'll do that, but I'll confess to not really understanding why two top-level documentation directories is better than one. was (Author: ctargett): OK, I'm not sure how being in a different directory than the one they were in is "burying" them - there's literally one other file there right now. If I approached the project fresh I don't know I would get that the "help" directory had build information I need, but I won't assume my experience is indicative of anyone else's. I made the suggestion over a week ago to move those files as I proposed and no one said there was any issue with doing it. You asked, Dawid, in your email to me that I revert the commit that moved them, so I'll do that, but I'll confess to not really understanding why two top-level documentation directories is better than one. > Add developer documentation to source repo > -- > > Key: SOLR-12930 > URL: https://issues.apache.org/jira/browse/SOLR-12930 > Project: Solr > Issue Type: Improvement > Components: Tests >Reporter: Mark Miller >Priority: Major > Attachments: solr-dev-docs.zip > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12930) Add developer documentation to source repo
[ https://issues.apache.org/jira/browse/SOLR-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023282#comment-17023282 ] Cassandra Targett edited comment on SOLR-12930 at 1/24/20 9:53 PM: --- OK, I'm not sure how being in a different directory than the one they were in is "burying" them - there's literally one other file there right now. If I approached the project fresh I don't know I would get that the "help" directory had build information I need, but I won't assume my experience is indicative of anyone else's. I made the suggestion over a week ago to move those files as I did and no one said there was any issue with doing it. You asked, Dawid, in your email to me that I revert the commit that moved them, so I'll do that, but I'll confess to not really understanding why two top-level documentation directories is better than one. was (Author: ctargett): OK, I'm not sure how being in a different directory than the one they were in is "burying" them - there's literally one other file there right now. If I approached the project fresh I don't know I would get that the "help" directory had build information I need, but I won't assume my experience is indicative of anyone else's. I made the suggestion over a week ago to move those files as I proposed and no one said there was any issue with doing it. You asked, Dawid, in your email to me that I revert the commit that moved them, so I'll do that, but I'll confess to not really understanding why two top-level documentation directories is better than one. > Add developer documentation to source repo > -- > > Key: SOLR-12930 > URL: https://issues.apache.org/jira/browse/SOLR-12930 > Project: Solr > Issue Type: Improvement > Components: Tests >Reporter: Mark Miller >Priority: Major > Attachments: solr-dev-docs.zip > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14214) Ref Guide: Clean up info about clients other than SolrJ
[ https://issues.apache.org/jira/browse/SOLR-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023284#comment-17023284 ] ASF subversion and git services commented on SOLR-14214: Commit ba77a5f2eb13ffb418b84dac1df957dc3e9e2247 in lucene-solr's branch refs/heads/master from Cassandra Targett [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ba77a5f ] SOLR-14214: Clean up client lists and references > Ref Guide: Clean up info about clients other than SolrJ > --- > > Key: SOLR-14214 > URL: https://issues.apache.org/jira/browse/SOLR-14214 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Priority: Major > > The Ref Guide page client-api-lineup.adoc may have been updated at some point > since Nov 2011, the last time it says it was updated, but I would guess > probably not very recently. > It really would be worth going through the list to see which ones are still > active and removing those that would not work with modern versions of Solr > (say, 6.x or 7.x+?). > My personal POV is that all info on clients should be kept in the Wiki > (cwiki) and the Ref Guide merely link to that - that would allow client > maintainers to keep info about their clients up to date without needing to be > a committer in order to update the Ref Guide. > That approach would mean pretty much removing everything from the > client-api-lineup.adoc page, and also likely removing most if not all of the > other Client pages for Ruby, Python, and JS. > However it plays out, we should take a look at those pages and update > according to the current state of the client universe. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12930) Add developer documentation to source repo
[ https://issues.apache.org/jira/browse/SOLR-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023285#comment-17023285 ] ASF subversion and git services commented on SOLR-12930: Commit 74e88deba78ea40f81c3072d6e014903773f4e92 in lucene-solr's branch refs/heads/master from Cassandra Targett [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=74e88de ] Revert "SOLR-12930: move Gradle docs from ./help/ to new ./dev-docs/ directory" This reverts commit 2d8650d36cc65b3161f009be85fcfd2fa8ff637c. > Add developer documentation to source repo > -- > > Key: SOLR-12930 > URL: https://issues.apache.org/jira/browse/SOLR-12930 > Project: Solr > Issue Type: Improvement > Components: Tests >Reporter: Mark Miller >Priority: Major > Attachments: solr-dev-docs.zip > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12930) Add developer documentation to source repo
[ https://issues.apache.org/jira/browse/SOLR-12930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023286#comment-17023286 ] Cassandra Targett commented on SOLR-12930: -- OK, I reverted the help files move - sorry about the confusion there. > Add developer documentation to source repo > -- > > Key: SOLR-12930 > URL: https://issues.apache.org/jira/browse/SOLR-12930 > Project: Solr > Issue Type: Improvement > Components: Tests >Reporter: Mark Miller >Priority: Major > Attachments: solr-dev-docs.zip > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14214) Ref Guide: Clean up info about clients other than SolrJ
[ https://issues.apache.org/jira/browse/SOLR-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023288#comment-17023288 ] ASF subversion and git services commented on SOLR-14214: Commit 6501577eb5244290cbe5c73e90150f4c40a6c0bd in lucene-solr's branch refs/heads/branch_8x from Cassandra Targett [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6501577 ] SOLR-14214: Clean up client lists and references > Ref Guide: Clean up info about clients other than SolrJ > --- > > Key: SOLR-14214 > URL: https://issues.apache.org/jira/browse/SOLR-14214 > Project: Solr > Issue Type: Improvement > Components: documentation >Reporter: Cassandra Targett >Priority: Major > > The Ref Guide page client-api-lineup.adoc may have been updated at some point > since Nov 2011, the last time it says it was updated, but I would guess > probably not very recently. > It really would be worth going through the list to see which ones are still > active and removing those that would not work with modern versions of Solr > (say, 6.x or 7.x+?). > My personal POV is that all info on clients should be kept in the Wiki > (cwiki) and the Ref Guide merely link to that - that would allow client > maintainers to keep info about their clients up to date without needing to be > a committer in order to update the Ref Guide. > That approach would mean pretty much removing everything from the > client-api-lineup.adoc page, and also likely removing most if not all of the > other Client pages for Ruby, Python, and JS. > However it plays out, we should take a look at those pages and update > according to the current state of the client universe. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370856696
##
File path: lucene/core/build.gradle
##
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl =
"https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip";
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip";
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
+ant.unzip(src: file("${buildDir}/jflex.zip"), dest:
file("${buildDir}/jflex"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+}
+ }
+}
+
+task installMoman(type: Download) {
Review comment:
Same comments apply here as for `installJFlex`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370858176
##
File path: lucene/core/build.gradle
##
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl =
"https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip";
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip";
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
+ant.unzip(src: file("${buildDir}/jflex.zip"), dest:
file("${buildDir}/jflex"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+}
+ }
+}
+
+task installMoman(type: Download) {
+ src momanUrl
+ dest file("${buildDir}/moman.zip")
+ onlyIfModified true
+
+ doLast {
+ant.unzip(src: file("${buildDir}/moman.zip"), dest:
file("${buildDir}/moman"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+}
+ }
+}
+
+task createPackedIntSources {
+ doLast {
+exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_BulkOperation.py']
+}
+exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_Packed64SingleBlock.py']
+}
+ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes:
'Packed64SingleBlock.java', encoding: 'UTF-8')
+ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes:
'BulkOperation*.java', encoding: 'UTF-8')
+ }
+}
+
+task createLevAutomaton(dependsOn: installMoman) {
+ doLast {
+exec {
Review comment:
```
[1, 2].each { edits ->
[True, False].each { transpose ->
exec {
workingDir...
executable...
args = ['-B', createLevAutomata.py, edits, transpose]
}
}
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370860485
##
File path:
lucene/queryparser/src/java/org/apache/lucene/queryparser/classic/QueryParser.java
##
@@ -879,7 +878,7 @@ private void jj_add_error_token(int kind, int pos) {
for (int i = 0; i < jj_endpos; i++) {
jj_expentry[i] = jj_lasttokens[i];
}
- jj_entries_loop: for (java.util.Iterator it = jj_expentries.iterator();
it.hasNext();) {
Review comment:
This needs to be a wildcard generic and not `int[]`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370858964
##
File path: lucene/core/build.gradle
##
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl =
"https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip";
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip";
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
+ant.unzip(src: file("${buildDir}/jflex.zip"), dest:
file("${buildDir}/jflex"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+}
+ }
+}
+
+task installMoman(type: Download) {
+ src momanUrl
+ dest file("${buildDir}/moman.zip")
+ onlyIfModified true
+
+ doLast {
+ant.unzip(src: file("${buildDir}/moman.zip"), dest:
file("${buildDir}/moman"), overwrite: "true") {
+ ant.cutdirsmapper(dirs: "1")
+}
+ }
+}
+
+task createPackedIntSources {
+ doLast {
+exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_BulkOperation.py']
+}
+exec {
+ workingDir 'src/java/org/apache/lucene/util/packed'
+ executable "${python_exe}"
+ args = ['-B', 'gen_Packed64SingleBlock.py']
+}
+ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes:
'Packed64SingleBlock.java', encoding: 'UTF-8')
+ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/packed', includes:
'BulkOperation*.java', encoding: 'UTF-8')
+ }
+}
+
+task createLevAutomaton(dependsOn: installMoman) {
+ doLast {
+exec {
+ workingDir 'src/java/org/apache/lucene/util/automaton'
+ executable "${python_exe}"
+ args = ['-B', 'createLevAutomata.py', '1', 'True']
+}
+exec {
+ workingDir 'src/java/org/apache/lucene/util/automaton'
+ executable "${python_exe}"
+ args = ['-B', 'createLevAutomata.py', '1', 'False']
+}
+exec {
+ workingDir 'src/java/org/apache/lucene/util/automaton'
+ executable "${python_exe}"
+ args = ['-B', 'createLevAutomata.py', '2', 'True']
+}
+exec {
+ workingDir 'src/java/org/apache/lucene/util/automaton'
+ executable "${python_exe}"
+ args = ['-B', 'createLevAutomata.py', '2', 'False']
+}
+ant.fixcrlf(srcdir: 'src/java/org/apache/lucene/util/automaton', includes:
'*ParametricDescription.java', encoding: 'UTF-8')
+ }
+}
+
+task runJFlex(dependsOn: installJFlex, type: JavaExec) {
+ classpath = files("${buildDir}/jflex/lib/jflex-full-1.7.0.jar")
+ main = "jflex.Main"
+
+ // arguments to pass to the application
+ args = ['--skel',
project.project(":lucene:core").projectDir.getAbsolutePath() +
"/src/data/jflex/skeleton.disable.buffer.expansion.txt",
+ '--nobak',
+
'src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.jflex'
+ ]
+ doLast {
+//Since the ZZ_BUFFERSIZE declaration is generated rather than in the
skeleton, we have to transform it here.
+ant.replace(token: "private static final int ZZ_BUFFERSIZE", value:
"private int ZZ_BUFFERSIZE", encoding: "UTF-8") {
+ ant.fileset(dir: "src/java/org/apache/lucene/analysis/standard",
includes: "StandardTokenizerImpl.java")
+}
+ }
+}
+
+task cleanJFlex {
Review comment:
if you define a `jflex` task that has registered outputs, then gradle will
automatically create a corresponding `cleanJflex` task for you.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370856887
##
File path: lucene/core/build.gradle
##
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl =
"https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip";
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip";
+
+// Download and extract Moman
Review comment:
nit: s/Moman/JFlex
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370861960
##
File path: lucene/queryparser/build.gradle
##
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+javaCCDeps
+
+dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+}
+ }
+}
+def conf = [
+"queryParser": ["inputFile":
file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'),
"outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+"surround" : ["inputFile":
file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'),
"outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+"flexible" : ["inputFile":
file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'),
"outputDir":
file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+javaCCClasspath =
project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+String hacky = javaCCHome + "/" + "javacc.jar"
Review comment:
I don't understand what is going on here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370855505
##
File path: lucene/core/build.gradle
##
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl =
"https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip";
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip";
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
Review comment:
I think this can just be `dest buildDir`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370856535
##
File path: lucene/core/build.gradle
##
@@ -1,7 +1,117 @@
-
apply plugin: 'java-library'
+apply plugin: "de.undercouch.download"
dependencies {
testImplementation project(':lucene:codecs')
testImplementation project(':lucene:test-framework')
}
+
+def momanCommitHash = "5c5c2a1e4dea"
+def momanUrl =
"https://bitbucket.org/jpbarrette/moman/get/${momanCommitHash}.zip";
+def python_exe = "python"
+
+def jflexUrl = "https://jflex.de/release/jflex-1.7.0.zip";
+
+// Download and extract Moman
+task installJFlex(type: Download) {
+ src jflexUrl
+ dest file("${buildDir}/jflex.zip")
+ onlyIfModified true
+ doLast {
Review comment:
I would create a separate Copy task for this part and use Gradle's built in
zip support. Example:
https://github.com/michel-kraemer/gradle-download-task#download-and-extract-a-zip-file
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370864083
##
File path: lucene/queryparser/build.gradle
##
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+javaCCDeps
+
+dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+}
+ }
+}
+def conf = [
+"queryParser": ["inputFile":
file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'),
"outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+"surround" : ["inputFile":
file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'),
"outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+"flexible" : ["inputFile":
file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'),
"outputDir":
file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+javaCCClasspath =
project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+String hacky = javaCCHome + "/" + "javacc.jar"
+File hackyFile = new File(hacky)
+println hackyFile
+println hackyFile.exists()
+if (hackyFile.exists() == false) {
+ hackyFile.bytes = new File(javaCCClasspath).bytes
+ hackyFile.setExecutable(true)
+}
+ }
+}
+String lineSeparator = System.lineSeparator()
+
+task cleanOldGeneratedFiles {
+ doLast {
+println "Removing old generated files"
+conf.each { key, val ->
+ val["outputDir"].eachFileMatch ~/.*\.java/, { f ->
+if (f.text =~ /Generated.*By.*JavaCC/) {
+ f.delete()
+}
+ }
+}
+ }
+}
+
+task runJavaccQueryParser(dependsOn: installJavacc) {
Review comment:
I think because there is so much commonality between all of the javacc
tasks, I would create a `task JavaCC` class and put all the clean up logic in
one place rather than repeating it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] madrob commented on a change in pull request #1199: LUCENE-9134: Port ant-regenerate tasks to Gradle build
madrob commented on a change in pull request #1199: LUCENE-9134: Port
ant-regenerate tasks to Gradle build
URL: https://github.com/apache/lucene-solr/pull/1199#discussion_r370863721
##
File path: lucene/queryparser/build.gradle
##
@@ -7,3 +7,305 @@ dependencies {
testImplementation project(':lucene:test-framework')
}
+
+configure(":lucene:queryparser") {
+ configurations {
+javaCCDeps
+
+dependencies {
+ javaCCDeps "net.java.dev.javacc:javacc:5.0"
+}
+ }
+}
+def conf = [
+"queryParser": ["inputFile":
file('src/java/org/apache/lucene/queryparser/classic/QueryParser.jj'),
"outputDir": file('src/java/org/apache/lucene/queryparser/classic')],
+"surround" : ["inputFile":
file('src/java/org/apache/lucene/queryparser/surround/parser/QueryParser.jj'),
"outputDir": file('src/java/org/apache/lucene/queryparser/surround/parser/')],
+"flexible" : ["inputFile":
file('src/java/org/apache/lucene/queryparser/flexible/standard/parser/StandardSyntaxParser.jj'),
"outputDir":
file('src/java/org/apache/lucene/queryparser/flexible/standard/parser')]
+]
+
+String javaCCClasspath
+String javaCCHome
+
+task installJavacc {
+ doLast {
+javaCCClasspath =
project.project(":lucene:queryparser").configurations.javaCCDeps.asPath
+javaCCHome = javaCCClasspath.substring(0, javaCCClasspath.lastIndexOf("/"))
+
+String hacky = javaCCHome + "/" + "javacc.jar"
+File hackyFile = new File(hacky)
+println hackyFile
+println hackyFile.exists()
+if (hackyFile.exists() == false) {
+ hackyFile.bytes = new File(javaCCClasspath).bytes
+ hackyFile.setExecutable(true)
+}
+ }
+}
+String lineSeparator = System.lineSeparator()
+
+task cleanOldGeneratedFiles {
+ doLast {
+println "Removing old generated files"
+conf.each { key, val ->
+ val["outputDir"].eachFileMatch ~/.*\.java/, { f ->
+if (f.text =~ /Generated.*By.*JavaCC/) {
+ f.delete()
+}
+ }
+}
+ }
+}
+
+task runJavaccQueryParser(dependsOn: installJavacc) {
+ doLast {
+println "Generating QueryParser parser"
+
+def inputFile = conf["queryParser"]["inputFile"].getAbsolutePath()
+def outputDir = conf["queryParser"]["outputDir"].getAbsolutePath()
+ant.javacc(target: inputFile, outputDirectory: outputDir, javacchome:
javaCCHome)
+
+
+ant.replaceregexp(file: new File(outputDir, "QueryParser.java"),
+byline: "true",
+match: "public QueryParser\\(CharStream ",
+replace: "protected QueryParser(CharStream ")
+ant.replaceregexp(file: new File(outputDir, "QueryParser.java"),
+byline: "true",
+match: "public QueryParser\\(QueryParserTokenManager ",
+replace: "protected QueryParser(QueryParserTokenManager ")
+ }
+}
+
+task runJavaccSurround(dependsOn: installJavacc) {
+ doLast {
+println "Generating Surroundf parser"
+def inputFile = conf["surround"]["inputFile"].getAbsolutePath()
+def outputDir = conf["surround"]["outputDir"].getAbsolutePath()
+
+ant.javacc(target: inputFile, outputDirectory: outputDir, javacchome:
javaCCHome)
+
+// This is very clumsy, but these are unused imports that are necessary
other generated files
+[
+"import java.util.Vector;",
+"import org.apache.lucene.queryparser.flexible.messages.Message;",
+"import org.apache.lucene.queryparser.flexible.messages.MessageImpl;",
+"import
org.apache.lucene.queryparser.flexible.core.QueryNodeParseException;",
+"import
org.apache.lucene.queryparser.flexible.core.messages.QueryParserMessages;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.AndQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.BooleanQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.BoostQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.FieldQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.FuzzyQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.ModifierQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.GroupQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.OrQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.standard.nodes.RegexpQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.SlopQueryNode;",
+"import org.apache.lucene.queryparser.flexible.core.nodes.QueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.nodes.QuotedFieldQueryNode;",
+"import
org.apache.lucene.queryparser.flexible.core.parser.SyntaxParser;",
+"import
org.apache.lucene.queryparser.flexible.standard.nodes.TermRangeQueryNode;",
+"import java.io.StringReader;",
+"import java.util.ArrayList;",
+"import java.util.Arrays;",
+"import java.util.HashSet;",
+"i
[jira] [Created] (SOLR-14216) Exclude HealthCheck from authentication
Jan Høydahl created SOLR-14216:
--
Summary: Exclude HealthCheck from authentication
Key: SOLR-14216
URL: https://issues.apache.org/jira/browse/SOLR-14216
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: Authentication
Reporter: Jan Høydahl
The {{HealthCheckHandler}} on {{/api/node/health}} and
{{/solr/admin/info/health}} should by default not be subject to authentication,
but be open for all. This allows for load balancers and various monitoring to
probe Solr's health without having to support the auth scheme in place. I can't
see any reason we need auth on the health endpoint.
It is possible to achieve the same by setting blockUnknown=false and
configuring three RBAC permissions: One for v1 endpoint, one for v2 endpoint
and one "all" catch all at the end of the chain. But this is cumbersome so
better have this ootb.
An alternative solution is to create a separate HttpServer for health check,
listening on a different port, just like embedded ZK and JMX.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14210) Introduce Node-level status handler for replicas
[
https://issues.apache.org/jira/browse/SOLR-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023298#comment-17023298
]
Jan Høydahl commented on SOLR-14210:
I created SOLR-14216 for turning off authentication for healt check.
> Introduce Node-level status handler for replicas
>
>
> Key: SOLR-14210
> URL: https://issues.apache.org/jira/browse/SOLR-14210
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: master (9.0), 8.5
>Reporter: Houston Putman
>Priority: Major
>
> h2. Background
> As was brought up in SOLR-13055, in order to run Solr in a more cloud-native
> way, we need some additional features around node-level healthchecks.
> {quote}Like in Kubernetes we need 'liveliness' and 'readiness' probe
> explained in
> [https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/n]
> determine if a node is live and ready to serve live traffic.
> {quote}
>
> However there are issues around kubernetes managing it's own rolling
> restarts. With the current healthcheck setup, it's easy to envision a
> scenario in which Solr reports itself as "healthy" when all of its replicas
> are actually recovering. Therefore kubernetes, seeing a healthy pod would
> then go and restart the next Solr node. This can happen until all replicas
> are "recovering" and none are healthy. (maybe the last one restarted will be
> "down", but still there are no "active" replicas)
> h2. Proposal
> I propose we make an additional healthcheck handler that returns whether all
> replicas hosted by that Solr node are healthy and "active". That way we will
> be able to use the [default kubernetes rolling restart
> logic|https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies]
> with Solr.
> To add on to [Jan's point
> here|https://issues.apache.org/jira/browse/SOLR-13055?focusedCommentId=16716559&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16716559],
> this handler should be more friendly for other Content-Types and should use
> bettter HTTP response statuses.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9170) wagon-ssh Maven HTTPS issue
[ https://issues.apache.org/jira/browse/LUCENE-9170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023313#comment-17023313 ] Jan Høydahl commented on LUCENE-9170: - Good catch. LGTM > wagon-ssh Maven HTTPS issue > --- > > Key: LUCENE-9170 > URL: https://issues.apache.org/jira/browse/LUCENE-9170 > Project: Lucene - Core > Issue Type: Bug >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Fix For: 8.5 > > Attachments: LUCENE-9170.patch, LUCENE-9170.patch > > > When I do, from lucene/ in branch_8_4: > ant -Dversion=8.4.2 generate-maven-artifacts > I see that wagon-ssh is being resolved from http://repo1.maven.org/maven2 > instead of https equivalent. This is surprising to me, since I can't find the > http URL anywhere. > Here's my log: > https://paste.centos.org/view/be2d3f3f > This is a critical issue since releases won't work without this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ebehrendt opened a new pull request #1208: SOLR-13101: Convert nanotime to ms
ebehrendt opened a new pull request #1208: SOLR-13101: Convert nanotime to ms URL: https://github.com/apache/lucene-solr/pull/1208 # Description Continue logging ms, but use System.nanoTime instead of System.currentTimeMillis for more precise measurement of elapsed time. # Solution Divide nanoseconds by 100 to get milliseconds. Converting nanoTime to milliseconds is more precise than using currentTimeMillis because it does not use the system clock time which could face disruptions # Tests Ran ant test and manually examined loglines This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ebehrendt commented on issue #1208: SOLR-13101: Convert nanotime to ms
ebehrendt commented on issue #1208: SOLR-13101: Convert nanotime to ms URL: https://github.com/apache/lucene-solr/pull/1208#issuecomment-578337408 FYI @andyvuong This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ebehrendt commented on a change in pull request #1208: SOLR-13101: Convert nanotime to ms
ebehrendt commented on a change in pull request #1208: SOLR-13101: Convert
nanotime to ms
URL: https://github.com/apache/lucene-solr/pull/1208#discussion_r370881326
##
File path:
solr/core/src/java/org/apache/solr/store/blob/metadata/CorePushPull.java
##
@@ -499,7 +500,7 @@ void enqueueForHardDelete(BlobCoreMetadataBuilder
bcmBuilder) throws Exception {
@VisibleForTesting
protected boolean okForHardDelete(BlobCoreMetadata.BlobFileToDelete file) {
// For now we only check how long ago the file was marked for delete.
- return System.nanoTime() - file.getDeletedAt() >=
deleteManager.getDeleteDelayMs();
+ return (System.nanoTime() / 100) - file.getDeletedAt() >=
deleteManager.getDeleteDelayMs();
Review comment:
From what I could find, deletedAt and deleteDelay is only set using
System.nanoTime/100, so there should not be any mismatch from comparing
times that used nanoTime and times that used currentTimeMillis
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on a change in pull request #1121: SOLR-11207: Add OWASP dependency checker to gradle build
janhoy commented on a change in pull request #1121: SOLR-11207: Add OWASP
dependency checker to gradle build
URL: https://github.com/apache/lucene-solr/pull/1121#discussion_r370888355
##
File path: gradle/validation/dependency-check.gradle
##
@@ -0,0 +1,14 @@
+// This adds OWASP vulnerability validation of project dependencies
+// Not part of 'check' task by default, must be called explicitly, e.g.
gradlew dependencyCheckAnalyze
+// Start build with -Pvalidation.owasp.fail=true to fail build on owasp errors
(CVSS >= 7)
+// Start build with -Pvalidation.owasp.skip=true to skip OWASP checks during
'check' phase
+
+dependencyCheck {
Review comment:
> Task 'dependencyCheckAnalyze' not found in project ':lucene:core'.
Will wrap it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on issue #1121: SOLR-11207: Add OWASP dependency checker to gradle build
janhoy commented on issue #1121: SOLR-11207: Add OWASP dependency checker to gradle build URL: https://github.com/apache/lucene-solr/pull/1121#issuecomment-578346092 Documentation added to workflow.txt File renamed to owasp-dependency-check Task included in 'check' by default, but skipped since property `-Pvalidation.owasp=[true|false]` defaults to false for now. Caller can choose to override cvss threshold with `-Pvalidation.owasp.threshold=` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] andyvuong commented on a change in pull request #1208: SOLR-13101: Convert nanotime to ms
andyvuong commented on a change in pull request #1208: SOLR-13101: Convert
nanotime to ms
URL: https://github.com/apache/lucene-solr/pull/1208#discussion_r370891913
##
File path:
solr/core/src/java/org/apache/solr/store/blob/metadata/CorePushPull.java
##
@@ -114,7 +114,7 @@ protected CoreContainer
getContainerFromServerMetadata(ServerSideMetadata solrSe
* @param newMetadataSuffix suffix of the new core.metadata file to be
created as part of this push
*/
public BlobCoreMetadata pushToBlobStore(String currentMetadataSuffix,
String newMetadataSuffix) throws Exception {
- long startTimeMs = System.nanoTime();
+ long startTimeMs = System.nanoTime() / 100;
Review comment:
I'd consider making System.nanoTime() / 100 a util method in
BlobStoreUtils
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] andyvuong commented on a change in pull request #1208: SOLR-13101: Convert nanotime to ms
andyvuong commented on a change in pull request #1208: SOLR-13101: Convert
nanotime to ms
URL: https://github.com/apache/lucene-solr/pull/1208#discussion_r370891787
##
File path:
solr/core/src/java/org/apache/solr/store/blob/metadata/CorePushPull.java
##
@@ -499,7 +500,7 @@ void enqueueForHardDelete(BlobCoreMetadataBuilder
bcmBuilder) throws Exception {
@VisibleForTesting
protected boolean okForHardDelete(BlobCoreMetadata.BlobFileToDelete file) {
// For now we only check how long ago the file was marked for delete.
- return System.nanoTime() - file.getDeletedAt() >=
deleteManager.getDeleteDelayMs();
+ return (System.nanoTime() / 100) - file.getDeletedAt() >=
deleteManager.getDeleteDelayMs();
Review comment:
It might be a good idea to update the BlobFileToDelete class to make it
explicit via the javadoc (actually it doesn't have any) that the getDeletedAt()
returns the timestamp in milliseconds
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] janhoy commented on issue #1207: LUCENE-9169: Add Github Workflow for Gradle Wrapper Validation
janhoy commented on issue #1207: LUCENE-9169: Add Github Workflow for Gradle Wrapper Validation URL: https://github.com/apache/lucene-solr/pull/1207#issuecomment-578349184 This workflow was triggered on a push to my fork (https://github.com/cominvent/lucene-solr/commit/9bae9c71ef632e141adedadc7ccfccabe2ca353d/checks?check_suite_id=419459737) and I got an error email. Can you disable for forks? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14194) Allow Highlighting to work for indexes with uniqueKey that is not stored
[
https://issues.apache.org/jira/browse/SOLR-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023362#comment-17023362
]
Lucene/Solr QA commented on SOLR-14194:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 24s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 84m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.schema.DateRangeFieldTest |
| | solr.search.facet.RangeFacetCloudTest |
| | solr.search.facet.DistributedFacetSimpleRefinementLongTailTest |
| | solr.cloud.TestSegmentSorting |
| | solr.schema.CurrencyFieldTypeTest |
| | solr.cloud.HttpPartitionTest |
| | solr.update.processor.AtomicUpdatesTest |
| | solr.schema.IndexSchemaRuntimeFieldTest |
| | solr.search.stats.TestExactStatsCache |
| | solr.cloud.BasicZkTest |
| | solr.update.PeerSyncWithLeaderTest |
| | solr.schema.IndexSchemaTest |
| | solr.search.function.TestSortByMinMaxFunction |
| | solr.handler.component.DistributedFacetPivotSmallTest |
| | solr.handler.component.MoreLikeThisComponentTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-14194 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12991755/SOLR-14194.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-170-generic #199-Ubuntu SMP
Thu Nov 14 01:45:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 74e88de |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/663/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/663/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/663/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Allow Highlighting to work for indexes with uniqueKey that is not stored
>
>
> Key: SOLR-14194
> URL: https://issues.apache.org/jira/browse/SOLR-14194
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
>Affects Versions: master (9.0)
>Reporter: Andrzej Wislowski
>Assignee: David Smiley
>Priority: Minor
> Labels: highlighter
> Fix For: master (9.0)
>
> Attachments: SOLR-14194.patch, SOLR-14194.patch
>
>
> Highlighting requires uniqueKey to be a stored field. I have changed
> Highlighter allow returning results on indexes with uniqueKey that is a not
> stored field, but saved as a docvalue type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gerlowskija commented on a change in pull request #1171: SOLR-13892: Add 'top-level' docValues Join implementation
gerlowskija commented on a change in pull request #1171: SOLR-13892: Add
'top-level' docValues Join implementation
URL: https://github.com/apache/lucene-solr/pull/1171#discussion_r370910102
##
File path:
solr/core/src/test/org/apache/solr/search/TestJoinQueryPerformance.java
##
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.search;
+
+import java.io.BufferedReader;
+import java.io.FileReader;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.UUID;
+
+import com.google.common.collect.Lists;
+import org.apache.solr.SolrTestCaseJ4;
+import org.apache.solr.client.solrj.SolrQuery;
+import org.apache.solr.client.solrj.SolrRequest;
+import org.apache.solr.client.solrj.impl.HttpSolrClient;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.common.SolrInputDocument;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+
+public class TestJoinQueryPerformance extends SolrTestCaseJ4 {
+ // Dictionary used to load String data
+ private static final String DICT_PATH = "/usr/share/dict/words";
+ private static final int NUM_DICT_TERMS = 235886;
+ private static final String[] LOADED_DICTIONARY = new String[NUM_DICT_TERMS];
+
+ // Performance run parameters: Indexing
+ private static final String FROM_COLLECTION_NAME = "user_acls";
+ private static final int NUM_FROM_DOCS = 5050; // 1 + 2 + 3 + 4 + ... + 100
+ private static final String TO_COLLECTION_NAME = "products";
+ private static final int NUM_TO_DOCS = 50;
+ private static final int PERMISSION_CARDINALITY = 5; // 50K unique
groups/roles/whatever
+ private static int BATCH_SIZE = 500;
+ private static int NUM_COMMITS = 500;
+ private static final int VAL_MAX = 1000;
+ private static final int USER_MAX = 100;
+
+ private static String COLLECTION_NAME= "foo";
+
+ /*
+ * As I start out here, I think I'll want a few different axes.
+ * - "from" collection matches (with "to" matches held constant)
+ * - "to" collection matches (with "from" matches held constant)
+ *
+ * So I think I should index a finite number of docs
+ */
+
+ @BeforeClass
+ public static void setUpCluster() throws Exception {
+loadDictionary();
+//loadCollectionData(DType.USER);
+//loadCollectionData(DType.DATA);
+ }
+
+ private static void loadDictionary() throws Exception {
+try (BufferedReader reader = new BufferedReader(new
FileReader(DICT_PATH))) {
+ for (int i = 0; i < NUM_DICT_TERMS; i++) {
+LOADED_DICTIONARY[i] = reader.readLine();
+ }
+}
+ }
+
+ public enum DType {
+USER(NUM_FROM_DOCS, FROM_COLLECTION_NAME) {
+ // id - unique string
+ // userid_s - username (user# from 1-100)...each user appears in #
entries
+ // permissions_ss - set of 300 string permissions (cardinality 50K)
+ @Override
+ SolrInputDocument buildDoc() {
+if (userRecordCounts[currentUser - 1] == currentUser) {
+ currentUser++;
+} else {
+ userRecordCounts[currentUser -1]++;
+}
+
+final SolrInputDocument newDoc = new SolrInputDocument("id",
UUID.randomUUID().toString());
+final String userString = "user" + currentUser;
+final String[] permissions = getAFewDictionaryWords(300,
PERMISSION_CARDINALITY);
+
+newDoc.addField("userid_s", userString);
+newDoc.addField("permissions_ss", permissions);
+
+return newDoc;
+ }
+},
+DATA(NUM_TO_DOCS, TO_COLLECTION_NAME) {
+ // id - unique string
+ // val_i - random int between 1-1000
+ // cost_d - random cost between 1-1000
+ // body_txt - random text string between 100 - 1 words
+ // acl_ss - set of 100-3000 string permissions (cardinality 50K)
+ @Override
+ SolrInputDocument buildDoc() {
+final SolrInputDocument newDoc = new SolrInputDocument("id",
UUID.randomUUID().toString());
+final int val = random().nextInt(1000) + 1;
+final double cost = random().nextDouble() * 1000d;
+final String body = String.join(" ",
getAFewDictionaryWords(random().nextInt(9900) + 100));
+final String[]
[jira] [Assigned] (SOLR-14038) Admin UI display for "state.json" should be in a scollable region
[ https://issues.apache.org/jira/browse/SOLR-14038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden reassigned SOLR-14038: --- Assignee: Kevin Risden > Admin UI display for "state.json" should be in a scollable region > - > > Key: SOLR-14038 > URL: https://issues.apache.org/jira/browse/SOLR-14038 > Project: Solr > Issue Type: Bug > Components: Admin UI >Reporter: Erick Erickson >Assignee: Kevin Risden >Priority: Major > Attachments: Screen Shot 2019-12-09 at 3.19.53 PM.png > > > Probably a result of some of the recent changes to the admin UI. See attached > screenshot -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14038) Admin UI display for "state.json" should be in a scollable region
[ https://issues.apache.org/jira/browse/SOLR-14038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023401#comment-17023401 ] Kevin Risden commented on SOLR-14038: - So its not SOLR-13987 that does this. I don't know exactly what originally caused this but I have a fix for it. I fixed it while looking at SOLR-14209 > Admin UI display for "state.json" should be in a scollable region > - > > Key: SOLR-14038 > URL: https://issues.apache.org/jira/browse/SOLR-14038 > Project: Solr > Issue Type: Bug > Components: Admin UI >Reporter: Erick Erickson >Assignee: Kevin Risden >Priority: Major > Attachments: Screen Shot 2019-12-09 at 3.19.53 PM.png > > > Probably a result of some of the recent changes to the admin UI. See attached > screenshot -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14038) Admin UI display for "state.json" should be in a scollable region
[ https://issues.apache.org/jira/browse/SOLR-14038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-14038: Attachment: SOLR-14038.patch > Admin UI display for "state.json" should be in a scollable region > - > > Key: SOLR-14038 > URL: https://issues.apache.org/jira/browse/SOLR-14038 > Project: Solr > Issue Type: Bug > Components: Admin UI >Reporter: Erick Erickson >Assignee: Kevin Risden >Priority: Major > Attachments: SOLR-14038.patch, Screen Shot 2019-12-09 at 3.19.53 > PM.png > > > Probably a result of some of the recent changes to the admin UI. See attached > screenshot -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14038) Admin UI display for "state.json" should be in a scollable region
[ https://issues.apache.org/jira/browse/SOLR-14038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-14038: Status: Patch Available (was: Open) > Admin UI display for "state.json" should be in a scollable region > - > > Key: SOLR-14038 > URL: https://issues.apache.org/jira/browse/SOLR-14038 > Project: Solr > Issue Type: Bug > Components: Admin UI >Reporter: Erick Erickson >Assignee: Kevin Risden >Priority: Major > Attachments: SOLR-14038.patch, Screen Shot 2019-12-09 at 3.19.53 > PM.png > > > Probably a result of some of the recent changes to the admin UI. See attached > screenshot -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14038) Admin UI display for "state.json" should be in a scollable region
[ https://issues.apache.org/jira/browse/SOLR-14038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-14038: Fix Version/s: 8.5 > Admin UI display for "state.json" should be in a scollable region > - > > Key: SOLR-14038 > URL: https://issues.apache.org/jira/browse/SOLR-14038 > Project: Solr > Issue Type: Bug > Components: Admin UI >Reporter: Erick Erickson >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Attachments: SOLR-14038.patch, Screen Shot 2019-12-09 at 3.19.53 > PM.png > > > Probably a result of some of the recent changes to the admin UI. See attached > screenshot -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14038) Admin UI display for "state.json" should be in a scollable region
[ https://issues.apache.org/jira/browse/SOLR-14038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023402#comment-17023402 ] Kevin Risden commented on SOLR-14038: - [~erickerickson] can you look at [^SOLR-14038.patch] and see if this fixes it for you? (sorry for the whitespace changes). I tried to make "Cloud -> Tree -> file" and "Collections -> Files -> file" look the same. > Admin UI display for "state.json" should be in a scollable region > - > > Key: SOLR-14038 > URL: https://issues.apache.org/jira/browse/SOLR-14038 > Project: Solr > Issue Type: Bug > Components: Admin UI >Reporter: Erick Erickson >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Attachments: SOLR-14038.patch, Screen Shot 2019-12-09 at 3.19.53 > PM.png > > > Probably a result of some of the recent changes to the admin UI. See attached > screenshot -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14038) Admin UI display for "state.json" should be in a scollable region
[ https://issues.apache.org/jira/browse/SOLR-14038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023402#comment-17023402 ] Kevin Risden edited comment on SOLR-14038 at 1/25/20 3:19 AM: -- [~erickerickson] can you look at [^SOLR-14038.patch] and see if this fixes it for you? (sorry for the whitespace changes). I tried to make "Cloud -> Tree -> file" and "Collections -> collection -> Files -> file" look the same. was (Author: risdenk): [~erickerickson] can you look at [^SOLR-14038.patch] and see if this fixes it for you? (sorry for the whitespace changes). I tried to make "Cloud -> Tree -> file" and "Collections -> Files -> file" look the same. > Admin UI display for "state.json" should be in a scollable region > - > > Key: SOLR-14038 > URL: https://issues.apache.org/jira/browse/SOLR-14038 > Project: Solr > Issue Type: Bug > Components: Admin UI >Reporter: Erick Erickson >Assignee: Kevin Risden >Priority: Major > Fix For: 8.5 > > Attachments: SOLR-14038.patch, Screen Shot 2019-12-09 at 3.19.53 > PM.png > > > Probably a result of some of the recent changes to the admin UI. See attached > screenshot -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13749) Implement support for joining across collections with multiple shards ( XCJF )
[
https://issues.apache.org/jira/browse/SOLR-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023403#comment-17023403
]
Gus Heck commented on SOLR-13749:
-
[~dsmiley] Ah, looks like 9x didn't have a new features section yet, and I just
put it in the most applicable existing section, but yes a new features section
would have been be better. I expect Dan was just following what I did in
master. (and it will need to move down to 8.5 in master too)
> Implement support for joining across collections with multiple shards ( XCJF )
> --
>
> Key: SOLR-13749
> URL: https://issues.apache.org/jira/browse/SOLR-13749
> Project: Solr
> Issue Type: New Feature
>Reporter: Kevin Watters
>Assignee: Gus Heck
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This ticket includes 2 query parsers.
> The first one is the "Cross collection join filter" (XCJF) parser. This is
> the "Cross-collection join filter" query parser. It can do a call out to a
> remote collection to get a set of join keys to be used as a filter against
> the local collection.
> The second one is the Hash Range query parser that you can specify a field
> name and a hash range, the result is that only the documents that would have
> hashed to that range will be returned.
> This query parser will do an intersection based on join keys between 2
> collections.
> The local collection is the collection that you are searching against.
> The remote collection is the collection that contains the join keys that you
> want to use as a filter.
> Each shard participating in the distributed request will execute a query
> against the remote collection. If the local collection is setup with the
> compositeId router to be routed on the join key field, a hash range query is
> applied to the remote collection query to only match the documents that
> contain a potential match for the documents that are in the local shard/core.
>
>
> Here's some vocab to help with the descriptions of the various parameters.
> ||Term||Description||
> |Local Collection|This is the main collection that is being queried.|
> |Remote Collection|This is the collection that the XCJFQuery will query to
> resolve the join keys.|
> |XCJFQuery|The lucene query that executes a search to get back a set of join
> keys from a remote collection|
> |HashRangeQuery|The lucene query that matches only the documents whose hash
> code on a field falls within a specified range.|
>
>
> ||Param ||Required ||Description||
> |collection|Required|The name of the external Solr collection to be queried
> to retrieve the set of join key values ( required )|
> |zkHost|Optional|The connection string to be used to connect to Zookeeper.
> zkHost and solrUrl are both optional parameters, and at most one of them
> should be specified.
> If neither of zkHost or solrUrl are specified, the local Zookeeper cluster
> will be used. ( optional )|
> |solrUrl|Optional|The URL of the external Solr node to be queried ( optional
> )|
> |from|Required|The join key field name in the external collection ( required
> )|
> |to|Required|The join key field name in the local collection|
> |v|See Note|The query to be executed against the external Solr collection to
> retrieve the set of join key values.
> Note: The original query can be passed at the end of the string or as the
> "v" parameter.
> It's recommended to use query parameter substitution with the "v" parameter
> to ensure no issues arise with the default query parsers.|
> |routed| |true / false. If true, the XCJF query will use each shard's hash
> range to determine the set of join keys to retrieve for that shard.
> This parameter improves the performance of the cross-collection join, but
> it depends on the local collection being routed by the toField. If this
> parameter is not specified,
> the XCJF query will try to determine the correct value automatically.|
> |ttl| |The length of time that an XCJF query in the cache will be considered
> valid, in seconds. Defaults to 3600 (one hour).
> The XCJF query will not be aware of changes to the remote collection, so
> if the remote collection is updated, cached XCJF queries may give inaccurate
> results.
> After the ttl period has expired, the XCJF query will re-execute the join
> against the remote collection.|
> |_All others_| |Any normal Solr parameter can also be specified as a local
> param.|
>
> Example Solr Config.xml changes:
>
> {{<}}{{cache}} {{name}}{{=}}{{"hash_vin"}}
> {{ }}{{class}}{{=}}{{"solr.LRUCache"}}
> {{ }}{{size}}{{=}}{{"128"}}
> {{ }}{{initialSize}}{{=}}{{"0"}}
> {{ }}{{regenerator}}{{=}}{{"solr.NoOpRegenerator"}}{{/>
[GitHub] [lucene-solr] risdenk opened a new pull request #1209: SOLR-14209: Upgrade JQuery to 3.4.1
risdenk opened a new pull request #1209: SOLR-14209: Upgrade JQuery to 3.4.1 URL: https://github.com/apache/lucene-solr/pull/1209 * JQuery 2.1.3 to 3.4.1 * jstree 1.0-rc1 to v3.3.8 Signed-off-by: Kevin Risden This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] risdenk commented on issue #1209: SOLR-14209: Upgrade JQuery to 3.4.1
risdenk commented on issue #1209: SOLR-14209: Upgrade JQuery to 3.4.1 URL: https://github.com/apache/lucene-solr/pull/1209#issuecomment-578373225 @janhoy @rmuir @gerlowskija @ctargett tagging you to review since you have either modified these files recently or have been helping review the Admin UI stuffs. @ErickErickson tagging you related to the Zookeeper handler change since looks like you originally implemented it and it hasn't changed. I'm adding some comments to the PR about why this changed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] risdenk commented on a change in pull request #1209: SOLR-14209: Upgrade JQuery to 3.4.1
risdenk commented on a change in pull request #1209: SOLR-14209: Upgrade JQuery
to 3.4.1
URL: https://github.com/apache/lucene-solr/pull/1209#discussion_r370911540
##
File path: solr/webapp/web/css/angular/common.css
##
@@ -626,16 +611,6 @@ pre.syntax .tex .formula
background-position: 0 0;
}
-#content .tree .jstree-hovered
-{
- background:#e7f4f9; border:1px solid #d8f0fa; padding:0 2px 0 1px;
-}
-
-#content .tree .jstree-clicked
-{
- background:#beebff; border:1px solid #99defd; padding:0 2px 0 1px;
-}
-
Review comment:
Included jstree CSS file which avoided needing this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] risdenk commented on a change in pull request #1209: SOLR-14209: Upgrade JQuery to 3.4.1
risdenk commented on a change in pull request #1209: SOLR-14209: Upgrade JQuery to 3.4.1 URL: https://github.com/apache/lucene-solr/pull/1209#discussion_r370911461 ## File path: solr/NOTICE.txt ## @@ -37,7 +37,7 @@ Copyright (c) 2013-2014 Klaus Hartl, https://github.com/carhartl/jquery-cookie This product includes jquery.form Javascript library created by Mike Alsup Copyright 2006-2014 (c) M. Alsup, https://github.com/malsup/form/ -This product includes the jquery.jstree.js Javascript library created by Ivan Bozhanov +This product includes the jstree Javascript library created by Ivan Bozhanov Review comment: filename changed and looks like it is just called `jstree` now This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] risdenk commented on a change in pull request #1209: SOLR-14209: Upgrade JQuery to 3.4.1
risdenk commented on a change in pull request #1209: SOLR-14209: Upgrade JQuery
to 3.4.1
URL: https://github.com/apache/lucene-solr/pull/1209#discussion_r370911504
##
File path:
solr/core/src/java/org/apache/solr/handler/admin/ZookeeperInfoHandler.java
##
@@ -529,18 +529,13 @@ boolean printTree(JSONWriter json, String path) throws
IOException {
label = idx > 0 ? path.substring(idx + 1) : path;
}
json.startObject();
- //writeKeyValue(json, "data", label, true );
- json.writeString("data");
- json.writeNameSeparator();
-
- json.startObject();
- writeKeyValue(json, "title", label, true);
+ writeKeyValue(json, "text", label, true);
json.writeValueSeparator();
- json.writeString("attr");
+ json.writeString("a_attr");
json.writeNameSeparator();
json.startObject();
- writeKeyValue(json, "href", "admin/zookeeper?detail=true&path=" +
URLEncoder.encode(path, "UTF-8"), true);
- json.endObject();
+ String href = "admin/zookeeper?detail=true&path=" +
URLEncoder.encode(path, StandardCharsets.UTF_8);
+ writeKeyValue(json, "href", href, true);
Review comment:
The `jstree` JSON data format changed so it requires a different structure.
It wasn't easy to modify this structure for ZK since it came directly from the
`ZookeeperInfoHandler`. I looked and didn't see other uses of this, so it
looked safe to change here.
@ErickErickson any concerns?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] risdenk commented on a change in pull request #1209: SOLR-14209: Upgrade JQuery to 3.4.1
risdenk commented on a change in pull request #1209: SOLR-14209: Upgrade JQuery
to 3.4.1
URL: https://github.com/apache/lucene-solr/pull/1209#discussion_r370911534
##
File path: solr/webapp/web/css/angular/common.css
##
@@ -591,21 +591,6 @@ pre.syntax .tex .formula
opacity: 0.5;
}
-#content .tree li,
-#content .tree ins
-{
- background-color: transparent;
- background-image: url( ../../img/tree.png );
- background-repeat: no-repeat;
-}
-
-#content .tree li
-{
- background-position: -54px 0;
- background-repeat: repeat-y;
- line-height: 22px;
-}
-
Review comment:
Included jstree CSS file which avoided needing this. I didn't remove
`tree.png`. I think there are a BUNCH of img files that can be cleaned up, but
want to do that separate.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
