[GitHub] [lucene-solr] asalamon74 commented on pull request #2309: SOLR-14301: Remove external commons-codec usage in gradle validateJarChecksums
asalamon74 commented on pull request #2309: URL: https://github.com/apache/lucene-solr/pull/2309#issuecomment-778613952 Yes, the invocation is longer with this patch, on the other hand we avoid using an external library so the .gradle file is shorter. We only require that SHA-1 is supported by the JDK, which is a requirement for all JDK implementations: https://docs.oracle.com/javase/7/docs/api/java/security/MessageDigest.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] epugh commented on a change in pull request #2306: SOLR-15121: Move XSLT (tr param) to scripting contrib
epugh commented on a change in pull request #2306:
URL: https://github.com/apache/lucene-solr/pull/2306#discussion_r575665009
##
File path:
solr/contrib/scripting/src/test/org/apache/solr/scripting/xslt/XSLTUpdateRequestHandlerTest.java
##
@@ -39,11 +38,17 @@
import org.junit.BeforeClass;
import org.junit.Test;
-public class XsltUpdateRequestHandlerTest extends SolrTestCaseJ4 {
-
+/**
+ *
+ * This tests the XSLTUpdateRequestHandler ability to work with XSLT
stylesheet and xml content.
+ *
+*/
+public class XSLTUpdateRequestHandlerTest extends SolrTestCaseJ4 {
Review comment:
I am down for making this change... XSLT -> Xslt for the files touched
in this PR, and then maybe a seperate JIRA for XML --> Xml in the other
classes?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] epugh commented on pull request #2306: SOLR-15121: Move XSLT (tr param) to scripting contrib
epugh commented on pull request #2306: URL: https://github.com/apache/lucene-solr/pull/2306#issuecomment-778617013 Thanks for getting your hands dirty @dsmiley and making these changes, they LGTM. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] uschindler commented on a change in pull request #2306: SOLR-15121: Move XSLT (tr param) to scripting contrib
uschindler commented on a change in pull request #2306:
URL: https://github.com/apache/lucene-solr/pull/2306#discussion_r575675351
##
File path:
solr/contrib/scripting/src/java/org/apache/solr/scripting/xslt/XSLTUpdateRequestHandler.java
##
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.scripting.xslt;
+
+import static org.apache.solr.scripting.xslt.XSLTConstants.*;
+
+import java.util.Map;
+import javax.xml.stream.XMLStreamException;
+import javax.xml.stream.XMLStreamReader;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.dom.DOMResult;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.sax.SAXSource;
+import com.google.common.annotations.VisibleForTesting;
+import org.apache.solr.common.EmptyEntityResolver;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStream;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.handler.UpdateRequestHandler;
+import org.apache.solr.handler.loader.XMLLoader;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.solr.update.processor.UpdateRequestProcessor;
+import org.xml.sax.InputSource;
+import org.xml.sax.XMLReader;
+
+/**
+ * Send XML formatted documents to Solr, transforming them from the original
XML
+ * format to the Solr XML format using an XSLT stylesheet via the 'tr'
parameter.
+ */
+public class XSLTUpdateRequestHandler extends UpdateRequestHandler {
+
+ @Override
+ public void init(@SuppressWarnings({"rawtypes"})NamedList args) {
+super.init(args);
+setAssumeContentType("application/xml");
+
+SolrParams p = null;
+if (args != null) {
+ p = args.toSolrParams();
+}
+final XsltXMLLoader loader = new XsltXMLLoader().init(p);
+loaders = Map.of("application/xml", loader, "text/xml", loader);
+ }
+
+ @VisibleForTesting
+ static class XsltXMLLoader extends XMLLoader {
+
+int xsltCacheLifetimeSeconds;
+
+@Override
+public XsltXMLLoader init(SolrParams args) {
+ super.init(args);
+
+ xsltCacheLifetimeSeconds = XSLT_CACHE_DEFAULT;
+ if (args != null) {
+xsltCacheLifetimeSeconds = args.getInt(XSLT_CACHE_PARAM,
XSLT_CACHE_DEFAULT);
+ }
+ return this;
+}
+
+@Override
+public void load(
+SolrQueryRequest req,
+SolrQueryResponse rsp,
+ContentStream stream,
+UpdateRequestProcessor processor)
+throws Exception {
+
+ String tr = req.getParams().get(TR, null);
+ if (tr == null) {
+super.load(req, rsp, stream, processor); // no XSLT; do standard
processing
+return;
+ }
+
+ if (req.getCore().getCoreDescriptor().isConfigSetTrusted() == false) {
+throw new SolrException(
+SolrException.ErrorCode.UNAUTHORIZED,
+"The configset for this collection was uploaded without any
authentication in place,"
++ " and this operation is not available for collections with
untrusted configsets. To use this feature, re-upload the configset"
++ " after enabling authentication and authorization.");
+ }
+
+ final Transformer t = TransformerProvider.getTransformer(req, tr,
xsltCacheLifetimeSeconds);
+ final DOMResult result = new DOMResult();
+
+ // first step: read XML and build DOM using Transformer (this is no
overhead, as XSL always
+ // produces
+ // an internal result DOM tree, we just access it directly as input for
StAX):
+ try (var is = stream.getStream()) {
+final XMLReader xmlr = saxFactory.newSAXParser().getXMLReader();
+xmlr.setErrorHandler(xmllog);
+xmlr.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE);
+final InputSource isrc = new InputSource(is);
+
isrc.setEncoding(ContentStreamBase.getCharsetFromContentType(stream.getContentType()));
+final SAXSource source = new SAXSource(xmlr, isrc);
+t.transform(source, result);
+
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #2306: SOLR-15121: Move XSLT (tr param) to scripting contrib
dsmiley commented on a change in pull request #2306:
URL: https://github.com/apache/lucene-solr/pull/2306#discussion_r575675609
##
File path:
solr/contrib/scripting/src/java/org/apache/solr/scripting/xslt/XSLTUpdateRequestHandler.java
##
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.scripting.xslt;
+
+import static org.apache.solr.scripting.xslt.XSLTConstants.*;
+
+import java.util.Map;
+import javax.xml.stream.XMLStreamException;
+import javax.xml.stream.XMLStreamReader;
+import javax.xml.transform.Transformer;
+import javax.xml.transform.TransformerException;
+import javax.xml.transform.dom.DOMResult;
+import javax.xml.transform.dom.DOMSource;
+import javax.xml.transform.sax.SAXSource;
+import com.google.common.annotations.VisibleForTesting;
+import org.apache.solr.common.EmptyEntityResolver;
+import org.apache.solr.common.SolrException;
+import org.apache.solr.common.params.SolrParams;
+import org.apache.solr.common.util.ContentStream;
+import org.apache.solr.common.util.ContentStreamBase;
+import org.apache.solr.common.util.NamedList;
+import org.apache.solr.handler.UpdateRequestHandler;
+import org.apache.solr.handler.loader.XMLLoader;
+import org.apache.solr.request.SolrQueryRequest;
+import org.apache.solr.response.SolrQueryResponse;
+import org.apache.solr.update.processor.UpdateRequestProcessor;
+import org.xml.sax.InputSource;
+import org.xml.sax.XMLReader;
+
+/**
+ * Send XML formatted documents to Solr, transforming them from the original
XML
+ * format to the Solr XML format using an XSLT stylesheet via the 'tr'
parameter.
+ */
+public class XSLTUpdateRequestHandler extends UpdateRequestHandler {
+
+ @Override
+ public void init(@SuppressWarnings({"rawtypes"})NamedList args) {
+super.init(args);
+setAssumeContentType("application/xml");
+
+SolrParams p = null;
+if (args != null) {
+ p = args.toSolrParams();
+}
+final XsltXMLLoader loader = new XsltXMLLoader().init(p);
+loaders = Map.of("application/xml", loader, "text/xml", loader);
+ }
+
+ @VisibleForTesting
+ static class XsltXMLLoader extends XMLLoader {
+
+int xsltCacheLifetimeSeconds;
+
+@Override
+public XsltXMLLoader init(SolrParams args) {
+ super.init(args);
+
+ xsltCacheLifetimeSeconds = XSLT_CACHE_DEFAULT;
+ if (args != null) {
+xsltCacheLifetimeSeconds = args.getInt(XSLT_CACHE_PARAM,
XSLT_CACHE_DEFAULT);
+ }
+ return this;
+}
+
+@Override
+public void load(
+SolrQueryRequest req,
+SolrQueryResponse rsp,
+ContentStream stream,
+UpdateRequestProcessor processor)
+throws Exception {
+
+ String tr = req.getParams().get(TR, null);
+ if (tr == null) {
+super.load(req, rsp, stream, processor); // no XSLT; do standard
processing
+return;
+ }
+
+ if (req.getCore().getCoreDescriptor().isConfigSetTrusted() == false) {
+throw new SolrException(
+SolrException.ErrorCode.UNAUTHORIZED,
+"The configset for this collection was uploaded without any
authentication in place,"
++ " and this operation is not available for collections with
untrusted configsets. To use this feature, re-upload the configset"
++ " after enabling authentication and authorization.");
+ }
+
+ final Transformer t = TransformerProvider.getTransformer(req, tr,
xsltCacheLifetimeSeconds);
+ final DOMResult result = new DOMResult();
+
+ // first step: read XML and build DOM using Transformer (this is no
overhead, as XSL always
+ // produces
+ // an internal result DOM tree, we just access it directly as input for
StAX):
+ try (var is = stream.getStream()) {
+final XMLReader xmlr = saxFactory.newSAXParser().getXMLReader();
+xmlr.setErrorHandler(xmllog);
+xmlr.setEntityResolver(EmptyEntityResolver.SAX_INSTANCE);
+final InputSource isrc = new InputSource(is);
+
isrc.setEncoding(ContentStreamBase.getCharsetFromContentType(stream.getContentType()));
+final SAXSource source = new SAXSource(xmlr, isrc);
+t.transform(source, result);
+
[GitHub] [lucene-solr] uschindler commented on pull request #2306: SOLR-15121: Move XSLT (tr param) to scripting contrib
uschindler commented on pull request #2306: URL: https://github.com/apache/lucene-solr/pull/2306#issuecomment-778630343 > @uschindler if you want to push up an example/make the change the way you are thinking, I'm 100% happy to have that! Your Java skillz are way beyond mine, and while I sort of understnad what you say, it may take you 20 minutes to get the change you want, and then I'll learn from you ;-) Sorry, my father died on Thursday. So excuse my ignorance! To me @dsmiley changes look fine. I will review later as some "long term specialist" 😉 on XML processing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on pull request #2306: SOLR-15121: Move XSLT (tr param) to scripting contrib
dsmiley commented on pull request #2306: URL: https://github.com/apache/lucene-solr/pull/2306#issuecomment-778632312 > Sorry, my father died on Thursday. Wow; so sorry to hear! I fear the same for my own Dad; I can't take his existence for granted so I'll be visiting him tomorrow. > I will review later as some "long term specialist" 😉 on XML processing. You're not the only one; I'm a long-time XML fan :-) It seems the new kids know little about such things :-/ "What is this weird syntax... " as he/she goes off to then write a dozen lines that an XPath does in one. I kept the all-caps XSLT despite my preferences because All of the XML related APIs we use are using all-caps for XML and thus it looked off to see it differently. But then... the user doesn't know that and sees XML in the config file so... Hmm. Maybe the XSLT response writer should demand "tr" and thus be more consistent with the XSLT response writer? After all, you're choosing to send the response there so why would you send it there if it wasn't? I guess it gives you choice and the ability to be more similar to what we have today. Shrug. We need some docs to show how to reference the XSLT update handler, or at the very least an upgrade note about this. I hate the crappy caching; it's inferior to a SolrCache but alas... I hold myself back from feature-creep. This code was written forever-ago so I can appreciate it has excuses. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] epugh commented on pull request #2306: SOLR-15121: Move XSLT (tr param) to scripting contrib
epugh commented on pull request #2306: URL: https://github.com/apache/lucene-solr/pull/2306#issuecomment-778641991 Sorry to hear that news @uschindler I did a pass through the Ref Guide to document how to use the XSLT stuff, and more explicitly called out the sample `.xsl` files that we have. I can add more if needed. So, if we aren't changing the name of the Java class in this PR, then I think we just need to update the Upgrade to Solr 9 page, and we're good to go? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on pull request #2306: SOLR-15121: Move XSLT (tr param) to scripting contrib
dsmiley commented on pull request #2306: URL: https://github.com/apache/lucene-solr/pull/2306#issuecomment-778648072 > So, if we aren't changing the name of the Java class in this PR, then I think we just need to update the Upgrade to Solr 9 page, and we're good to go? +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-15152) Export Tool should export nested docs cleanly in .json, .jsonl, and javabin
[ https://issues.apache.org/jira/browse/SOLR-15152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284268#comment-17284268 ] David Smiley commented on SOLR-15152: - RE compress; I'd rather we not and let the user pipe it to standard utilities. These are composable building blocks and we only need to provide the Solr piece. But since you've added it -- okay. I'd be happy to review anything anyone does related to nested docs! ... > Export Tool should export nested docs cleanly in .json, .jsonl, and javabin > --- > > Key: SOLR-15152 > URL: https://issues.apache.org/jira/browse/SOLR-15152 > Project: Solr > Issue Type: Improvement > Components: SolrCLI >Affects Versions: 8.8 >Reporter: David Eric Pugh >Assignee: David Eric Pugh >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > ExportTool doesn't properly handle anonymous child docs or nested docs.  It > also confuses the JSONL format with the JSON format.  > I'd like to have the JSON Lines format output as .jsonl, which is the > standard, and have the JSON format to be a .json, which is the same output as > if you wanted to post a Solr doc as a JSON to upload the data...   This will > let us round trip the data. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-15152) Export Tool should export nested docs cleanly in .json, .jsonl, and javabin
[ https://issues.apache.org/jira/browse/SOLR-15152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated SOLR-15152: Labels: NestedDocuments (was: ) > Export Tool should export nested docs cleanly in .json, .jsonl, and javabin > --- > > Key: SOLR-15152 > URL: https://issues.apache.org/jira/browse/SOLR-15152 > Project: Solr > Issue Type: Improvement > Components: SolrCLI >Affects Versions: 8.8 >Reporter: David Eric Pugh >Assignee: David Eric Pugh >Priority: Major > Labels: NestedDocuments > Time Spent: 40m > Remaining Estimate: 0h > > ExportTool doesn't properly handle anonymous child docs or nested docs.  It > also confuses the JSONL format with the JSON format.  > I'd like to have the JSON Lines format output as .jsonl, which is the > standard, and have the JSON format to be a .json, which is the same output as > if you wanted to post a Solr doc as a JSON to upload the data...   This will > let us round trip the data. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #2356: SOLR-15152: Export Tool should export nested docs cleanly in .json, .jsonl, and javabin
dsmiley commented on a change in pull request #2356:
URL: https://github.com/apache/lucene-solr/pull/2356#discussion_r575699438
##
File path:
solr/solrj/src/test/org/apache/solr/client/solrj/SolrExampleTests.java
##
@@ -1935,21 +1935,22 @@ public void testChildDoctransformer() throws
IOException, SolrServerException {
Map allDocs = new HashMap<>();
for (int i =0; i < numRootDocs; i++) {
- client.add(genNestedDocuments(allDocs, 0, maxDepth));
+ client.add(generateNestedDocuments(allDocs, 0, maxDepth));
}
client.commit();
// sanity check
-SolrQuery q = new SolrQuery("q", "*:*", "indent", "true");
+SolrQuery q = new SolrQuery("q", "*:*", "rows", "0");
QueryResponse resp = client.query(q);
assertEquals("Doc count does not match",
allDocs.size(), resp.getResults().getNumFound());
-// base check - we know there is an exact number of these root docs
-q = new SolrQuery("q","level_i:0", "indent", "true");
-q.setFields("*", "[child parentFilter=\"level_i:0\"]");
+// base check - we know there the exact number of these root docs
+q = new SolrQuery("{!parent which=\"*:* -_nest_path_:*\"}");
+q.addField("*,[child limit=\"-1\"]");
Review comment:
I think it was clearer before with setFields (plural) as you are adding
two fields
##
File path:
solr/core/src/java/org/apache/solr/response/transform/ChildDocTransformerFactory.java
##
@@ -160,8 +161,18 @@ protected static String
processPathHierarchyQueryString(String queryString) {
int indexOfLastPathSepChar = queryString.lastIndexOf(PATH_SEP_CHAR,
indexOfFirstColon);
if (indexOfLastPathSepChar < 0) {
// regular filter, not hierarchy based.
- return ClientUtils.escapeQueryChars(queryString.substring(0,
indexOfFirstColon))
Review comment:
I think there should **not** have been escaping here in the first place.
Unless the user is using the special syntax, we should handle it plainly like
we do everywhere else -- simple. It may mean the user has to escape characters
like '/' but that's normal. In a Java String that has an embedded local-params
for the child doc transformer, that winds up being four back-slashes. So be
it. It may seem like we're trying to do the user a favor by escaping what we
see but I think it's actually more confusing to reason about because it's
inconsistent. Besides, there are other ways of constructing the query syntax
that results in zero escaping. like `childFilter='{!field
f=_nest_path_}/toppings'` So that's longer but no escaping at least. Up to
the user's prerogative.
CC @moshebla
I'll fix this in your PR momentarily.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on pull request #2362: LUCENE-9767: infrastructure for icu regeneration in place.
dweiss commented on pull request #2362: URL: https://github.com/apache/lucene-solr/pull/2362#issuecomment-778675424 Hi Robert. Added an experimental task foo with initial steps downloading the matching ICU source package. Have to go again - family. But it should be doable to compile it from within gradle, it looks relatively easy from this point on. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] zacharymorn commented on pull request #2342: LUCENE-9406: Add IndexWriterEventListener to track events in IndexWriter
zacharymorn commented on pull request #2342: URL: https://github.com/apache/lucene-solr/pull/2342#issuecomment-778678349 > If such a situation arises folks will have the ability to write a custom proxy listener with the ability to switch, add and/or remove delegates. This keeps the freedom to do anything you said and at the same time simplifies implementation (and concurrency-issues) on IW side? Ah that's very true! I've pushed a commit to remove volatile then. Please let me know if it looks good. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-15145) Older versions of SolrJ (pre-8.8.0) hit an NPE when computing the base_url for core node props
[
https://issues.apache.org/jira/browse/SOLR-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284313#comment-17284313
]
ASF subversion and git services commented on SOLR-15145:
Commit 6a50a0315ac7e4979abb0b530857c7795bb3b928 in lucene-solr's branch
refs/heads/branch_8_8 from Timothy Potter
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6a50a03 ]
SOLR-15145: Restore base_url if node_name is set
> Older versions of SolrJ (pre-8.8.0) hit an NPE when computing the base_url
> for core node props
> --
>
> Key: SOLR-15145
> URL: https://issues.apache.org/jira/browse/SOLR-15145
> Project: Solr
> Issue Type: Bug
> Components: SolrJ
>Affects Versions: 8.8
>Reporter: Timothy Potter
>Assignee: Timothy Potter
>Priority: Critical
> Fix For: 8.8.1
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> From the mailing list:
> {code}
> Caused by: java.lang.NullPointerException
> at
> deployment.uleaf.ear//org.apache.solr.common.cloud.ZkCoreNodeProps.getCoreUrl(ZkCoreNodeProps.java:53)
> at
> deployment.uleaf.ear//org.apache.solr.client.solrj.impl.BaseCloudSolrClient.lambda$sendRequest$2(BaseCloudSolrClient.java:1161)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> deployment.uleaf.ear//org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1159)
> at
> deployment.uleaf.ear//org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:934)
> ... 166 more
> {code}
> see:
> https://lists.apache.org/thread.html/r3d131030f0a7026235451f71fabdae6d6b7c2f955822c75dcad4d41f%40%3Csolr-user.lucene.apache.org%3E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley merged pull request #2358: LUCENE-9762: FunctionScoreQuery must guard score() called twice
dsmiley merged pull request #2358: URL: https://github.com/apache/lucene-solr/pull/2358 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9762) FunctionScoreQuery can fail when the score is requested twice
[
https://issues.apache.org/jira/browse/LUCENE-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284329#comment-17284329
]
ASF subversion and git services commented on LUCENE-9762:
-
Commit 6c140b6dcf3948c4e198ebfd5978673c621216ee in lucene-solr's branch
refs/heads/master from David Smiley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6c140b6 ]
LUCENE-9762: FunctionScoreQuery must guard score() called twice (#2358)
The score() may be called multiple times. It should take care to call
DoubleValues.advanceExact only the first time, or risk faulty behavior
including exceptions.
> FunctionScoreQuery can fail when the score is requested twice
> -
>
> Key: LUCENE-9762
> URL: https://issues.apache.org/jira/browse/LUCENE-9762
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.8
>Reporter: Chris M. Hostetter
>Assignee: David Smiley
>Priority: Major
> Attachments: LUCENE-9762.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> As originally reported by Nicolás Lichtmaier on the java-user list, there are
> some trivial situations which can trigger an assertion error in the
> PostingsReader when enumerating PhrasePositions for a sloppy PhraseQuery...
> {noformat}
> Exception in thread "main" java.lang.AssertionError
> at
> org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.nextPosition(Lucene84PostingsReader.java:940)
> at
> org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)
> at
> org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.java:46)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.initSimple(SloppyPhraseMatcher.java:368)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.initPhrasePositions(SloppyPhraseMatcher.java:356)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.reset(SloppyPhraseMatcher.java:153)
> at org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:49)
> at
> org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)
> at
> org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)
> at
> org.apache.lucene.search.DoubleValues$1.doubleValue(DoubleValues.java:48)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)
> at
> org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:659)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:572)
> at
> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:419)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:430)
> at LuceneCrash.main(LuceneCrash.java:51)
> {noformat}
> http://mail-archives.apache.org/mod_mbox/lucene-java-user/202102.mbox/%3C177a65ec-5ec3-e1aa-99c3-b478e165d5e8%40wolfram.com%3E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9762) FunctionScoreQuery can fail when the score is requested twice
[
https://issues.apache.org/jira/browse/LUCENE-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284341#comment-17284341
]
David Smiley commented on LUCENE-9762:
--
I think my assessment is incorrect.
The documentation on DoubleValues.advanceExact is:
{quote}
Advance this instance to the given document id
{quote}
Rather ambiguous; ehh? However I think it's trying to match the semantics of
the same method name on DocValuesIterator which has more javadocs:
{quote}
Advance the iterator to exactly target and return whether target has a value.
target must be greater than or equal to the current doc ID and must be a valid
doc ID, ie. ≥ 0 and < maxDoc. After this method returns, docID() returns target.
{quote}
**Or equal** thus repeated calls at the same ID is okay. There are
implementations of DoubleValues/LongValues that simply delegate advanceExact to
a DocValuesIterator.
Sigh... I'll revert the master commit, and submit a new PR to fix
WeightDoubleValuesSource.
> FunctionScoreQuery can fail when the score is requested twice
> -
>
> Key: LUCENE-9762
> URL: https://issues.apache.org/jira/browse/LUCENE-9762
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.8
>Reporter: Chris M. Hostetter
>Assignee: David Smiley
>Priority: Major
> Attachments: LUCENE-9762.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> As originally reported by Nicolás Lichtmaier on the java-user list, there are
> some trivial situations which can trigger an assertion error in the
> PostingsReader when enumerating PhrasePositions for a sloppy PhraseQuery...
> {noformat}
> Exception in thread "main" java.lang.AssertionError
> at
> org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.nextPosition(Lucene84PostingsReader.java:940)
> at
> org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)
> at
> org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.java:46)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.initSimple(SloppyPhraseMatcher.java:368)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.initPhrasePositions(SloppyPhraseMatcher.java:356)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.reset(SloppyPhraseMatcher.java:153)
> at org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:49)
> at
> org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)
> at
> org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)
> at
> org.apache.lucene.search.DoubleValues$1.doubleValue(DoubleValues.java:48)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)
> at
> org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:659)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:572)
> at
> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:419)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:430)
> at LuceneCrash.main(LuceneCrash.java:51)
> {noformat}
> http://mail-archives.apache.org/mod_mbox/lucene-java-user/202102.mbox/%3C177a65ec-5ec3-e1aa-99c3-b478e165d5e8%40wolfram.com%3E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@luc
[jira] [Commented] (LUCENE-9762) FunctionScoreQuery can fail when the score is requested twice
[
https://issues.apache.org/jira/browse/LUCENE-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284344#comment-17284344
]
ASF subversion and git services commented on LUCENE-9762:
-
Commit 99f011a06edaaa91ee545d0a451b6c5cd3b5cbb0 in lucene-solr's branch
refs/heads/master from David Smiley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=99f011a ]
Revert "LUCENE-9762: FunctionScoreQuery must guard score() called twice (#2358)"
This reverts commit 6c140b6d
> FunctionScoreQuery can fail when the score is requested twice
> -
>
> Key: LUCENE-9762
> URL: https://issues.apache.org/jira/browse/LUCENE-9762
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.8
>Reporter: Chris M. Hostetter
>Assignee: David Smiley
>Priority: Major
> Attachments: LUCENE-9762.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> As originally reported by Nicolás Lichtmaier on the java-user list, there are
> some trivial situations which can trigger an assertion error in the
> PostingsReader when enumerating PhrasePositions for a sloppy PhraseQuery...
> {noformat}
> Exception in thread "main" java.lang.AssertionError
> at
> org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.nextPosition(Lucene84PostingsReader.java:940)
> at
> org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)
> at
> org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.java:46)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.initSimple(SloppyPhraseMatcher.java:368)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.initPhrasePositions(SloppyPhraseMatcher.java:356)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.reset(SloppyPhraseMatcher.java:153)
> at org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:49)
> at
> org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)
> at
> org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)
> at
> org.apache.lucene.search.DoubleValues$1.doubleValue(DoubleValues.java:48)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)
> at
> org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:659)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:572)
> at
> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:419)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:430)
> at LuceneCrash.main(LuceneCrash.java:51)
> {noformat}
> http://mail-archives.apache.org/mod_mbox/lucene-java-user/202102.mbox/%3C177a65ec-5ec3-e1aa-99c3-b478e165d5e8%40wolfram.com%3E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley opened a new pull request #2365: LUCENE-9762: DoubleValuesSource.fromQuery bug
dsmiley opened a new pull request #2365: URL: https://github.com/apache/lucene-solr/pull/2365 When same doc visited twice and when the query is TPI based, an exception can be thrown. We need to ensure the internal TPI.matches() is never called repeatedly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on pull request #2358: LUCENE-9762: FunctionScoreQuery must guard score() called twice
dsmiley commented on pull request #2358: URL: https://github.com/apache/lucene-solr/pull/2358#issuecomment-778735554 As explained in the JIRA issue, I reverted and filed a new PR: #2365 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9762) DoubleValuesSource.fromQuery can fail when doc visited twice
[
https://issues.apache.org/jira/browse/LUCENE-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated LUCENE-9762:
-
Summary: DoubleValuesSource.fromQuery can fail when doc visited twice
(was: FunctionScoreQuery can fail when the score is requested twice)
> DoubleValuesSource.fromQuery can fail when doc visited twice
>
>
> Key: LUCENE-9762
> URL: https://issues.apache.org/jira/browse/LUCENE-9762
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.8
>Reporter: Chris M. Hostetter
>Assignee: David Smiley
>Priority: Major
> Attachments: LUCENE-9762.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> As originally reported by Nicolás Lichtmaier on the java-user list, there are
> some trivial situations which can trigger an assertion error in the
> PostingsReader when enumerating PhrasePositions for a sloppy PhraseQuery...
> {noformat}
> Exception in thread "main" java.lang.AssertionError
> at
> org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.nextPosition(Lucene84PostingsReader.java:940)
> at
> org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)
> at
> org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.java:46)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.initSimple(SloppyPhraseMatcher.java:368)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.initPhrasePositions(SloppyPhraseMatcher.java:356)
> at
> org.apache.lucene.search.SloppyPhraseMatcher.reset(SloppyPhraseMatcher.java:153)
> at org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:49)
> at
> org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)
> at
> org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)
> at
> org.apache.lucene.search.DoubleValues$1.doubleValue(DoubleValues.java:48)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)
> at
> org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)
> at
> org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)
> at
> org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:659)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:572)
> at
> org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:419)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:430)
> at LuceneCrash.main(LuceneCrash.java:51)
> {noformat}
> http://mail-archives.apache.org/mod_mbox/lucene-java-user/202102.mbox/%3C177a65ec-5ec3-e1aa-99c3-b478e165d5e8%40wolfram.com%3E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
