Solr scoring confusion
We are getting inconsistent scoring results in Solr. It works about 95% of the time, where a search on one term returns the results which equal exactly that one term at the top, and results with multiple terms that also contain that one term are returned lower. Occasionally, however, if a subset of the data has been re-indexed (the same data just added to the index again) then the results will be slightly off, for example the data from the earlier index will get a higher score than it should, until we re-index all the data. Our assumption here is that setting omitNorms to false, then indexing the data, then searching, should result in scores where the data with an exact match has a higher score. We usually see this but not always. Is something added to the score besides the value that is being searched that we are not understaning? Thanks. .. Scott Johnson Data Advantage Group, Inc. 604 Mission Street San Francisco, CA 94105 Office: +1.415.947.0400 x204 Fax: +1.415.947.0401 Take the first step towards a successful meta data initiative with MetaCenter - the only plug and play, real-time meta data solution.<http://www.dag.com/> www.dag.com ..
Inconsistent response time
We are attempting to improve our Solr response time as our application uses
Solr for large and time consuming queries. We have found a very inconsistent
result in the time elapsed when pinging Solr. If we ping Solr from a desktop
Windows 7 machine, there is usually a 5 ms elapsed time. But if we ping the
same Solr instance from a Windows Server 2008 machine, it takes about 15 ms.
This could be the difference between a 1 hour process and a 3 hour process,
so it is something we would like to debug and fix if possible.
Does anybody have any ideas about why this might be? We get these same
results pretty consistently (testing on multiple desktops and servers). One
thing that seemed to have an impact is removing various additional JDKs that
had been installed, and JDK 1.7u67 specifically seemed to make a difference.
Finally, the code we are suing to test this is below. If there is a better
test I would be curious to hear that as well.
Thanks,
Scott
package solr;
import org.apache.commons.lang.StringUtils;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrRequest.METHOD;
import org.apache.solr.client.solrj.impl.BinaryRequestWriter;
import org.apache.solr.client.solrj.impl.BinaryResponseParser;
import org.apache.solr.client.solrj.impl.HttpSolrServer;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.client.solrj.response.SolrPingResponse;
import org.apache.solr.common.SolrDocumentList;
public class SolrTest {
private HttpSolrServer server;
/**
* @param args
* @throws Exception
*/
public static void main(String[] args) throws Exception {
SolrTest solr = new SolrTest(args);
// Run it a few times, the second time runs
a lot faster.
for (int i=0; i<3; i++) {
solr.execute();
}
}
public SolrTest(String[] args) throws Exception {
String targetUrl = args[0];
System.out.println("=System
properties=");
System.out.println("Start solr test " +
targetUrl);
server = new HttpSolrServer("http://"; +
targetUrl + ":8111/solr/search/");
server.setRequestWriter(new
BinaryRequestWriter());
server.setParser(new
BinaryResponseParser());
server.setAllowCompression(true);
server.setDefaultMaxConnectionsPerHost(128);
server.setMaxTotalConnections(128);
SolrPingResponse response = server.ping();
System.out.println("Ping time: " +
response.getElapsedTime() + " ms");
System.out.println("Ping time: " +
response.getElapsedTime() + " ms");
}
private void execute() throws Exception {
SolrQuery query = new SolrQuery();
query.setParam("start", "0");
query.setParam("rows", "1");
long startTime = System.currentTimeMillis();
QueryResponse queryResponse =
server.query(query, METHOD.POST);
long elapsedTime =
(System.currentTimeMillis() - startTime);
SolrDocumentList results =
queryResponse.getResults();
long totalHits = results.getNumFound();
System.out.println("Search hits:" +
totalHits
+ ". Total
elapsed time:" + elapsedTime + " ms"
+ ". Solr
elapsed time:" + queryResponse.getElapsedTime() + " ms"
+ ". Solr
query time:" + queryResponse.getQTime() + " ms"
+ ". Params:
" + getSearchParams(query));
}
/**
* Formats solr query parameters so that we know what's passed to solr.
* @param query
* @return
*/
private String getSearchParams(SolrQuery query) {
RE: Inconsistent response time
Thanks for the recommendation, but that is not making a difference here.
-Original Message-
From: Michael Della Bitta [mailto:michael.della.bi...@appinions.com]
Sent: Friday, October 03, 2014 2:00 PM
To: solr-user@lucene.apache.org
Subject: Re: Inconsistent response time
Hi Scott,
Any chance this could be an IPv6 thing? What if you start both server and
client with this flag:
-Djava.net.preferIPv4Stack=true
Michael Della Bitta
Senior Software Engineer
o: +1 646 532 3062
appinions inc.
"The Science of Influence Marketing"
18 East 41st Street
New York, NY 10017
t: @appinions | g+: plus.google.com/appinions
w: appinions.com
On Oct 3, 2014, at 15:08, Scott Johnson wrote:
> We are attempting to improve our Solr response time as our application
> uses Solr for large and time consuming queries. We have found a very
> inconsistent result in the time elapsed when pinging Solr. If we ping
> Solr from a desktop Windows 7 machine, there is usually a 5 ms elapsed
> time. But if we ping the same Solr instance from a Windows Server 2008
machine, it takes about 15 ms.
> This could be the difference between a 1 hour process and a 3 hour
> process, so it is something we would like to debug and fix if possible.
>
>
>
> Does anybody have any ideas about why this might be? We get these same
> results pretty consistently (testing on multiple desktops and
> servers). One thing that seemed to have an impact is removing various
> additional JDKs that had been installed, and JDK 1.7u67 specifically
seemed to make a difference.
>
>
>
> Finally, the code we are suing to test this is below. If there is a
> better test I would be curious to hear that as well.
>
>
>
> Thanks,
>
>
> Scott
>
>
>
>
>
> package solr;
>
>
>
> import org.apache.commons.lang.StringUtils;
>
> import org.apache.solr.client.solrj.SolrQuery;
>
> import org.apache.solr.client.solrj.SolrRequest.METHOD;
>
> import org.apache.solr.client.solrj.impl.BinaryRequestWriter;
>
> import org.apache.solr.client.solrj.impl.BinaryResponseParser;
>
> import org.apache.solr.client.solrj.impl.HttpSolrServer;
>
> import org.apache.solr.client.solrj.response.QueryResponse;
>
> import org.apache.solr.client.solrj.response.SolrPingResponse;
>
> import org.apache.solr.common.SolrDocumentList;
>
>
>
> public class SolrTest {
>
>
>
>private HttpSolrServer server;
>
>
>
>/**
>
>* @param args
>
>* @throws Exception
>
> */
>
>public static void main(String[] args) throws Exception
> {
>
>SolrTest solr = new SolrTest(args);
>
>// Run it a few times, the second time
> runs a lot faster.
>
>for (int i=0; i<3; i++) {
>
>solr.execute();
>
>}
>
>}
>
>
>
>public SolrTest(String[] args) throws Exception {
>
>String targetUrl = args[0];
>
>
>
>System.out.println("=System
> properties=");
>
>System.out.println("Start solr test
> " + targetUrl);
>
>
>
>server = new HttpSolrServer("http://"; +
> targetUrl + ":8111/solr/search/");
>
>server.setRequestWriter(new
> BinaryRequestWriter());
>
>server.setParser(new
> BinaryResponseParser());
>
>server.setAllowCompression(true);
>
>
> server.setDefaultMaxConnectionsPerHost(128);
>
>server.setMaxTotalConnections(128);
>
>
>
>SolrPingResponse response =
> server.ping();
>
>System.out.println("Ping time: " +
> response.getElapsedTime() + " ms");
>
>System.out.println("Ping time: " +
> response.getElapsedTime() + " ms");
>
>}
>
>
>
>private void execute() throws Exception {
>
>SolrQuery query = new SolrQuery();
>
>query.setParam("start", "0");
>
>
