Re: Solr 5.0.0 integration with Nutch 1.9

I followed the given steps and created a core named foo with sample_techproducts_configs but when I give the indexing command to nutch "bin/nutch solrindex http://localhost:8983/solr crawl/crawldb/ -linkdb crawl/linkdb/ crawl/segments/20150406231502/ -filter -normalize" it gives an error that Inde

Re: Setting up SolrCloud 5.0.0 and ZooKeeper 3.4.6

Hi Erick, I think I'll just setup the ZooKeeper server in standalone mode first, before I get more confused as I'm quite new to both Solr and ZooKeeper too. Better not to jump the gun. However, I face this error when I try to start it in standalone mode. 2015-04-07 11:59:51,789 [myid:] - ERROR [

Re: Measuring QPS

Hi Daniel, See SPM , which will give you QPS and a bunch of other Solr, JVM, and OS metrics, along with alerting, anomaly detection, and not-yet-announced transaction tracing . It has percentiles Wun

Re: Problem with new solr.xml format and core swaps

On 4/6/2015 6:40 PM, Erick Erickson wrote: > What version are you migrating _from_? 4.9.0? There were some > persistence issues at one point, but AFAIK they were fixed by 4.9, I > can check if you're on an earlier version... Effectively there is no previous version. Whenever I upgrade, I delete a

Re: Setting up SolrCloud 5.0.0 and ZooKeeper 3.4.6

Believe me, I'm no Zookeeper expert, but it looks to me like you're mixing Solr ports and Zookeeper ports. AFAIK, the two ports in the zoo.cfg file are exclusively for the Zookeeper instances to talk to each other. Zookeeper isn't aware that the listening nodes are Solr noodes, so putting Solr port

Re: Config join parse in solrconfig.xml

df does not allow multiple fields, it stands for "default field", not "default fields". To get what you're looking for, you need to use edismax or explicitly create the multiple clauses. I'm not quite sure what the join parser is doing with the df parameter. So my first question is "what happens i

Re: Facet

fc.method=enum will create an entry in the filter cache for each and every value. But since the filterCache is bounded, each result will pretty much be thrown away immediately. At least that's what I remember. Which neatly accounts for your issue I think; you're spending a huge amount of time/cycl

Setting up SolrCloud 5.0.0 and ZooKeeper 3.4.6

Hi, I'm using Solr 5.0.0 and ZooKeeper 3.4.6. I'm trying to set up a ZooKeeper with simulation of 3 servers, but they are all located on the same machine for testing purpose. In my zoo.cfg file, I have listed down the 3 servers to be as follows: server.1=localhost:8983:3888 server.2=localhost:898

Re: Problem with new solr.xml format and core swaps

Shawn: What version are you migrating _from_? 4.9.0? There were some persistence issues at one point, but AFAIK they were fixed by 4.9, I can check if you're on an earlier version... Erick On Sun, Apr 5, 2015 at 2:05 PM, Shawn Heisey wrote: > I'm having two problems with Solr 4.9.1. I can't up

Re: Solr 5.0.0 integration with Nutch 1.9

On 4/6/2015 2:14 PM, Anchit Jain wrote: > I want to index nutch results using *Solr 5.0* but as mentioned in > https://wiki.apache.org/nutch/NutchTutorial there is no directory > ${APACHE_SOLR_HOME}/example/solr/collection1/conf/ > in solr 5.0 . So where I have to copy *schema.xml*? > Also there

Re: Measuring QPS

That sounds neat. Our QA people are moving to Gatling, so we probably won’t change our JMeter approach now. We use the JMeter Plugs CMDrunner, telling it to generate only CSV. http://jmeter-plugins.org/wiki/JMeterPluginsCMD/ Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/

RE: Are there known issues with Java 8 in older versions of Solr?

I can at least say that Solr 3.x works fine with Java 7. -Michael -Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Monday, April 06, 2015 5:26 PM To: solr-user@lucene.apache.org Subject: Re: Are there known issues with Java 8 in older versions of Solr? On 4/6/201

Re: Measuring QPS

Hi Walter, sort of shameless plug - I ran into similar issues and wrote a JMeter SLA Reporting Backend - https://github.com/sgoeschl/jmeter-sla-report * It reads the CSV/XML JMeter report file and sorts the response times in logarithmic buckets

Re: Are there known issues with Java 8 in older versions of Solr?

On 4/6/2015 3:10 PM, chillra wrote: We are currently running Solr 3.6.1. The release notes for Solr 4.8 state that it is verified to be compatible with Java 8. Does this mean that older releases of Solr were not compatible, or just that they have not be tested? None of the bug fixes associated

Are there known issues with Java 8 in older versions of Solr?

We are currently running Solr 3.6.1. The release notes for Solr 4.8 state that it is verified to be compatible with Java 8. Does this mean that older releases of Solr were not compatible, or just that they have not be tested? None of the bug fixes associated with the 4.8 release seem to be relate

Re: Measuring QPS

The load testing is the easiest part. We use JMeter to replay the prod logs. We start about a hundred threads and use ConstantThroughputTimer to control the traffic level. JMeter tends to fall over with two much data graphing, so we run it headless. Then we post process with JMeter Plugins to g

Re: Measuring QPS

The good-sounding thing - you can do that easily with JMeter running the GUI or the command-line Cheers, Siegfried Goeschl > On 06 Apr 2015, at 21:35, Davis, Daniel (NIH/NLM) [C] > wrote: > > This sounds really good: > > "For load testing, we replay production logs to test that we meet the

Solr 5.0.0 integration with Nutch 1.9

I want to index nutch results using *Solr 5.0* but as mentioned in https://wiki.apache.org/nutch/NutchTutorial there is no directory ${APACHE_SOLR_HOME}/example/solr/collection1/conf/ in solr 5.0 . So where I have to copy *schema.xml*? Also there is no *start.jar* present in example directory.

RE: Measuring QPS

This sounds really good: "For load testing, we replay production logs to test that we meet the SLA at a given traffic level." The rest sounds complicated. Ah well, that's the job. -Original Message- From: Walter Underwood [mailto:wun...@wunderwood.org] Sent: Monday, April 06, 2015 2:

Trouble GetSpans lucene 4

Hi,  I'm working on TamingText's book.I try to upgrade the code from solr 3.6 to solr 4.10.2.At the moment, i have a problem about the method "getSpans"."spans.next()" returns always "false".Anyone can helps? SpanNearQuery sQuery = (SpanNearQuery) origQuery;SolrIndexSearcher searcher = rb.req.ge

Re: Config join parse in solrconfig.xml

The error message was from the query with "debug=query". On Mon, Apr 6, 2015 at 11:49 AM, Frank li wrote: > Hi Erick, > > > Thanks for your response. > > Here is the query I am sending: > > http://dev-solr:8080/solr/collection1/select?q={!join+from=litigation_id_ls+to=lit_id_lms}all_text:apple&f

Re: Backup within SolrCloud

I wrote a simple backup utility for a Collection that uses the replication handler, see: https://github.com/LucidWorks/solr-scale-tk/blob/master/src/main/java/com/lucidworks/SolrCloudTools.java#L614 feel free to borrow / steal if useful. On Mon, Apr 6, 2015 at 12:42 PM, Davis, Daniel (NIH/NLM) [C]

Re: Config join parse in solrconfig.xml

Hi Erick, Thanks for your response. Here is the query I am sending: http://dev-solr:8080/solr/collection1/select?q={!join+from=litigation_id_ls+to=lit_id_lms}all_text:apple&fq=type:PartyLawyerLawfirm&facet=true&facet.field=lawyer_id_lms&facet.mincount=1&rows=0 You can see it has "all_text:apple

Re: Measuring QPS

We built a servlet request filter that is configured in front of the Solr servlets. It reports response times to metricsd, using the Codahale library. That gives us counts, rates, and response time metrics. We mostly look at percentiles, because averages are thrown off by outliers. Average is ju

RE: Backup within SolrCloud

I withdraw this question - it is covered in the Solr 5 reference manual. The suggestion is to use the replication handler, which suggests that this scheme still works. That's how I will go. From: Davis, Daniel (NIH/NLM) [C] Sent: Monday, April 06, 2015 2:29 PM To: solr-user@lucene.apache.org

Re: Measuring QPS

Appreciated :-) Siegfried Goeschl > On 06 Apr 2015, at 20:31, Davis, Daniel (NIH/NLM) [C] > wrote: > > OK, > > I have a lot of chutzpah posting that here ;)The other guys answering the > questions can probably explain it better. > I love showing off, however, so please forgive me. > > -

RE: Measuring QPS

OK, I have a lot of chutzpah posting that here ;)The other guys answering the questions can probably explain it better. I love showing off, however, so please forgive me. -Original Message- From: Davis, Daniel (NIH/NLM) [C] Sent: Monday, April 06, 2015 2:25 PM To: solr-user@lucene.a

Backup within SolrCloud

So, we have replication, but what if something bad is indexed into the cluster, or someone accidentally deletes *:* on some collection? How do people manage backup in SolrCloud? I'm primarily interested in smaller indexes where backup is at all feasible. I imagine a system such as Facebook rea

RE: Measuring QPS

Its very common to do autocomplete based on popular queries/titles over some sliding time window. Some enterprise search systems even apply age weighting so that they don't need to re-index but continuously add to the index. This way, they can do autocomplete based on what's popular these da

Re: Measuring QPS

Hi Daniel, interesting - I never thought of autocompletion but for keeping track of user behaviour :-) * the numbers are helpful for the online advertisement team to sell campaigns * it is used for sanity checks - sensible queries returning no results or returning too many results Cheers, Sie

RE: Measuring QPS

Siegfried, It is early days as yet. I don't think we need a code drop. AFAIK, none of our current Solr applications autocomplete the search box based on popular query/title keywords. We have other applications that do that, but they don't use Solr. Thanks again, Dan -Original Messa

Re: HDFS Locking

Looks like after 900 seconds, it times out and starts up. I think the issue is that I'm using the bin/solr start/stop script, and it waits only 5 seconds before sending a kill -9. In my experience with solr 4.10.x and HDFS, that is not enough time to wait for a large shard to stop when using

Re: Measuring QPS

Hi Dan, at willhaben.at (customer of mine) two SOLR components were written for SOLR 3 and ported to SORL 4 1) SlowQueryLog which dumps long-running search requests into a log file 2) Most Frequent Search Terms allowing to query & filter the most frequent user search terms over the browser So

HDFS Locking

Having a couple issues with restarts of a 27 shard cluster using SolrCloud 5.0.0 and HDFS. I'm getting errors that a lock file exists and the shard will not start. When I delete the file, that shard starts OK. On another shard, I'm getting the following messsage: 538220 [coreLoadExecutor-5-th

RE: Measuring QPS

Siegfried, This is a wonderful find. The second presentation is a nice write-up of a large number of free tools. The first presentation prompts a question - did you add custom request handlers/code to automate determination of best user search terms? Did any of your custom work end-up in

Re: Unable to update config file using zkcli or RELOAD

The behavior has changed from Solr 5.0 onwards Please refer to the "How does it work section" here https://cwiki.apache.org/confluence/display/solr/Config+API TL:DR * Every node watches the conf set directory it is using * Updating individual files WILL NOT trigger a config reload. BUt if you m

filtering indexed documents with multiple filters

Dear all, Hi, I am looking for a way to filtering lucene index with multiple conditions. For this purpose I checked two different method of filtering search, none of them work for me: Using BooleanQuery: BooleanQuery query = new BooleanQuery(); String lower = "*"; String upper = "*";

Solr 4.2.0 index corruption issue

Hi Guys, I am using 4.2.0 since more than a year and since last October 2014 facing index corruption issue. However, now it is happening everyday and have to built a fresh index for the temporary fix. Please find the logs below where i can see an error while replicating data from master to slave a

Spellchecker for Hindi (Indian Language) Content

Hi, I have been trying to make Solr Spellchecker for Indian Local Language content (Hindi specifically) but it doesn't seem to work no matter what I try. It could be that I am missing something small, or Solr isn't equipped to handle spellcheck for Hindi content. Please let me know your thoughts.