Re: Replication for SolrCloud

I wonder why need to use SolrCloud replication on HDFS at all, given HDFS already provides replication and availability? The way to optimize performance and scalability should be tweaking shards, just like tweaking regions on HBase - which doesn't provide "region replication" too, isn't it? I have

Re: JSON Facet & Analytics API in Solr 5.1

Flatter please. The other nested stuff makes my head hurt. Until recently I thought I was the only person on the planet who had a hard time mentally parsing anything but the simplest JSON, but then I learned that I'm not alone at all it's just that nobody is saying it. :) Otis -- Monitoring

Re: Solr Performance with Ram size variation

Hi, Because you went over 31-32 GB heap you lost the benefit of compressed pointers and even though you gave the JVM more memory the GC may have had to work harder. This is a relatively well educated guess, which you can confirm if you run tests and look at GC counts, times, JVM heap memory pool

Re: Solr Performance with Ram size variation

On 4/17/2015 8:14 PM, Kamal Kishore Aggarwal wrote: Hi, As per this article, the linux machine is preferred to have 1.5 times RAM with respect to index size. So, to verify this, I tried testing the solr performance in different volumes of RAM allocation keeping other configuration (i.e Solid Sta

Re: Solr Performance with Ram size variation

Hi, This may be irrelevant but your machine configuration reminded me of some reading I had done some time back on memory vs ssd. Do a search on solr ssd and you should get some meaningful posts. Like this one https://sbdevel.wordpress.com/2013/06/06/memory-is-overrated/ Regards Puneet On 18 Apr 2

Solr Performance with Ram size variation

Hi, As per this article, the linux machine is preferred to have 1.5 times RAM with respect to index size. So, to verify this, I tried testing the solr performance in different volumes of RAM allocation keeping other configuration (i.e Solid State Drives, 8 core processor, 64-Bit) to be same in bot

Re: solr 4.8.0 update synonyms in zookeeper splitted files

On 4/17/2015 7:45 PM, Vincenzo D'Amore wrote: Hi Shawn, thanks for your answer. I apologise for my english, for "floating results" I meant random results in queries. As far as I know, we should split the synonyms file because of zookeeper, there is a limit in the size of files (1MB). All my sy

Re: solr 4.8.0 update synonyms in zookeeper splitted files

Hi Shawn, thanks for your answer. I apologise for my english, for "floating results" I meant random results in queries. As far as I know, we should split the synonyms file because of zookeeper, there is a limit in the size of files (1MB). All my synonyms are about 10MB. I have tried again in de

Re: Enrich search results with external data

Hi Ha, Yes, I think if you want to facet on the external field, the custom component seems to be the best option IMO. -sujit On Fri, Apr 17, 2015 at 3:02 PM, wrote: > Hi Sujit, > > > > Many thanks for your blog post, responding to my question, and suggesting > the alternative option ☺ > > > >

Re: SolrCloud 4.8.0 upgrade

Great!! Thank you very much. On Fri, Apr 17, 2015 at 7:36 PM, Erick Erickson wrote: > Solr/Lucene are supposed to _always_ read one major version back. Thus > your 4.10 should be able to read indexes produced all the way back to > (and including) 3.x. Sometimes "experimental" formats are excepte

Re: MoreLikeThis (mlt) in sharded SolrCloud

The other issue that would fix half of your problems is: https://issues.apache.org/jira/browse/SOLR-7143 On Fri, Apr 17, 2015 at 4:35 PM, Anshum Gupta wrote: > Ah, I meant SOLR-7418 . > > On Fri, Apr 17, 2015 at 4:30 PM, Anshum Gupta > wrote: > >

Re: MoreLikeThis (mlt) in sharded SolrCloud

Ah, I meant SOLR-7418 . On Fri, Apr 17, 2015 at 4:30 PM, Anshum Gupta wrote: > Hi Ere, > > Those seem like valid issues. I've created an issue : SOLR-7275 > and will create more as > I find more of

Re: MoreLikeThis (mlt) in sharded SolrCloud

Hi Ere, Those seem like valid issues. I've created an issue : SOLR-7275 and will create more as I find more of those. I plan to get to them and fix over the weekend. On Wed, Apr 15, 2015 at 5:13 AM, Ere Maijala wrote: > Hi, > > I'm trying to gat

Re: JSON Facet & Analytics API in Solr 5.1

Agreed, I also prefer the second way. I find it more readible, less verbose while communicating the same information, less confusing to mentally parse ("is 'terms' the name of my facet, or the type of my facet?..."), and less prone to syntactlcally valid, but logically invalid inputs. Let's break

Re: Solr Cloud reclaiming disk space from deleted documents

Thanks Shawn for the quick reply. Our indexes are running on SSD, so 3 should be ok. Any recommendation on bumping it up? I guess will have to run optimize for entire solr cloud and see if we can reclaim space. Thanks, Rishi. -Original Message- From: Shawn Heisey To: solr-u

Re: Using synonyms API

Try creating a document with "Crouton" in the "dummy_stuff" field and then test queries for both "Crota" and "Crouton" I suspect what you are seeing is that the mappig is one way -- ie: as if you had put "Crota => Crouton" in a synonyms.txt file. I don't believe there is any REST support yet

Re: Solr Cloud reclaiming disk space from deleted documents

On 4/17/2015 2:15 PM, Rishi Easwaran wrote: > Running into an issue and wanted to see if anyone had some suggestions. > We are seeing this with both solr 4.6 and 4.10.3 code. > We are running an extremely update heavy application, with millions of writes > and deletes happening to our indexes cons

RE: Enrich search results with external data

Hi Sujit, Many thanks for your blog post, responding to my question, and suggesting the alternative option ☺ I think I prefer your approach because we can supply our own Comparator. The reason is that we need to meet some strict requirements: we can only call the external system once to r

Re: Solr 5.0, defaultSearchField, defaultOperator ?

: df and q.op are the ones you are looking for. : You can define them in defaults section. specifically... https://cwiki.apache.org/confluence/display/solr/InitParams+in+SolrConfig : : Ahmet : : : : On Friday, April 17, 2015 9:18 PM, Bruno Mannina wrote: : Dear Solr users, : : Since toda

Re: 5.1 'unique' facet function / calcDistinct

Perfect, thank you for the information -- will have a look through those classes. Thank you, Levan -- View this message in context: http://lucene.472066.n3.nabble.com/5-1-unique-facet-function-calcDistinct-tp4200110p4200535.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: 5.1 'unique' facet function / calcDistinct

I've posted the issue here, please let me know if any additional information needs to be provided. https://issues.apache.org/jira/browse/SOLR-7417 Happy to provide the feedback, using the sub-facets has been a lot of fun, the nested facet query is especially useful. -- View this message in co

help with schema containing nested documents

Hi, I need some documentation/samples on how to create a SOLR schema with nested documents. I have been looking online but could not find anything. Thank you in advance, Nick Pandrea

RE: Spurious _version_ conflict?

Ah, starting to see the light ... thanks for your patience. First, this is a Java REST service using solrj. I am using default transport (wt=javabin, I think). But right-clicking the URL at the top of the Admin query page and selecting "open in new tab" displays the non-truncated _version_

Solr Cloud reclaiming disk space from deleted documents

Hi All, Running into an issue and wanted to see if anyone had some suggestions. We are seeing this with both solr 4.6 and 4.10.3 code. We are running an extremely update heavy application, with millions of writes and deletes happening to our indexes constantly. An issue we are seeing is that so

Re: Differentiating user search term in Solr

: It looks to me that "f" with "qq" is doing phrase search, that's not what I : want. The data in the field "title" is "Apache Solr Release Notes" if you don't wnat phrase queries then you don't want pharse queries and that's fine -- but it wasn't clear from any of your original emails because

Re: Java.net.socketexception: broken pipe Solr 4.10.2

I haven't had time to really take a look at this. But read a couple of articles regarding the hard commit and it actually makes sense. We were seeing tlogs in the multiple GBs during ingest. I will have some time in a couple of weeks to come back to testing indexing. Thanks for the help. Vy -

Re: Solr 5.0, defaultSearchField, defaultOperator ?

Hi, df and q.op are the ones you are looking for. You can define them in defaults section. Ahmet On Friday, April 17, 2015 9:18 PM, Bruno Mannina wrote: Dear Solr users, Since today I used SOLR 5.0 (I used solr 3.6) so i try to adapt my old schema for solr 5.0. I have two questions: - how c

Re: JSON Facet & Analytics API in Solr 5.1

I prefer the second way. I find it more readable and shorter. Thanks for making Solr even better ;) From: Yonik Seeley Sent: Friday, April 17, 2015 12:20 PM To: solr-user@lucene.apache.org Subject: Re: JSON Facet & Analytics API in Solr 5.1 Does anyone h

Solr 5.0, defaultSearchField, defaultOperator ?

Dear Solr users, Since today I used SOLR 5.0 (I used solr 3.6) so i try to adapt my old schema for solr 5.0. I have two questions: - how can I set the defaultSearchField ? I don't want to use in the query the df tag because I have a lot of modification to do for that on my web project. - how c

Re: JSON Facet & Analytics API in Solr 5.1

I like the first way. It matches how elasticsearch does it http://www.elastic.co/guide/en/elasticsearch/reference/1.x/search-aggregations-bucket-range-aggregation.html Can we specify explicit ranges in Solr now like we can in elasticsearch? I do like how Solr's version of aggs can be much shorte

Re: SolrCloud 4.8.0 upgrade

Solr/Lucene are supposed to _always_ read one major version back. Thus your 4.10 should be able to read indexes produced all the way back to (and including) 3.x. Sometimes "experimental" formats are excepted. In your case you should be fine since you're upgrading from 4.8.. As always, though, I'd

Re: SolrCloud 4.8.0 upgrade

Thanks for your answers, I looked at changes and we don't use DocValuesFormat. The question is, if I upgrade the SolrCloud version to 4.10, should I reload entirely all documents? Is there a binary compatibility between these two versions reading the solar home? On Fri, Apr 17, 2015 at 7:04 PM, Er

Re: JSON Facet & Analytics API in Solr 5.1

Personally I find the second form easier to read. The second level of nesting in the first example confuses me at first glance. I don't have a really strong preference here, but I vote for the second form. On Fri, Apr 17, 2015 at 9:20 AM, Yonik Seeley wrote: > Does anyone have any thoughts on

Re: facets on external field

Hi Jainam, One workaround is to use facet.query and frange query parser. facet.query={!frange l=50 u=100}field(price) Ahmet On Thursday, April 16, 2015 1:01 PM, jainam vora wrote: Hi, I am using external field for price field since it changes frequently. generate facets using external field

Re: search ignoring accents

Hi Pedro, Requirement of "Filter by "edr" should give the result "Pedro"" can be done expanding terms at index time only. You can remove the ngram filter from query analyzer. But remember that ngram filter produces a lot of tokens. Try it on analysis page. Regarding starting at the beginning o

Re: SolrCloud 4.8.0 upgrade

Look at CHANGES.txt for both Lucene and Solr, there's always an "upgrading" section for each release. Best, Erick On Fri, Apr 17, 2015 at 5:31 AM, Toke Eskildsen wrote: > Vincenzo D'Amore wrote: >> I have a SolrCloud cluster with 3 server, I would like to use stats.facet, >> but this feature i

Re: Bad contentType for search handler :text/xml; charset=UTF-8

Off the cuff, it sounds like you are making a POST request to the SearchHandler (ie: /search or /query) and the Content-TYpe you are sending is "text/xml; charset=UTF-8" In the past SearchHandler might have ignored that Content-Type, but now that structured queries can be sent as POST data, it

Re: Solr 5.x deployment in production

Thanks Shawn, this makes a lot of sense. With WAR going away and no mention of Solr deployment strategy (see: https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production) isn't good; there is a gab in Solr's release. It feels as if Solr 5.x was rushed out ignoring Windows Servers de

Re: search ignoring accents

Pedro: For your example, don't use EdgeNgrams, use just NGrams. That'll index tokens like (in the 2gram case) pe er dr ro and searching against edr would look for "ed dr". which would match. However, this isn't in line with your first example where you got results you didn't expect. You'll have t

Re: Merge indexes in MapReduce

The core admin MERGEINDEXES will work for you I'm pretty sure. You copy the NRT index over to the all-the-time box. MERGEINDEXES just takes the path to the index you want to add to the existing core. Note the warnings in the reference guide about taking care that the indexes aren't changing and co

RE: Spurious _version_ conflict?

you still haven't provided any details on what your client code looks like -- ie: what code is talking to solr? what response format is it asking for? is it JSON? what is parsing that JSON? as for the admin UI: if you are looking at a JSON response in the "Query" screen of the Admin UI, then t

RE: Spurious _version_ conflict?

Here's another data point. To work around this issue, I am converting all non-null _version_ values to the constant 1 on the way into Solr. As a result, updates work fine. Immediately after the update+commit, a /select?q=*:* returns the _version_ value of 1498715798795976700 for id == '553d0f

Re: HttpSolrServer and CloudSolrServer

Additionally when indexing, CloudSolrServer collects up the documents for each shard and routes them to the leader for that shard, moving that processing away from whatever node you happen so contact using HttpSolrServer. Finally, HttpSolrServer is a single point of failure if the node you point t

Re: Bad contentType for search handler :text/xml; charset=UTF-8

Not unless you provide a lot more details. Specifically, anything in your Solr logs that looks suspicious _and_ in your container logs (Tomcat? Jetty?). Plus the message you sent. Please review: http://wiki.apache.org/solr/UsingMailingLists Best, Erick On Thu, Apr 16, 2015 at 10:57 PM, Pavel Hl

Re: JSON Facet & Analytics API in Solr 5.1

Does anyone have any thoughts on the current general structure of JSON facets? The current general form of a facet command is: : { : } For example: top_authors : { terms : { field : author, limit : 5, }} One alternative I considered in the past is having the type in the args: top_author

Multilevel nested level support using Solr

Hi folks, In my DB, my records are nested in a folder base hierarchy: record_1 record_2 record_3 record_4 record_5 record_6 record_7 record_8 You got the idea.

RE: Spurious _version_ conflict?

Thanks for getting back. Something like that crossed my mind but I checked the values on the way into SolrJ SolrInputDocument match the values printed in the Admin Query interface and they both match the expected value in the error message exactly. Besides the difference is only in the last f

Highlighting

Hello All, I am new to solr and trying to configure highlighting. If I look at the result in xml, or json format, I can see the highlighting part of the data and it looks good. However the velocity page does not show the highlighted words on my result page. Do I need to do something extra for

Re: Nno servers hosting shard.

Hi, sounds like you hit a Full GC. Check your GC.log. Ugo On 17 Apr 2015 08:24, "Modassar Ather" wrote: > Hi, > > Any suggestion will be really helpful. Kindly provide your inputs. > > Thanks, > Modassar > > On Thu, Apr 16, 2015 at 4:27 PM, Modassar Ather > wrote: > > > Hi, > > > > I have a se

Re: 1:M connectivity

On 4/16/2015 2:27 PM, Oded Sofer wrote: > The issue is the firewall setting needed for the cloud. We do not want to > open all nodes to all others nodes. However, we found that add-index to a > specific node tries to access all other nodes though we set it to index > locally on that node only.

Re: SolrCloud Core Reload

On 4/17/2015 7:21 AM, Vincenzo D'Amore wrote: > this morning I have optimised my SolrCloud cluster (3 instances). > I have many collections, all are in shard and replica for each node. > At the end of optimisation task (about 10 minutes) all cores are optimised > on every node. > > How can be sure

Re: Range facets in sharded search

Thanks for the fast turnaround, you beat me to opening the Jira and fixed it too! Much appreciated. Thanks, Will From: Tomás Fernández Löbbe Sent: Thursday, April 16, 2015 10:26 PM To: solr-user@lucene.apache.org Subject: Re: Range facets in sharded searc

Re: Solr 5.x deployment in production

On 4/16/2015 2:07 PM, Steven White wrote: > In my case, I have to deploy Solr on Windows, AIX, and Linux (all server > edition). We are a WebSphere shop, moving away from it means I have to > deal with politics and culture. You *can* run Solr 5.0 (and 5.1) in another container, just like you coul

Re: SolrCloud Core Reload

Hi, this morning I have optimised my SolrCloud cluster (3 instances). I have many collections, all are in shard and replica for each node. At the end of optimisation task (about 10 minutes) all cores are optimised on every node. How can be sure than also reload affects all the cores? On Fri, Ap

Re: solr 4.8.0 update synonyms in zookeeper splitted files

On 4/17/2015 6:02 AM, Vincenzo D'Amore wrote: > I have solr synonyms stored in multiple files as defined in the schema: > > "sinonimi_freeling/sfaa,sinonimi_freeling/sfab,sinonimi_freeling/sfac,sinonimi_freeling/sfad,sinonimi_freeling/sfae,sinonimi_freeling/sfaf,sinonimi_freeling/sfag,sinonimi_fr

Re: spellcheck enabled but not getting any suggestions.

Shouldn't you specify a spellcheck.dictionary in your request handler? Best regards, Elisabeth 2015-04-17 11:24 GMT+02:00 Derek Poh : > Hi > > I have enabled spellcheck but not getting any suggestions withincorrectly > spelled keywords. > I added the spellcheck into the/select request handler. >

Re: SolrCloud 4.8.0 upgrade

Vincenzo D'Amore wrote: > I have a SolrCloud cluster with 3 server, I would like to use stats.facet, > but this feature is available only if I upgrade to 4.10. > May I simply redeploy new solr cloud version in tomcat or should reload all > the documents? > There are other drawbacks? Support for

solr 4.8.0 update synonyms in zookeeper splitted files

Hi All, I have solr synonyms stored in multiple files as defined in the schema: so that I can specify synonym resource in this way: I'm quite worried because I tried to update one synonym file adding at the end the new synonyms. SolrCloud didn't update its synonyms list. So I reloaded the co

RE: search ignoring accents

And for this example what filter should I use? Filter by "edr" should give the result "Pedro" The NGram create tokens starting at the beginning or the ending, and in the middle? Thanks! Pedro Figueiredo Senior Engineer pjlfigueir...@criticalsoftware.com M. 934058150 Rua Engº Frederico Ulric

RE: search ignoring accents

Hi Ahmet, Yes... the EdgeNGram is what produces those results... I need it to improve the search by name by the applications users. Thanks. Pedro Figueiredo Senior Engineer pjlfigueir...@criticalsoftware.com M. 934058150 Rua Engº Frederico Ulrich, nº 2650 4470-605 Moreira da Maia, Portugal T

Re: search ignoring accents

Hi Pedro, solr.ASCIIFoldingFilterFactory is one way to remove diacritics. Confusion comes from EdgeNGram, why do you need it? Ahmet On Friday, April 17, 2015 1:38 PM, Pedro Figueiredo wrote: Hello, What is the best way to search in a field ignoring accents? The field has the type:

search ignoring accents

Hello, What is the best way to search in a field ignoring accents? The field has the type:

spellcheck enabled but not getting any suggestions.

Hi I have enabled spellcheck but not getting any suggestions withincorrectly spelled keywords. I added the spellcheck into the/select request handler. What steps did I miss out? spellcheck list in return result: solrconfig.xml: explicit 10 text

Re: Merge indexes in MapReduce

Hi Norgorn, I think there is no ready-made tool out of the box, but you have the spare parts in the MapreduceIndexerTool :-) With little effort you can decouple the index merging component from MRIndexerTool and use based on the needs. I did the same. On Fri, Apr 17, 2015 at 10:40 AM, Norgorn wr

Re: HttpSolrServer and CloudSolrServer

If you're using SolrCloud then you should use CloudSolrServer as it is able to abstract / hide the interaction with the cluster. HttpSolrServer communicates directly with a Solr instance. Best, Andrea On 04/17/2015 10:59 AM, Vijay Bhoomireddy wrote: Hi All, Good Morning!! For Solr

HttpSolrServer and CloudSolrServer

Hi All, Good Morning!! For SolrCloud deployment, for indexing data through SolrJ, which is the preferred / correct SolrServer class to use? HttpSolrServer of CloudSolrServer? In case both can be used, when to use which? Any help please. Thanks & Regards Vijay -- The contents of t

Re: Merge indexes in MapReduce

Thank you for the reply. Out schema is: 1) Index real-time (on separate machine). 2) NRT index becomes large. 3) Copy NRT index on other machine. 3) Merge NRT-made indexes with large ("all-the-time") index. 4) Remove NRT index (until now it was available for searching). At the end we have big, opt

Re: SolrCloud Core Reload

I don't think there is any Collection level support at this point in the Solr admin UI. Whatever you do via the UI would be core level, unless I'm forgetting something. On Thu, Apr 16, 2015 at 5:15 PM, Vincenzo D'Amore wrote: > Hi all, > > I have a solrcloud cluster with 3 server and there are m

Bad contentType for search handler :text/xml; charset=UTF-8

Hi, we have migrated Solr from 5.0 do 5.1 and we can't search now, we have a ERROR for SolrCore like in subject. I can't get any info through Google. Please, can someone help what is going on? Thanks, Pavel -- View this message in context: http://lucene.472066.n3.nabble.com/Bad-contentType-

Re: 5.1 'unique' facet function / calcDistinct

> > II. Is there a way to use the stats.calcdistinct functionality and only > return the countDistinct portion of the response and not the full list of > distinct values -- as provided in the distinctValues portion of the > response. In a field with high cardinality the response size becomes too >

Re: SolrCloud Core Reload

Optimize will be distributed to all shards/replicas. I believe reload will only reload the specific core. For reloading the complete collection use the Collections API: https://cwiki.apache.org/confluence/display/solr/Collections+API On Thu, Apr 16, 2015 at 5:15 PM, Vincenzo D'Amore wrote: > Hi

Re: Nno servers hosting shard.

Hi, Any suggestion will be really helpful. Kindly provide your inputs. Thanks, Modassar On Thu, Apr 16, 2015 at 4:27 PM, Modassar Ather wrote: > Hi, > > I have a setup of 5 node SolrCloud (Lucene/Solr version 5.1.0) without > replicas. When I am executing complex and large queries with wild-ca