shard splitting

Hi I tried to split a shard but it failed. If I try to do it again it does not start again. I see the to extra shards in /collections/messages/leader_elect/ and /collections/messages/leaders/ How can I fix this? root@solr07-dcg:/solr/messages_shard3_replica2# curl 'http://localhost:8983/so

setting the collection in cloudsolrserver without using setdefaultcollection.

Is there any way to set the collection without passing setDefaultCollection in cloudsolrserver? I'm using cloudsolrserver with spring, and would like to autowire it. Thanks msj

Re: Expanding sets of words

Fantastic! Thanks for following up - this is great. Mike On Tue, May 21, 2013 at 11:17 PM, Jack Krupansky wrote: > Ah... and the answer is: > > curl "http://localhost:8983/solr/**select/?q=(assembly+W+** > language+OR+scala)+W+**programming\

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

On 5/21/2013 9:22 PM, Umesh Prasad wrote: > This is our own implementation of data source (canon name > com.flipkart.w3.solr.MultiSPCMSProductsDataSource) , which pulls the data > from out downstream service and it doesn't cache data in RAM. It fetches > the data in batches of 200 and iterates

Re: Expanding sets of words

Ah... and the answer is: curl "http://localhost:8983/solr/select/?q=(assembly+W+language+OR+scala)+W+programming\ &df=features&defType=surround&indent=true" IOW, any quoted phrase like "a b c d" can be written in surround as a W b W c W d. Presto! I'll make sure that example is in the book

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

Hi Shawn, This is our own implementation of data source (canon name com.flipkart.w3.solr.MultiSPCMSProductsDataSource) , which pulls the data from out downstream service and it doesn't cache data in RAM. It fetches the data in batches of 200 and iterates over it when DIH asks for it. I will che

Re: MoreLikeThisHandler + Facets

The other option is to use various MLT params to return fewer similar documents to begin with. Otis Solr & ElasticSearch Support http://sematext.com/ On May 21, 2013 7:00 PM, "Jack Krupansky" wrote: > I think I follow. AFAIK, Solr does not have a provision for limiting > faceting to the "top n"

Re: How to use $docBoost in data import handler?

Alex, Thanks for your quick reply. This has almost worked! 'id' was an int field with omitNorms set to true. I set it to false, so my schema.xml has now got: --- --- I am getting the following output now: 50.60443 = (MATCH) weight(id:`#8;#0;#0;#2;#20; in 0) [DefaultSimilarity], result of: 5

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

On 5/21/2013 5:14 PM, Umesh Prasad wrote: We have sufficient RAM on machine ..64 GB and we have given JVM 32 GB of memory. The machine runs Indexing primarily. The JVM doesn't run out of memory. It is the particular IndexWriterSolrCore which has .. May be we have specified too low a memory for I

Re: How to use $docBoost in data import handler?

I believe the document level boost translates into boosts for all the fields. If so, the question is what type your fields are. The primitive fields (int, bool, and - importantly - string) do not have boosts enabled by default. Try setting omitNorms=false on field definition and see if the reindexi

RE: How do I use CachedSqlEntityProcessor?

Try this.. SKU* from CAT_TABLE WHERE CategoryLevel=1" cacheKey=*"Cat1.SKU"* cacheLookup="Product.SKU" processor="CachedSqlEntityProcessor"> Also not sure if you are using Alpha / Beta release of SOLR 4.0. In Solr 3.6, 3.6.1, 4.0-Alpha &

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

We have sufficient RAM on machine ..64 GB and we have given JVM 32 GB of memory. The machine runs Indexing primarily. The JVM doesn't run out of memory. It is the particular IndexWriterSolrCore which has .. May be we have specified too low a memory for IndexWriter .. We index mainly product data

How to use $docBoost in data import handler?

Solr version: 4.2.1 As explained here: https://wiki.apache.org/solr/DataImportHandler#Special_Commands, I am trying to use index-time document-level boost in data import handler. I have a database table 'boosts' in which the document boosts are stored against the solr unique key. In data-config.x

Re: MoreLikeThisHandler + Facets

I think I follow. AFAIK, Solr does not have a provision for limiting faceting to the "top n" documents, but that does see like a reasonable feature request. At the Lucene I presume it would simply be a matter of having a hit collector that only accepts the top n documents. But, I'm not familiar

Re: MoreLikeThisHandler + Facets

Our current index contains nearly 400k documents and will grow to a few millions. Our "more like this"-search is always based on a single document, so my query is "id:some_doc_id". For such a query I usually get at least 150k "similar" documents. This definition of "similar" is way so relaxed. U

Re: Solr4.2 - Fuzzy Search Problems

: : 2) although I set editing distance to 1 in my query (e.g. worde~1), solr : returns me results having 2 editing distance (like WORDOES, WORHEE, WORKEE, : .. ect. ) fuzzy search works on *terms* in your index -- if you use a stemme when you index your data (your schema shows that you are) then

Solr 4.x replacement for termsIndexDivisor

Due to multiple languages and dirty OCR, our indexes have over 2 billion unique terms ( http://www.hathitrust.org/blogs/large-scale-search/too-many-words-again ). In Solr 3.6 and previous we needed to reduce the memory used for storing the in-memory representation of the tii file. We originall

SPLITSHARD: Error if I try to split a shard again

Hi, I have this error if a try to split a shard again (The version of solr is 4.3.0): 19911726 [qtp1949819426-565] INFO org.apache.solr.handler.admin.CollectionsHandler ? Splitting shard : shard=shard1&action=SPLITSHARD&collection=RPS-00-12 19911729 [main-EventThread] INFO org.apache.solr.clou

RE: How do I use CachedSqlEntityProcessor?

Thank you James & bbarani. This worked in the sense that there was no error or exception in the data import. Unfortunately, I do not see any of my Category1, Category2 etc. when I retrieve the documents. If I use the first configuration of the db-data-config.xml posted in my original post, I see

Re: Date Field

: And DateFormatTransformer does not take timezone into account when it : parses the text. A bit of a stuck situation. Gah! .. i'm sorry about my previous response pointing to the wiki I realize now that I missread the docs and replied to quickly. Alex's response appears correct -- evident

Re: Date Field

Ok, so we are talking about DataImportHandler as a client. It looks like the only (documented) way to correct the timezone in DIH is the evaluator http://wiki.apache.org/solr/DataImportHandler#formatDate . However, I am not sure if evaluator can be used inside DIH. And DateFormatTransformer does n

Re: AW: Date Field

: : : Schema.xml : : : Result in Solr: 2013-04-30T22:00:00Z : I think the date should look like this: 2013-05-01T00:00:00Z Did you check the docs? if you are using Solr prior to Solr 4.1 what you are describing looks appropriate if your server is running on a machine with a UTC offset of 2

AW: Date Field

For example: original data: 05/13 Reading: Schema.xml Result in Solr: 2013-04-30T22:00:00Z I think the date should look like this: 2013-05-01T00:00:00Z How can I change this? Thanks -Ursprüngliche Nachricht- Von: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Gesendet: Dienstag, 2

ShingleFilterFactory

Hi All, I have a use case where I need to Search like this- "Apple Corporation Limited" should create the pairs like -Apple Corporation,Corporation Apple,Corporation Limited,Limited Corporation. Below is the filter I am using-

Re: Date Field

This depends on what you use. If you are using SolrJ, it takes the date itself. Other clients, you may need to ensure you use Localized formatter with UTC TimeZone when you generate the text representation of the date. This is a 'client' question, not so much the 'Solr' one. Regards, Alex. Pers

Re: Date Field

This may help you: http://stackoverflow.com/questions/2370171/another-datetime-question 2013/5/21 Benjamin Kern > How can I convert the UTC Time? > > -Ursprüngliche Nachricht- > Von: Alexandre Rafalovitch [mailto:arafa...@gmail.com] > Gesendet: Dienstag, 21. Mai 2013 22:37 > An: solr-use

AW: Date Field

How can I convert the UTC Time? -Ursprüngliche Nachricht- Von: Alexandre Rafalovitch [mailto:arafa...@gmail.com] Gesendet: Dienstag, 21. Mai 2013 22:37 An: solr-user@lucene.apache.org Betreff: Re: Date Field The date field is stored in UTC, absolute time. You need to convert to and from

Re: Date Field

The date field is stored in UTC, absolute time. You need to convert to and from UTC in the client/middleware as you see fit. Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events f

Re: Date Field

Did you check here: http://lucene.apache.org/solr/4_3_0/solr-core/org/apache/solr/schema/DateField.html 2013/5/21 Benjamin Kern > Hi all, > > how can I set the time zone for a date field? > > > > > > Thanks > >

Date Field

Hi all, how can I set the time zone for a date field? Thanks

Re: solr.LatLonType type vs solr.SpatialRecursivePrefixTreeFieldType

Thanks a lot for your response!!! -- View this message in context: http://lucene.472066.n3.nabble.com/solr-LatLonType-type-vs-solr-SpatialRecursivePrefixTreeFieldType-tp4061113p4065031.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: How do I use CachedSqlEntityProcessor?

Try like this... -- View this message in context: http://lucene.472066.n3.nabble.com/How-do-I-use-CachedSqlEntityProcessor-tp4064919p4065030.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr UI logging when using logback?

Ah I vaguely remember seeing that when we first used logback (on 4.0), as Shawn says I think its the problem that as logback is starting up, where can it log before it has configured its logging (catch-22), answer it has to log to its own internal format. If memory serves we just disabled that

Re: SOLR 4.3.0 Migration- How to use DeleteUpdateCommand?

I was able to make it work as below.. SolrQueryResponse rsp = new SolrQueryResponse(); SolrQueryRequest req = new LocalSolrQueryRequest( _requestHandler.getCore(), new ModifiableSolrParams()); SolrRequestInfo.setReque

Re: The SPLITSHARD action doesn't honor the replicationFactor of a collection

It may not be obvious but as of now the replication factor doesn't really persist in SolrCloud. There's a JIRA opened for the same but it's not as trivial as it may look. As far as the replication factor of the sub shards is concerned, the number of replicas upon split is equal to the number of

Re: Store complex (i.e. label + id) meta data in SOLR document

Have you looked at field collapsing? http://wiki.apache.org/solr/FieldCollapsing You would collapse on the id and the when solr returns the results you could extract the facet label from solr document from each group. -- View this message in context: http://lucene.472066.n3.nabble.com/Store-co

RE: How do I use CachedSqlEntityProcessor?

First remove the "where" condition from the child entities, then use the "cacheKey" and "cacheLookup" parameters to instruct DIH how to do the join. Example: See http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor , particularly the 3rd configuration option. James Dyer Ing

The SPLITSHARD action doesn't honor the replicationFactor of a collection

Hi, I'm playing a little with the new feature to SPLIT shards. In my first tests, I realised the fact that if I do a split on a shard with replicationFactor=2 per example, the result of the split operation doesn't have the same replicationFactor, Is this the supposed behaviour of split? If is the

Re: Boosting Documents

Why don't you boost during query time? Something like q=superman&qf=title^2 subject You can refer: http://wiki.apache.org/solr/SolrRelevancyFAQ -- View this message in context: http://lucene.472066.n3.nabble.com/Boosting-Documents-tp4064955p4064966.html Sent from the Solr - User mailing list

SOLR 4.3.0 - SolrCore 'collection1' is not available due to init failure

I am trying to migrate the tests for custom SOLR components written for SOLR 3.5.0 to SOLR 4.3.0..This is a simple index test for distributed search / index (not using solrcloud, just using shards)... For some reason one of the test fails with the below error message. The test pass without any issu

Re: Not able to search Spanish word with ascent in solr

Thanks Shawn, Will check & try your suggestion. -- View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-search-Spanish-word-with-ascent-in-solr-tp4064404p4064959.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Not able to search Spanish word with ascent in solr

UTF-8 characters should solve the problem! :O On Tue, May 21, 2013 at 6:12 PM, jignesh wrote: > Hello > > Thanks for your reply > > I will try your suggestion and let you know... > > Thanks for your reply again > > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/

Boosting Documents

Hi everyone, I have a small (I hope) issue, and I wish someone could point me to the right direction. I have been indexing some documents using Solr 4.1 and specifying different "boost"s for different types of documents (boost for the whole document). But when searching, I noticed that the s

Re: Not able to search Spanish word with ascent in solr

Hello Thanks for your reply I will try your suggestion and let you know... Thanks for your reply again -- View this message in context: http://lucene.472066.n3.nabble.com/Not-able-to-search-Spanish-word-with-ascent-in-solr-tp4064404p4064954.html Sent from the Solr - User mailing list archi

Re: Solr 4.0 war startup issue - apache-solr-core.jar Vs solr-core

Thanks Steve, I could find solr-core.jar in the repo but could not find apache-solr-core.jar. I think my issue got misunderstood - which is totally my fault. Anyway, I took into account Shawn's comment and will use solr-core.jar only for compiling the project - not for deploying. Thanks, Sandeep

RE: Upgrade Solr index from 4.0 to 4.2.1

Why LUCENE_42?Why not LUCENE_41? Do I still need to run IndexUpgrader or just loading will be enough? Thanks. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, May 21, 2013 2:52 PM To: solr-user@lucene.apache.org Subject: Re: Upgrade Solr index from

Re: Solr 4.0 war startup issue - apache-solr-core.jar Vs solr-core

The 4.0 solr-core jar is available in Maven Central: Steve On May 21, 2013, at 11:26 AM, Sandeep Mestry wrote: > Hi Steve, > > Solr 4.0 - mentioned in the subject.. :-) > > Thanks, > Sandeep > > > On 21

Re: Expanding sets of words

Hmmm... I did a quick test and quoted phrase wasn't working for me either. Oh well. But... it should work for the LucidWorks Search query parser! -- Jack Krupansky -Original Message- From: Mike Hugo Sent: Tuesday, May 21, 2013 11:26 AM To: solr-user@lucene.apache.org Subject: Re: Exp

Re: Expanding sets of words

I'll buy that book :) Does this work with mutli-word terms? (common lisp or assembly language) (programming or coding or development) I tried: {!surround}(common lisp OR assembly language) W (programming) but that returns a parse error. Putting quotes around the multi-word terms parses but r

Re: Solr 4.0 war startup issue - apache-solr-core.jar Vs solr-core

Hi Steve, Solr 4.0 - mentioned in the subject.. :-) Thanks, Sandeep On 21 May 2013 14:58, Steve Rowe wrote: > Sandeep, > > What version of Solr are you using? > > Steve > > On May 21, 2013, at 6:55 AM, Sandeep Mestry wrote: > > > Hi Shawn, > > > > Thanks for your reply. > > > > I'm not mixin

How do I use CachedSqlEntityProcessor?

I am using the DataImportHandler to Query a SQL Server and populate Solr with data that has hierarchical relationships. The following is an outline of my table structure: PROD_TABLE -> SKU (Primary Key) -> Title (varchar) -> Descr (varchar) CAT_TABLE -> SKU (Foreign Key) -> CategoryLe

Re: Solr 4.0 war startup issue - apache-solr-core.jar Vs solr-core

Sandeep, What version of Solr are you using? Steve On May 21, 2013, at 6:55 AM, Sandeep Mestry wrote: > Hi Shawn, > > Thanks for your reply. > > I'm not mixing versions. > The problem I faced is I want to override Highlighter from solr-core jar > and if I add that as a dependency in my proje

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

Try again on a machine with more memory. Or did you do that already? -- Jack Krupansky -Original Message- From: Umesh Prasad Sent: Tuesday, May 21, 2013 1:57 AM To: solr-user@lucene.apache.org Subject: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1 Hi All, I am hitting a

Re: Expanding sets of words

I'll make sure to include that specific example in the new Solr book. -- Jack Krupansky -Original Message- From: Mike Hugo Sent: Tuesday, May 21, 2013 12:29 AM To: solr-user@lucene.apache.org Subject: Re: Expanding sets of words Fantastic! Thanks! On Mon, May 20, 2013 at 11:21 PM,

RE: Existing Project using Hibernate, Spring and Lucene and Looking to Add Solr

Shawn, Thank you so much for your help. I appreciate you taking the time to confirm what I had a hunch about with Solr. Sometimes I don't see the forest from the trees. I was hoping Solr would help me out, but like you said, since I have an understanding of what Lucene is doing and the fact

Re: MoreLikeThisHandler + Facets

Any particular reason you would want to limit the documents for facet calculation? I mean, the whole point of the facet numbers is to let users know what's out there. You must have some other rationale in mind - what is it? -- Jack Krupansky -Original Message- From: Achim Domma Sent

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

On 5/20/2013 11:57 PM, Umesh Prasad wrote: >I am hitting an OOM error while trying to do an hard commit on one of > the cores. > > Transaction log dir is Empty and DIH shows indexing going on for 13 hrs.. > > *Indexing since 13h 22m 22s* > Requests: 5,211,392 (108/s), Fetched: 1,902,792 (40/

Re: Solr 4.0 war startup issue - apache-solr-core.jar Vs solr-core

On 5/21/2013 4:55 AM, Sandeep Mestry wrote: > I'm not mixing versions. > The problem I faced is I want to override Highlighter from solr-core jar > and if I add that as a dependency in my project then there was a clash > between solr-core.jar and the apache-solr-core.jar that comes bundled > within

Re: Insane FieldCache usage when using group.facet=true

Hi, I found a different solution. Instead of using the parameter &group.facet=true, I am now using &group.truncate=true, which is giving me the correct number of counts. Best, Elisabeth On 21 May 2013 09:55, Elisabeth Adler wrote: > Hi, > I did a few more tests but still can't get my Solr giving

Re: MLT in SolrJ vs. URL?

On 5/21/2013 4:44 AM, Darren Govoni wrote: > I compose a mlt query in a URL and get the queried result back and a > list of documents in the "moreLikeThis" section in my browser. > > When I try to execute the same query in SolrJ setting the same params, I > only get the queried result document

Re: solr UI logging when using logback?

On 5/20/2013 11:23 PM, Boogie Shafer wrote: > in the _MM_DD-HHmmssSSS.start.log i get messages like this > > cat 2013_05_15-141100827.start.log > > Establishing start.log on Wed May 15 14:11:12 PDT 2013 > 14:11:15,756 |-INFO in null - Will use configuration file > [resources/logback-access.xm

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

Hi, Maybe you van share more info, such as your java command line or jstat output from right before the oom ... Otis Solr & ElasticSearch Support http://sematext.com/ On May 21, 2013 1:58 AM, "Umesh Prasad" wrote: > Hi All, >I am hitting an OOM error while trying to do an hard commit on one

Re: Posting schema as a stream instead of filename

There's no way that I know of to do this, and there are a series of problems if you try. For instance, if you refer to stopwords, how do you get the stopwords file up? Ditto for protected words. Ditto for currency.xml and any other file referenced from schema.xml. This sounds like an XY problem, w

Thai language and removed position filter

Hi, The wiki mentions to use the removed position filter at query time for the Thai language. How is it supposed to work now without the position filter? http://wiki.apache.org/solr/LanguageAnalysis#Thai Thanks, Markus

Re: Upgrade Solr index from 4.0 to 4.2.1

This is always something that gives me a headache, but what happens if you change in solrconfig.xml to LUCENE_40? I'm assuming it's LUCENE_42... Best Erick On Tue, May 21, 2013 at 5:48 AM, Elran Dvir wrote: > Hi all, > > I have a 4.0 Solr (sharded/cored) index. > I upgraded Solr to 4.2.1 and tr

Re: How to handle special characters in Solr search

You need to post (at least) the original query, the results of escaping the query that produces the exception and the stack trace in order for us to offer much in the way of advice. You might review: http://wiki.apache.org/solr/UsingMailingLists Best Erick On Tue, May 21, 2013 at 1:08 AM, kreton

Posting schema as a stream instead of filename

I want to create a solr core for my application and want to post schema as an input stream instead of giving a hard coded file name. Is it possible to do that ? Any suggestions would be appreciated . Regards, Harsh -- View this message in context: http://lucene.472066.n3.nabble.com/Posting-sch

Re: Solr 4.0 war startup issue - apache-solr-core.jar Vs solr-core

Hi Shawn, Thanks for your reply. I'm not mixing versions. The problem I faced is I want to override Highlighter from solr-core jar and if I add that as a dependency in my project then there was a clash between solr-core.jar and the apache-solr-core.jar that comes bundled within the solr distribut

MLT in SolrJ vs. URL?

Hi, I compose a mlt query in a URL and get the queried result back and a list of documents in the "moreLikeThis" section in my browser. When I try to execute the same query in SolrJ setting the same params, I only get the queried result document back and no MLT docs. What's the trick here? th

Re: indexing unrelated tables in single core

thanks gora I got that...have to change in the DB itself... On Fri, May 17, 2013 at 6:27 PM, Gora Mohanty wrote: > On 16 May 2013 19:11, Rohan Thakur wrote: > > hi Mohanty > > > > I tried what you suggested of using id as common field and changing the > SQL > > query to point to id > > and us

Upgrade Solr index from 4.0 to 4.2.1

Hi all, I have a 4.0 Solr (sharded/cored) index. I upgraded Solr to 4.2.1 and tried to load the existing index with it. I got the following exception: May 21, 2013 12:03:42 PM org.apache.solr.common.SolrException log SEVERE: null:org.apache.solr.common.SolrException: Unable to create core: othe

MoreLikeThisHandler + Facets

Im calling the MoreLikeThisHandler with a query like "id:some_doc_id", so I would like to get documents which are similar to one specific document. I restrict the result to 25 rows and I calculate facets for some fields. On what data are those facets calculated? According to the documentation ou

Re: Bangalore Apache Lucene/Solr meetup

Just an update, the it has been rescheduled due to some venue availability related issues and is now on the 8th of June. On Tue, May 21, 2013 at 11:58 AM, Anshum Gupta wrote: > Hi folks, > > We just created a new meetup group for all Lucene/Solr enthusiasts in and > around Bangalore. We're holdi

Re: Insane FieldCache usage when using group.facet=true

Hi, I did a few more tests but still can't get my Solr giving me the group counts on faceting instead of the document counts. Any ideas? Thanks, Elisabeth On 17 May 2013 14:11, Elisabeth Adler wrote: > Dear all, > > I am running a grouped query including facets in my Junit Test cases > against a