Solr performance improved under heavy load

2014-07-09 Thread Utkarsh Sengar
I run a small solr cloud cluster (4.5) of 3 nodes, 3 collections with 3 shards each. Total index size per node is about 20GB with about 70M documents. In regular traffic (27-50 rpm) the performance is ok and response time ranges from 100 to 500ms. But when I start loading (overwriting) 70M documen

Denormalize or use multivalued field for nested data?

2014-04-30 Thread Utkarsh Sengar
I have to modify a schema where I can attach nested "pricing per store" information for a product. For example: 10010137332:{ title:"iPad 64gb" description: "iPad 64gb with retina" pricing:{ merchantid64354:{ locationid643:{ "USD|600" }

Re: Investigating performance issues in solr cloud

2014-04-08 Thread Utkarsh Sengar
d this? Thanks, -Utkarsh On Tue, Apr 8, 2014 at 5:37 PM, Shawn Heisey wrote: > On 4/8/2014 6:00 PM, Utkarsh Sengar wrote: > > Lots of questions indeed :) > > > > 1. Total virtual machines: 3 > > 2. Replication factor: 0 (don't have any replicas yet) > > 3

Re: Investigating performance issues in solr cloud

2014-04-08 Thread Utkarsh Sengar
f ZK is running on these machine. 6. top screenshot: http://i.imgur.com/g6w9Bim.png Thanks! On Tue, Apr 8, 2014 at 4:48 PM, Shawn Heisey wrote: > On 4/8/2014 5:30 PM, Utkarsh Sengar wrote: > > I see sudden drop in throughput once every 3-4 days. The "downtime" is >

Investigating performance issues in solr cloud

2014-04-08 Thread Utkarsh Sengar
I see sudden drop in throughput once every 3-4 days. The "downtime" is for about 2-6minutes and things stabilize after that. But I am not sure what is causing it the problem. I have 3 shards with 20GB of data on each shard. Solr dashboard: http://i.imgur.com/6RWT2Dj.png Newrelic graphs when durin

Re: implement relevency

2014-01-28 Thread Utkarsh Sengar
Hi Rashmi, Relevancy needs some kind of training data which can lead to a chicken and egg problem. If you dont have that training set, then you need to come up with it or train manually (provide some seed). Our existing search had 2 years worth clickstream data, i.e. we know if someone searches fo

Re: Complex nested structure in solr

2014-01-27 Thread Utkarsh Sengar
,merchantid_800 892828282,[82,82], 922932932,,[22,23] http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201208.mbox/%3CCAEFAe-Hew1CKk=EyqACFUTKqGHExXZLSHtyrgym09aYQVJf=t...@mail.gmail.com%3E Thanks, -Utkarsh On Fri, Jan 24, 2014 at 12:05 PM, Utkarsh Sengar wrote: > Hi guys, > >

Complex nested structure in solr

2014-01-24 Thread Utkarsh Sengar
Hi guys, I have to load extra meta data to an existing collection. This is what I am looking for: For a UPC: Store availability by merchantId per location (which has lat/lon) My query pattern will be: Given a keyword, find all available products for a merchantId around the given lat/lon. Exampl

Re: shard merged into a another shard as replica

2014-01-22 Thread Utkarsh Sengar
soon. > > - Mark > > > > On Jan 22, 2014, 6:14:10 PM, Utkarsh Sengar > wrote: solr 4.4.0 > > > On Wed, Jan 22, 2014 at 3:12 PM, Mark Miller > wrote: > > > What version of Solr are you running? > > > > - Mark > > > > > > > &

Re: shard merged into a another shard as replica

2014-01-22 Thread Utkarsh Sengar
solr 4.4.0 On Wed, Jan 22, 2014 at 3:12 PM, Mark Miller wrote: > What version of Solr are you running? > > - Mark > > > > On Jan 22, 2014, 5:42:30 PM, Utkarsh Sengar > wrote: I am not sure what happened, I updated merchant collection and then > restarted all the

shard merged into a another shard as replica

2014-01-22 Thread Utkarsh Sengar
I am not sure what happened, I updated merchant collection and then restarted all the solr machines. This is what I see right now: http://i.imgur.com/4bYuhaq.png merchant collection looks fine. But deals and prodinfo collections should have a total of 3 shards. But someone shard1 has converted to

Re: Trigger event on change of a field in a document

2013-12-27 Thread Utkarsh Sengar
Solr & ElasticSearch Support > http://sematext.com/ > On Dec 27, 2013 6:19 PM, "Otis Gospodnetic" > wrote: > > > Hi, > > > > This sounds like it would be best implemented outside the search engine. > > > > Otis > > Solr & Elas

Trigger event on change of a field in a document

2013-12-27 Thread Utkarsh Sengar
I am experimenting with implementing a price drop feature. Can I register some document's fields and trigger some sort of events if the values change in those fields? For example: 1. Price of itemX is $10 2. Say the price changes to $17 or $5 (increases or decreases) when the new data loads. 3. Tr

Re: What is the difference between "attorney:(Roger Miller)" and "attorney:Roger Miller"

2013-11-19 Thread Utkarsh Sengar
Also, attorney:(Roger Miller) is same as attorney:"Roger Miller" right? Or the term "Roger Miller" is run against attorney? Thanks, -Utkarsh On Tue, Nov 19, 2013 at 12:42 PM, Rafał Kuć wrote: > Hello! > > In the first one, the two terms 'Roger' and 'Miller' are run against > the attorney field

Re: High disk IO during UpdateCSV

2013-11-13 Thread Utkarsh Sengar
its at each point. > > You will have a bottleneck somewhere, usually disk or CPU. Yours appears > to be disk. If you get faster disks, it might become the CPU. > > wunder > > On Nov 13, 2013, at 8:22 AM, Utkarsh Sengar wrote: > > > Bumping this one again, any suggestio

Re: High disk IO during UpdateCSV

2013-11-13 Thread Utkarsh Sengar
; https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts > > > w: appinions.com <http://www.appinions.com/> > > > On Wed, Nov 13, 2013 at 11:22 AM, Utkarsh Sengar >wrote: > > > Bumping this one again, any suggestions? > > > > > >

Re: High disk IO during UpdateCSV

2013-11-13 Thread Utkarsh Sengar
Bumping this one again, any suggestions? On Tue, Nov 12, 2013 at 3:58 PM, Utkarsh Sengar wrote: > Hello, > > I load data from csv to solr via UpdateCSV. There are about 50M documents > with 10 columns in each document. The index size is about 15GB and I am > using a 3 node d

High disk IO during UpdateCSV

2013-11-12 Thread Utkarsh Sengar
Hello, I load data from csv to solr via UpdateCSV. There are about 50M documents with 10 columns in each document. The index size is about 15GB and I am using a 3 node distributed solr cluster. While loading the data the disk IO goes to 100%. if the load balancer in front of solr hits the machine

Re: Stop/Restart Solr

2013-10-22 Thread Utkarsh Sengar
We use this to start/stop solr: Start: java -Dsolr.clustering.enabled=true -Dsolr.solr.home=multicore -Djetty.class.path=lib/ext/* -Dbootstrap_conf=true -DnumShards=3 -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar Stop: java -Dsolr.solr.home=multicore -Dbootstrap_conf=true -DnumShards=3 -

Re: Check if dynamic columns exists and query else ignore

2013-10-18 Thread Utkarsh Sengar
Thanks Chris! That worked! I overengineered my query! Thanks, -Utkarsh On Fri, Oct 18, 2013 at 12:02 PM, Chris Hostetter wrote: > > : I trying to do this: > : > : if (US_offers_i exists): > :fq=US_offers_i:[1 TO *] > : else: > :fq=offers_count:[1 TO *] > > "if()" and "exist()" are funct

Re: Check if dynamic columns exists and query else ignore

2013-10-18 Thread Utkarsh Sengar
Bumping this one, any suggestions? Looks like if() and exists() are meant to solve this problem, but I am using it in a wrong way. -Utkarsh On Thu, Oct 17, 2013 at 1:16 PM, Utkarsh Sengar wrote: > I trying to do this: > > if (US_offers_i exists): >fq=US_offers_i:[1 TO *] >

Check if dynamic columns exists and query else ignore

2013-10-17 Thread Utkarsh Sengar
I trying to do this: if (US_offers_i exists): fq=US_offers_i:[1 TO *] else: fq=offers_count:[1 TO *] Where: US_offers_i is a dynamic field containing an int offers_count is a status field containing an int. I have tried this so far but it doesn't work: http://solr_server/solr/col1/select?

Re: Using split in updateCSV for SolrCloud 4.4

2013-10-11 Thread Utkarsh Sengar
gt; to check that all of the input params are being parsed as expected. > > -Yonik > > On Thu, Oct 10, 2013 at 8:10 PM, Utkarsh Sengar > wrote: > > Didn't help. > > > > This is the complete data: https://gist.github.com/utkarsh2012/6927649(see > > merc

Re: Using split in updateCSV for SolrCloud 4.4

2013-10-10 Thread Utkarsh Sengar
rue&f.features.split=true&f.features.separator=%3A&f.features.encapsulator=%22"; -H "Content-Type: text/csv" -d ' id,name,features doc-1,doc1,"feat1:feat2"' Thanks, -Utkarsh On Thu, Oct 10, 2013 at 5:10 PM, Utkarsh Sengar wrote: > Didn'

Re: Using split in updateCSV for SolrCloud 4.4

2013-10-10 Thread Utkarsh Sengar
s.split=true&f.features.separator=%3A&f.features.encapsulator=%22>" > -H "Content-Type: text/csv" -d ' > id,name,features > doc-1,doc1,feat1:feat2' > > You may need to add &stream.contentType=text/csv to you command. > > -- Jack Krupansky >

Using split in updateCSV for SolrCloud 4.4

2013-10-10 Thread Utkarsh Sengar
Hello, I am trying to use split: http://wiki.apache.org/solr/UpdateCSV#split while loading some csv data via updateCSV. This is the field: This is the column in CSV (merchantList): values,"16179:10950",.values.. This is the URL I call: http://localhost/solr/coll1/update/csv?commit

Re: Some text not indexed in solr4.4

2013-09-24 Thread Utkarsh Sengar
WordDelimiterFilterFactory was the culprit. Removing that fixed the problem. Thanks, -Utkarsh On Tue, Sep 24, 2013 at 12:17 PM, Utkarsh Sengar wrote: > @Furkan Yes, I have run a commit, other text is searchable. > Not sure what you mean there for MultiPhraseQuery. It is mentioned in >

Re: Some text not indexed in solr4.4

2013-09-24 Thread Utkarsh Sengar
e value is actually indexed into the field by using the > Terms request handler: > > http://localhost:8983/solr/terms?terms.fl=text&terms.prefix=d > > (adjust the prefix to whatever you're looking for) > > This should get you going in the right direction. > > Jaso

Re: Some text not indexed in solr4.4

2013-09-17 Thread Utkarsh Sengar
To add to it, I see the exact problem with the queries: "nikon d7100", "nikon d5100", "samsung ps-we450" etc. Thanks, -Utkarsh On Tue, Sep 17, 2013 at 2:20 PM, Utkarsh Sengar wrote: > I have a copyField called allText with type text_general: > https://gist

Some text not indexed in solr4.4

2013-09-17 Thread Utkarsh Sengar
I have a copyField called allText with type text_general: https://gist.github.com/utkarsh2012/6167128#file-schema-xml-L68 I have ~100 documents which have the text: dyson and dc44 or dc41 etc. For example: "title": "Dyson DC44 Animal Digital Slim Cordless Vacuum" "description": "The DC44 Animal i

Re: Dynamic row sizing for documents via UpdateCSV

2013-09-17 Thread Utkarsh Sengar
re empty? > > Otherwise I'd think about parsing them externally and using, say, SolrJ > to transmit the individual records to Solr. > > Best, > Erick > > > On Mon, Sep 16, 2013 at 2:47 PM, Utkarsh Sengar >wrote: > > > Hello, > > > > I am

Dynamic row sizing for documents via UpdateCSV

2013-09-16 Thread Utkarsh Sengar
Hello, I am using UpdateCSV to load data in solr. Currently I load this schema with a static set of values: userid,name,age,location john8322,John,32,CA tom22,Tom,30,NY But now I have this usecase where john8322 might have a state specific dynamic field for example: userid,name,age,location, ca

Re: What does it mean when a shard is down in solr4.4?

2013-08-29 Thread Utkarsh Sengar
bumping this one, any suggestions? I am sure this is solrcloud 101 but I couldn't find documentation anywhere. Thanks, -Utkarsh On Wed, Aug 28, 2013 at 2:37 PM, Utkarsh Sengar wrote: > I have a 3 node solrcloud cluster with 3 shards for each collection/core. > > At times when

What does it mean when a shard is down in solr4.4?

2013-08-28 Thread Utkarsh Sengar
I have a 3 node solrcloud cluster with 3 shards for each collection/core. At times when I rebuild the index say on collectionA on nodeA (shard1) via UpdateCSV, the "Cloud" status page says that collectionA on nodeA (shard1) is down. Observations: 1. Other collections on nodeA work. 2. collectionA

solr-user@lucene.apache.org

2013-08-27 Thread Utkarsh Sengar
ffort is of real benefit > to the user and not just something someone noticed that's actually > only useful 0.001% of the time. > > Best > Erick > > > On Tue, Aug 27, 2013 at 5:00 PM, Utkarsh Sengar >wrote: > > > Yup, the query "o'reilly" work

solr-user@lucene.apache.org

2013-08-27 Thread Utkarsh Sengar
#x27;t" The char "&" will be ignored I guess. *So, my question is:* Is there a way I can make "m&m" index as one string AND also keep StandardTokenizerFactory since I need it for other searches. Thanks, -Utkarsh On Tue, Aug 27, 2013 at 11:44 A

solr-user@lucene.apache.org

2013-08-27 Thread Utkarsh Sengar
to do is look at the admin/analysis page and see what happens > at index and query time to things like o'reilly. You have > WordDelimiterFilterFactory > configured in your query but not index analysis chain. My bet on that is > that > you're getting different tokens at que

solr-user@lucene.apache.org

2013-08-26 Thread Utkarsh Sengar
Some of the queries (not all) with special chars return no documents. Example: queries returning no documents q=m&m (this can be explained, when I search for "m m", no documents are returned) q=o'reilly (when I search for "o reilly", I get documents back) Queries returning documents: q=hello&wor

Re: loading solr from Pig?

2013-08-21 Thread Utkarsh Sengar
That's a good point, we load data from pig to solr everyday. 1. What we do: Pig jobs creates a csv dump, scp it over to a solr node and UpdateCSV request handler loads the data in solr. A complete rebuild of index for about 50M documents (20GB) takes 20mins (pig job which pulls and processes data

Re: What filter to use to search with spaces omitted/included between words?

2013-08-20 Thread Utkarsh Sengar
replacing it with did the trick. Not sure how it worked. The field value I am searching is "Best Buy", but when I search for "bestbuy", it returns a result. Thanks, -Utkarsh On Tue, Aug 20, 2013 at 4:48 PM, Utkarsh Sengar wrote: > Thanks Tamanjit and Erick. >

Re: What filter to use to search with spaces omitted/included between words?

2013-08-20 Thread Utkarsh Sengar
Thanks Tamanjit and Erick. I tried out the filters, most of the usecases work except "q=bestbuy". As mentioned by Erick, that is a hard one to crack. I am looking into DictionaryCompoundWordTokenFilterFactory but compound words like these: http://www.manythings.org/vocabulary/lists/a/words.php?f=c

What filter to use to search with spaces omitted/included between words?

2013-08-19 Thread Utkarsh Sengar
I have a field which consists of a store name. How can I make sure that these queries return relevant results when searched against this column: *Example1: "Best Buy"* q=best (tokenizer filter makes this work) q=bestbuy q=buy (tokenizer filter makes this work) q=best buy (lower case filter makes t

Re: Load a list of values in a solr field and query over its items

2013-08-14 Thread Utkarsh Sengar
=query&fq=tags:tag1 Thanks! -Utkarsh On Wed, Aug 14, 2013 at 11:57 AM, Utkarsh Sengar wrote: > Thanks Aloke! > > So a multivalued field assumes: > > 1. if data is inserted in this form: "8738,624623,7272,82272,733", there > are 5 unique values separated by a com

Re: Load a list of values in a solr field and query over its items

2013-08-14 Thread Utkarsh Sengar
AM, Aloke Ghoshal wrote: > Should work once you set up both fields as multiValued ( > http://wiki.apache.org/solr/SchemaXml#Common_field_options). > > > On Thu, Aug 15, 2013 at 12:07 AM, Utkarsh Sengar >wrote: > > > Hello, > > > > Is it possible to load a

Load a list of values in a solr field and query over its items

2013-08-14 Thread Utkarsh Sengar
Hello, Is it possible to load a list in a solr filed and query for items in that list? example_core1: document1: FieldName=user_ids Value=8,6,1,9,3,5,7 FieldName=allText Value=text to be searched over with title and description document2: FieldName=user_ids Value=8738,624623,7272.82272,733 Fiel

Re: Suggest aka "autocomplete" request handler with solr 4.4

2013-08-08 Thread Utkarsh Sengar
rue, then "appl" was found in solr index. In this > case, maybe the EnglishMinimalStemFilterFactory filter in text_general > fieldType is messing your suggestion. > > > On 6 August 2013 15:33, Utkarsh Sengar wrote: > > > Jack/Chris, > > > > 1. Thi

Re: Suggest aka "autocomplete" request handler with solr 4.4

2013-08-06 Thread Utkarsh Sengar
Jack/Chris, 1. This is my complete schema.xml: https://gist.github.com/utkarsh2012/6167128/raw/1d5ac6520b666435cd040b5cc6dcb434cdfd7925/schema.xml More specifically, allText is of type: text_general which has a LowerCaseFatcory during index time. 2. allText has values: http://solr_server/solr/pro

Re: Suggest aka "autocomplete" request handler with solr 4.4

2013-08-05 Thread Utkarsh Sengar
Bumping this one, is this feature maintained anymore? Thanks, -Utkarsh On Fri, Aug 2, 2013 at 2:27 PM, Utkarsh Sengar wrote: > I am trying to get autocorrect and suggest feature work on my solr 4.4 > setup. > > As recommended here: http://wiki.apache.org/solr/Suggester, this is my

Suggest aka "autocomplete" request handler with solr 4.4

2013-08-02 Thread Utkarsh Sengar
I am trying to get autocorrect and suggest feature work on my solr 4.4 setup. As recommended here: http://wiki.apache.org/solr/Suggester, this is my solrconfig: http://apaste.info/eBPr Where "allText" is a copy field which indexes all the content I have in document title, description etc. I am t

Re: Sort top N results in solr after boosting

2013-07-30 Thread Utkarsh Sengar
Thanks guys! Will play around with it function query. Thanks, -Utkarsh On Tue, Jul 30, 2013 at 10:50 AM, Chris Hostetter wrote: > > : bq: I am also trying to figure out if I can place > : extra dimensions to the solr score which takes other attributes into > : consideration > > To re-iterate er

Re: monitor jvm heap size for solrcloud

2013-07-26 Thread Utkarsh Sengar
We have been using newrelic (they have a free plan too) and gives all needed info like: jvm heap usage in eden space, survivor space and old gen. Garbage collection info, detailed info about the solr requests and its response times, error rates etc. I highly recommend using newrelic to monitor you

Re: Sort top N results in solr after boosting

2013-07-25 Thread Utkarsh Sengar
> way to do that, I don't know how you'd manage to separate > the signal from the noise.... > > Best > Erick > > On Wed, Jul 24, 2013 at 4:37 PM, Utkarsh Sengar > wrote: > > I have a solr query which has a bunch of boost params for relevancy. This > >

Sort top N results in solr after boosting

2013-07-24 Thread Utkarsh Sengar
I have a solr query which has a bunch of boost params for relevancy. This search works fine and returns the most relevant documents as per the user query. For example, if user searches for: "iphone 5", keywords like "apple", "wifi" etc are boosted. I get these keywords from external training. The t

Re: How to use joins in solr 4.3.1

2013-07-16 Thread Utkarsh Sengar
}*:* I don't see any errors, but my original problem still persists, no documents are returned. The two fields on which I am trying to join is: Merchant: Location: Thanks, -Utkarsh On Tue, Jul 16, 2013 at 11:39 AM, Utkarsh Sengar wrote: > Looks like the JoinQParserPlugin is throwin

Re: How to use joins in solr 4.3.1

2013-07-16 Thread Utkarsh Sengar
solr log? > Sometimes what's returned in the response packet does not > include the true source of the problem. > > Best > Erick > > On Mon, Jul 15, 2013 at 7:40 PM, Utkarsh Sengar > wrote: > > I have also tried these queries (as per this SO answer: > >

Re: How to use joins in solr 4.3.1

2013-07-15 Thread Utkarsh Sengar
utureTask.java:303)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:138)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)\n\tat java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:138)\n\tat java.util.concurren

How to use joins in solr 4.3.1

2013-07-15 Thread Utkarsh Sengar
Hello, I am trying to join data between two cores: merchant and location This is my query: http://_server_.com:8983/solr/location/select?q={!join from=merchantId to=merchantId fromIndex=merchant}walgreens Ref: http://wiki.apache.org/solr/Join Merchants core has documents for the query: "walgree

Re: Improving performance to return 2000+ documents

2013-07-01 Thread Utkarsh Sengar
be > > using standard Solr queries for this or building a custom > > component that goes at the base Lucene index > > and "does the right thing". Or even re-indexing your > > entire corpus periodically to add this kind of data. > > > > FWIW, > > E

Re: Improving performance to return 2000+ documents

2013-06-30 Thread Utkarsh Sengar
't built for this use-case, is it actually > > a compelling situation? > > > > And having your document cache set at 1M is kinda > > high if you have very big documents. > > > > FWIW, > > Erick > > > > > > On Fri, Jun 28, 2013 at 8:44 P

Re: Improving performance to return 2000+ documents

2013-06-28 Thread Utkarsh Sengar
225 75%252 80%272 90%319 95%364 98%420 99%453 100%497 (longest request) Sometimes it takes a lot of time, sometimes its pretty quick. Thanks, -Utkarsh On Fri, Jun 28, 2013 at 5:39 PM, Utkarsh Sengar wrote: > Hello, > > I have a usecase wh

Improving performance to return 2000+ documents

2013-06-28 Thread Utkarsh Sengar
Hello, I have a usecase where I need to retrive top 2000 documents matching a query. What are the parameters (in query, solrconfig, schema) I shoud look at to improve this? I have 45M documents in 3node solrcloud 4.3.1 with 3 shards, with 30GB RAM, 8vCPU and 7GB JVM heap size. I have documentCac

Re: Updating solrconfig and schema.xml for solrcloud in multicore setup

2013-06-25 Thread Utkarsh Sengar
I believe I am hitting this bug: https://issues.apache.org/jira/browse/SOLR-4805 I am using solr 4.3.1 -Utkarsh On Tue, Jun 25, 2013 at 2:56 AM, Utkarsh Sengar wrote: > Yes, I have tried zkCli and it works. > But I also need to restart solr after the schema change right? > > I tri

Re: Updating solrconfig and schema.xml for solrcloud in multicore setup

2013-06-25 Thread Utkarsh Sengar
.xml for a specific collection using zkCli? Any > issues? > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > 25. juni 2013 kl. 11:24 skrev Utkarsh Sengar : > > > But as when I launch a solr instance without "-Dbootstrap_conf=tru

Re: Updating solrconfig and schema.xml for solrcloud in multicore setup

2013-06-25 Thread Utkarsh Sengar
th Solr. > See http://wiki.apache.org/solr/SolrCloud#Command_Line_Util > This means you will NOT need to start Solr with -Dboostrap_confdir at all. > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > 25. juni 2013 kl. 10:29 skrev Utkarsh Sengar : > >

Updating solrconfig and schema.xml for solrcloud in multicore setup

2013-06-25 Thread Utkarsh Sengar
Hello, I am trying to update schema.xml for a core in a multicore setup and this is what I do to update it: I have 3 nodes in my solr cluster. 1. Pick node1 and manually update schema.xml 2. Restart node1 with -Dbootstrap_conf=true java -Dsolr.solr.home=multicore -DnumShards=3 -Dbootstrap_conf=

Re: solrcloud 4.3.1 - stability and failure scenario questions

2013-06-23 Thread Utkarsh Sengar
http://wiki.apache.org/hadoop/ZooKeeper/FAQ#A7 > > Depending on you use-case, 2-3 replicas might be okay. We don't have > enough information to answer that question. > > > > On Sat, Jun 22, 2013 at 10:40 PM, Utkarsh Sengar > wrote: > > Thanks Anshum. > > > >

Re: solrcloud 4.3.1 - stability and failure scenario questions

2013-06-22 Thread Utkarsh Sengar
the SolrCloud setup > to work for you. > When you kill 1 shard, you essentially are taking away 1/3 of the range of > shard key. > > > On Sat, Jun 22, 2013 at 4:31 PM, Utkarsh Sengar wrote: > >> Hello, >> >> I am testing a 3 node solrcloud cluster wit

Re: solrcloud 4.3.1 - stability and failure scenario questions

2013-06-22 Thread Utkarsh Sengar
Just to be clear here, I when I say "I killed a node". I just killed the solr process on that node. zk on all the 3 nodes were still running. Thanks, -Utkarsh On Sat, Jun 22, 2013 at 4:01 AM, Utkarsh Sengar wrote: > Hello, > > I am testing a 3 node solrcloud cluster with 3

solrcloud 4.3.1 - stability and failure scenario questions

2013-06-22 Thread Utkarsh Sengar
Hello, I am testing a 3 node solrcloud cluster with 3 shards. 3 zk nodes are running in a different process in the same machines. I wanted to know the recommended size of a solrcloud cluster (min zk nodes?) This is the SolrCloud dump: https://gist.github.com/utkarsh2012/5840455 And, I am not su

Re: Solr4 cluster setup for high performance reads

2013-06-21 Thread Utkarsh Sengar
s? Thanks, -Utkarsh On Thu, Jun 13, 2013 at 7:28 PM, Shawn Heisey wrote: > On 6/13/2013 7:51 PM, Utkarsh Sengar wrote: > > Sure, I will reduce the count and see how it goes. The problem I have is, > > after such a change, I need to reindex everything again, which again is > >

Re: Running solr cloud

2013-06-18 Thread Utkarsh Sengar
Looks like zk does not contain the configuration called: collection1. You can use zkCli.sh to see what's inside "configs" zk node. You can manually push config via zkCli's upconfig (not very sure how it works). Try adding this arg: " -Dbootstrap_conf=true" in place of "-Dbootstrap_confdir=./solr/c

Re: Solr4 cluster setup for high performance reads

2013-06-13 Thread Utkarsh Sengar
I will hit solr 1.3M times and request 2000 documents in each query. By the current speed (just one machine), it will take me ~20 days to do the initial training. Thanks, -Utkarsh On Thu, Jun 13, 2013 at 6:25 PM, Shawn Heisey wrote: > On 6/13/2013 5:53 PM, Utkarsh Sengar wrote: > >

Solr4 cluster setup for high performance reads

2013-06-13 Thread Utkarsh Sengar
Hello, I am evaluating solr for indexing about 45M product catalog info. Catalog mainly contains title and description which takes most of the space (other attributes are brand, category, price, etc) The data is stored in cassandra and I am using datastax's solr (DSE 3.0.2) which handles incremen

Re: Not able to see newly added copyField in the response (indexing is 80% complete)

2013-05-02 Thread Utkarsh Sengar
Thanks Shawn. Find my answers below. On Thu, May 2, 2013 at 2:34 PM, Shawn Heisey wrote: > On 5/2/2013 3:13 PM, Utkarsh Sengar wrote: > > Hello, > > > > I updated my schema to use a copyField and have triggered a reindex, 80% > of > > the reindexing is complete.

Not able to see newly added copyField in the response (indexing is 80% complete)

2013-05-02 Thread Utkarsh Sengar
Hello, I updated my schema to use a copyField and have triggered a reindex, 80% of the reindexing is complete. Although when I query the data, I don't see "myNewCopyFieldName" being returned with the documents. Is there something wrong with my schema or I need to wait for the indexing to complete

How to recover from "Error opening new searcher" when machine crashed while indexing

2013-04-30 Thread Utkarsh Sengar
Solr 4.0 was indexing data and the machine crashed. Any suggestions on how to recover my index since I don't want to delete my data directory? When I try to start it again, I get this error: ERROR 12:01:46,493 Failed to load Solr core: xyz.index1 ERROR 12:01:46,493 Cause: ERROR 12:01:46,494 Error

Re: Solr's physical memory and JVM memory

2013-04-16 Thread Utkarsh Sengar
I'm not seeing your inlined image. > > It's not just you. > > > On Tue, Apr 16, 2013 at 7:52 PM, Utkarsh Sengar >wrote: > >> So I have 2 questions: > >> 1. What is the recommended memory for those 2 nodes? > >> 2. I am not sure what does

Solr's physical memory and JVM memory

2013-04-16 Thread Utkarsh Sengar
Hello, I have setup a solr4 instance (just one node) and I see this memory pattern: [image: Inline image 1] Physical memory is nearly full and JVM memory is ok. I have ~40M documents (where 1 document=1KB) indexed and in production env I am planning to setup 2 solr cloud nodes. So I have 2 ques

Re: Getting started with solr 4.2 and cassandra

2013-04-01 Thread Utkarsh Sengar
er/products-services/datastax-enterprise> > > -- Jack Krupansky > > -Original Message- From: Utkarsh Sengar > Sent: Monday, April 01, 2013 6:34 PM > To: solr-user@lucene.apache.org > Subject: Getting started with solr 4.2 and cassandra > > > Hello, > > I

Getting started with solr 4.2 and cassandra

2013-04-01 Thread Utkarsh Sengar
Hello, I am evaluating solr 4.2 and ElasticSearch (I am new to both) for a search API, where data sits in cassandra. Getting started with elasticsearch is pretty straight forward and I was able to write an ES "river" which pulls data from cassa