Re: long QTime for big index

2013-02-14 Thread Mou
t; > > > > > > -Original Message- > From: Mou <[hidden email]> > To: solr-user <[hidden email]> > Sent: Thu, Feb 14, 2013 2:35 pm > Subject: Re: long QTime for big index > > > Just to close this discussion , we solved the problem by split

Re: long QTime for big index

2013-02-14 Thread alxsss
separate cores are the same as from one core? Thanks. Alex. -Original Message- From: Mou To: solr-user Sent: Thu, Feb 14, 2013 2:35 pm Subject: Re: long QTime for big index Just to close this discussion , we solved the problem by splitting the index. It turned out that

Re: long QTime for big index

2013-02-14 Thread Mou
Just to close this discussion , we solved the problem by splitting the index. It turned out that distributed search with 12 cores are faster than searching two cores. All queries ,tomcat configuration, jvm configuration remain same. Now queries are served in milliseconds. On Thu, Jan 31, 2013 at

Re: long QTime for big index

2013-01-31 Thread Mou
Thank you again. Unfortunately the index files will not fit in the RAM.I have to try using document cache. I am also moving my index to SSD again, we took our index off when fusion IO cards failed twice during indexing and index was corrupted.Now with the bios upgrade and new driver, it is suppose

RE: long QTime for big index

2013-01-31 Thread Toke Eskildsen
Shawn Heisey [s...@elyograg.org] wrote: [...] > If you have a total index size for this JVM of 240GB, then you may not > have enough RAM to let the OS disk cache work efficiently. For that > size of index, I would plan on a system with at least 128GB of RAM, > 256GB would be better. [...] > On

Re: long QTime for big index

2013-01-31 Thread Shawn Heisey
On 1/31/2013 12:47 PM, Mou wrote: To clarify, the third shard is used to store the recently added/updated data. Two main big cores take very long to replicate ( when a full replication is required) so the third one helps us to return the newly indexed documents quickly. It gets deleted every hour

Re: long QTime for big index

2013-01-31 Thread Mou
Thank you Shawn for reading all of my previous entries and for a detailed answer. To clarify, the third shard is used to store the recently added/updated data. Two main big cores take very long to replicate ( when a full replication is required) so the third one helps us to return the newly indexe

Re: long QTime for big index

2013-01-31 Thread Shawn Heisey
On 1/31/2013 1:01 AM, Mou wrote: I am running solr 3.4 on tomcat 7. Our index is very big , two cores each 120G. We are searching the slaves which are replicated every 30 min. I am using filtercache only and We have more than 90% cache hits. We use lot of filter queries, queries are usually pr

Re: long QTime for big index

2013-01-31 Thread Mou
Thanks for your reply. No, there is no eviction, yet. The time is spent mostly on org.apache.solr.handler.component.QueryComponent to process the request. Again, the time varies widely for same query. -- View this message in context: http://lucene.472066.n3.nabble.com/long-QTime-for-big-inde

Re: long QTime for big index

2013-01-31 Thread Dmitry Kan
Does debugQuery=true tell anything useful for these? Like what is the component taking most of the 30 seconds. Do you have evictions in your solr caches? Dmitry On Thu, Jan 31, 2013 at 10:01 AM, Mou wrote: > I am running solr 3.4 on tomcat 7. > > Our index is very big , two cores each 120G. We