Thanks Brian. Excellent suggestion. I haven't used VisualVM before but I am going to use it to see where CPU is going. I saw that CPU is overly used. I haven't seen so much CPU use in testing. Although I think GC is not a problem, splitting the jvm per shard would be a good idea.
On Mon, Jul 16, 2012 at 9:44 PM, Bryan Loofbourrow [via Lucene] < ml-node+s472066n3995446...@n3.nabble.com> wrote: > 5 min is ridiculously long for a query that used to take 65ms. That ought > to be a great clue. The only two things I've seen that could cause that > are thrashing, or GC. Hard to see how it could be thrashing, given your > hardware, so I'd initially suspect GC. > > Aim VisualVM at the JVM. It shows how much CPU goes to GC over time, in a > nice blue line. And if it's not GC, try out its Sampler tab, and see where > the CPU is spending its time. > > FWIW, when asked at what point one would want to split JVMs and shard, on > the same machine, Grant Ingersoll mentioned 16GB, and precisely for GC > cost reasons. You're way above that. Maybe multiple JVMs and sharding, > even on the same machine, would serve you better than a monster 70GB JVM. > > -- Bryan > > > -----Original Message----- > > From: Mou [mailto:[hidden > > email]<http://user/SendEmail.jtp?type=node&node=3995446&i=0>] > > > Sent: Monday, July 16, 2012 7:43 PM > > To: [hidden email]<http://user/SendEmail.jtp?type=node&node=3995446&i=1> > > Subject: Using Solr 3.4 running on tomcat7 - very slow search > > > > Hi, > > > > Our index is divided into two shards and each of them has 120M docs , > > total > > size 75G in each core. > > The server is a pretty good one , jvm is given memory of 70G and about > > same > > is left for OS (SLES 11) . > > > > We use all dynamic fields except th eunique id and are using long > queries > > but almost all of them are filter queires, Each query may have 10 -30 fq > > parameters. > > > > When I tested the index ( same size) but with max heap size 40 G, > queries > > > were blazing fast. I used solrmeter to load test and it was happily > > serving > > 12000 queries or more per min with avg 65 ms qtime.We had an excellent > > filtercache hit ratio. > > > > This index is only used for searching and being replicated every 7 sec > > from > > the master. > > > > But now in production server it is horribly slow and taking 5 > mins(qtime) > > > to > > return a query ( same query). > > What could go wrong? > > > > Really appreciate your suggestions on debugging this thing.. > > > > > > > > -- > > View this message in context: http://lucene.472066.n3.nabble.com/Using- > > Solr-3-4-running-on-tomcat7-very-slow-search-tp3995436.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/Using-Solr-3-4-running-on-tomcat7-very-slow-search-tp3995436p3995446.html > To unsubscribe from Using Solr 3.4 running on tomcat7 - very slow search, > click > here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3995436&code=bW91bmFuZGlAZ21haWwuY29tfDM5OTU0MzZ8Mjg1MTA5MTUw> > . > NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/Using-Solr-3-4-running-on-tomcat7-very-slow-search-tp3995436p3995449.html Sent from the Solr - User mailing list archive at Nabble.com.
