On Tue, October 22, 2013 5:23 PM Michael Lemke wrote: >On Tue, October 22, 2013 9:23 AM Toke Eskildsen wrote: >>On Mon, 2013-10-21 at 16:57 +0200, Lemke, Michael SZ/HZA-ZSW wrote: >>> QTime fc: >>> never returns, webserver restarts itself after 30 min with 100% CPU >>> load >> >>It might be because it dies due to garbage collection. But since more >>memory (as your test server presumably has) just leads to the too many >>values-error, there isn't much to do. > >Essentially, fc is out then. > >> >>> QTime=41205 facet.prefix= q=frequent_word >>> numFound=44532 >>> >>> Same query repeated: >>> QTime=225810 facet.prefix= q=ottomotor >>> numFound=909 >>> QTime=199839 facet.prefix= q=ottomotor >>> numFound=909 >> >>I am stumped on this, sorry. I do not understand why the 'ottomotor' >>query can take 5 times as long as the 'frequent_word'-one. > >I looked into this some more this morning. I noticed the java process was >doing >a lot of I/O as shown in Process Explorer. For the frequent_word it read >about >180MB, for ottomotor is was about seven times as much, ~ 1,200 MB. >
Got another observation today. The response time for q=ottomotor depends on facet.limit: QTime=59300 facet.limit=2 QTime=69395 facet.limit=4 QTime=85208 facet.limit=6 QTime=158150 facet.limit=8 QTime=186276 facet.limit=10 QTime=231763 facet.limit=15 QTime=260437 facet.limit=20 QTime=312268 facet.limit=30 For q=frequent_word the result is much less pronounced and shows only for facet.limit >= 15 : QTime=0 facet.limit=0 QTime=20535 facet.limit=1 QTime=13456 facet.limit=2 QTime=13925 facet.limit=4 QTime=13705 facet.limit=6 QTime=13924 facet.limit=8 QTime=13799 facet.limit=10 QTime=14361 facet.limit=15 QTime=14704 facet.limit=20 QTime=15189 facet.limit=30 QTime=16783 facet.limit=50 QTime=57128 facet.limit=500 Looks to me for solr to collect enough facets to fulfill the limit constraint it has to read much more of the index in the case of the infrequent word. >jconsole didn't show anything unusual according to our more experienced Java >experts here. Nor was the machine swapping. > >Is it possible to screw up an index such that this sort of faceting leads to >constant reading of the index? Something like full table scans in a db? > Michael
