On 12/1/06, Kevin Lewandowski <[EMAIL PROTECTED]> wrote:
My solr installation has been running fine for a few weeks but now after a server reboot it starts and runs for a few seconds, then stops responding. I don't see any errors in the logfiles, apart from snapinstaller not being able to issue a commit. Also, the process is using 100% cpu and stops responding to http requests (admin interface and queries).
- check what got changed after the server reboot... anything? - turn on GC logging to see if you are running out of memory and the GC system is taking up the CPU: -Xloggc:gc.log - get a JVM thread dump via kill -3 (assuming Linux/Unix) - try rebuilding the index to see if it's something funky there
Is there anything in solr that would cause this?
Yes, if it's normally a heavily loaded server that was restarted, there can be a problem of too many search requests hitting it all at once. Many of the requests may want to sort by the same field, and all of them start generating the same fieldcache entry or filter entries at the same time. This causes a lot of memory to quickly be allocated, GC starts churning trying to free up memory, more threads pile up, etc... Part of the fix for this has recently been committed into Lucene (multiple threads won't generate the same FieldCache entry). We could also do more in Solr by not releasing the first searcher until it has been warmed by the static solrconfig.xml entries. To see if this is your problem, restart the server and make sure no traffic goes to it. Then run some queries of the same type that will be hitting it to warm it up, then turn on normal traffic. -Yonik
