Hey, my SOLR setup looks like the following:
server running apache-tomcat with solr1.2, index size is about 1G (a bit more than 4 million documents). I have another machine that basically every minute or so sends some documents to be indexed. I have autocommit turned on with maxDocs: 5000, maxTime: 300000ms. Also on the server cron job runs twice per day to optimize the index. And sometimes it happens that the index messages come at the time of optimize running. I know that running optimize on an index that is about to have documents added or deleted is not that useful, but it did help eliminate "too many file handles open" problem. Is it bad that I try to index when the optimize is running? I do see failures on the client side from time to time, but the messages get resent and indexed eventually. One recurring problem is that once per 36 hours or so SOLR server becomes really unresponsive, just spinning crazy on CPU and it is all in java (solr) process. When I try to shut down apache, apache goes down but the java process is left running. I am trying to pin point where the problem is, and wonder if my indexing-commit is not right. The box is solely dedicated for solr, so there is really nothing else running on it. Any pointers or observations appreciated. thanks, L
