Hi, everyone, this is a problem I've had for quite a while, and have basically avoided optimizing because of it. However, eventually we will get to the point where we must delete as well as add docs continuously.
I have a Solr 1.3 index with ~4M docs at around 90G. This is a single instance running inside tomcat 6, so no replication. Merge factor is the default 10. ramBufferSizeMB is 32. maxWarmingSearchers=4. autoCommit is set at 3 sec. We continually push new data into the index, at somewhere between 1-10 docs every 10 sec or so. Solr is running on a quad-core 3.0GHz server. under IBM java 1.6. The index is sitting on a local 15K scsi disk. There's nothing else of substance running on the box. Optimizing the index takes about 65 min. As long as I'm not optimizing, search and indexing times are satisfactory. When I start the optimize, I see massive problems with timeouts pushing new docs into the index, and search times balloon. A typical search while optimizing takes about 1 min instead of a few seconds. Can anyone offer me help with fixing the problem? Thanks, Jerry Quinn
