Something else to consider is using the compound file format to reduce the number of files for your index.
this is mentioned in the Lucen FAQ... http://wiki.apache.org/jakarta-lucene/LuceneFAQ#head-48921635adf2c968f7936dc07d51dfb40d638b82 ...and configurable in solrconfig.xml, search the example for "useCompoundFile" : Date: Mon, 14 Aug 2006 07:36:00 +0100 : From: David Wiblin <[EMAIL PROTECTED]> : Reply-To: solr-user@lucene.apache.org : To: solr-user@lucene.apache.org : Subject: RE: SOlr crashes : : : A common problem with any server running lucene becasue it needs to open lots of file handles - often more than the linux default of 1024. : : Here's the fix for a linux system... : : Edit the file: /etc/security/limits.conf : : Add to the bottom: : : * soft nofile 16384 : * hard nofile 16384 : : Restart any console windows you have open and when you run ulimit -a you should get something like: : : core file size (blocks, -c) 0 : data seg size (kbytes, -d) unlimited : file size (blocks, -f) unlimited : max locked memory (kbytes, -l) unlimited : max memory size (kbytes, -m) unlimited : open files (-n) 16384 : pipe size (512 bytes, -p) 8 : stack size (kbytes, -s) 8192 : cpu time (seconds, -t) unlimited : max user processes (-u) 7168 : virtual memory (kbytes, -v) unlimited : : Restarting Solr... your "too many file handles" crash should be reoslved. : : Best regards, : : David : : : : -----Original Message----- : From: farhan diwan [mailto:[EMAIL PROTECTED] : Sent: 14 August 2006 07:29 : To: solr-user@lucene.apache.org : Subject: SOlr crashes : : : It crashes giving this output. : : =java.net.SocketException: Too many open files : at java.net.ServerSocket.createImpl(ServerSocket.java:255) : at java.net.ServerSocket.getImpl(ServerSocket.java:205) : at java.net.ServerSocket.bind(ServerSocket.java:319) : at java.net.ServerSocket.<init>(ServerSocket.java:185) : at org.mortbay.util.ThreadedServer.newServerSocket( : ThreadedServer.java:391) : at org.mortbay.util.ThreadedServer.open(ThreadedServer.java:477) : at org.mortbay.util.ThreadedServer.start(ThreadedServer.java:503) : at org.mortbay.http.SocketListener.start(SocketListener.java:204) : at org.mortbay.http.HttpServer.doStart(HttpServer.java:716) : at org.mortbay.util.Container.start(Container.java:72) : at org.mortbay.jetty.Server.main(Server.java:460) : at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) : at sun.reflect.NativeMethodAccessorImpl.invoke( : NativeMethodAccessorImpl.java:39) : at sun.reflect.DelegatingMethodAccessorImpl.invoke( : DelegatingMethodAccessorImpl.java:25) : at java.lang.reflect.Method.invoke(Method.java:585) : at org.mortbay.start.Main.invokeMain(Main.java:151) : at org.mortbay.start.Main.start(Main.java:476) : at org.mortbay.start.Main.main(Main.java:94) : -Hoss
