it sounds like we may have a very bad bug in the XmlUpdateRequestHandler to clarify for people who may not know: the long standing "/update" URL has historicaly been handled using a custom servlet, recently some of that code was refactored into a RequestHandler along with a new Dispatcher for RequestHandlers that works based on path mapping -- the goal being to allow more customizable update processing and start accepting updates in a variety of input formats ... if XmlUpdateRequestHandler is mapped to the name "/update" it intercepts requests to the legacy update servlet, and should have functioned exactly the same way.
Based on Brain's email, it sounds like it didn't work in *exactly* the same way, because it caused some filedescriptor leaks (and possibly some memory leaks) Hopefully Ryan will be a rock star and spot the probably immediately -- but i'll try to look into it later this weekend. : Date: Fri, 23 Feb 2007 22:33:10 -0500 : From: Brian Whitman <[EMAIL PROTECTED]> : Reply-To: solr-user@lucene.apache.org : To: solr-user@lucene.apache.org : Subject: Re: lots of inserts very fast, out of heap or file descs : : On Feb 23, 2007, at 8:31 PM, Yonik Seeley wrote: : : >> -- it does not go down until I : >> restart solr. This would be the cause of my too many files open : >> problem. Turning off autocommit / not commiting after every add keeps : >> this count steady at 100-200. The files are all of type: : > [...] : >> Bug or feature? : > : > If the searchers holding these index files open are still working, : > then this is a problem, but not exactly a bug. If not, you may have : > hit a new bug in searcher synchronization. : : It doesn't look like it. I hope I'm not getting a reputation on here : for "discovering" bugs that seem to be my own fault, you'd all laugh : if you knew how much time I wasted before posting about it this : evening... : : But I just narrowed this down to a bad line in my solrconfig.xml. : : The one I was using said this for some reason : : : <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" : : and the trunk version says this: : : <requestHandler name="/update/xml" : class="solr.XmlUpdateRequestHandler" : : changing my line to the trunk line fixed the fdesc problem. : : The confounding thing to me is that the solr install worked fine : otherwise. I don't know what would make removing the /xml path make a : ton of files open but everything else work OK. : : If you want to reproduce it: : : 1) Download trunk/nightly : 2) Change line 347 of example/solr/conf/solrconfig.xml to : <requestHandler name="/update" class="solr.XmlUpdateRequestHandler"> : 3) java -jar start.jar... : 3) Run post.sh a bunch of times on the same xml file... (in a shell : script or whatever) : 4) After a few seconds/minutes jetty will crash with "too many open : files" : : Now, to see if this also caused my heap overflow problems. Thanks : Mike and Yonik... : : : -Hoss
