I have encountered an issue where using DirectXmlRequest to index data on a
remote host results in eventually running out have temp disk space in the
java.io.tmpdir directory. This occurs when I process a sufficiently large
batch of files. About 30% of the temporary files end up permanent. The
filenames look like: upload__2341cdae_13c02829b77__7ffd_00029003.tmp. Has
anyone else had this happen before? The relevant code is:
DirectXmlRequest up = new DirectXmlRequest( "/update", xml );
up.process(solr);
where `xml` is a String containing Solr formatted XML, and `solr` is the
SolrServer. When disk space is eventually exhausted, this is the error message
that is repeatedly seen on the master host:
2013-01-07 19:22:16,911 [http-bio-8090-exec-2657] [] ERROR
org.apache.solr.servlet.SolrDispatchFilter [] -
org.apache.commons.fileupload.FileUploadBase$IOFileUploadException: Processing
of multipart/form-data request failed. No space left on device
at
org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:367)
at
org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:126)
at
org.apache.solr.servlet.MultipartRequestParser.parseParamsAndFillStreams(SolrRequestParsers.java:344)
at
org.apache.solr.servlet.StandardRequestParser.parseParamsAndFillStreams(SolrRequestParsers.java:397)
at
org.apache.solr.servlet.SolrRequestParsers.parse(SolrRequestParsers.java:115)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:244)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
... truncated stack trace
I am running Solr 3.6 on an Ubuntu 12.04 server. I am considering working
around this by pulling out as much as I can from XMLLoader into my client, and
processing the XML myself into SolrInputDocuments for indexing, but this is
certainly not ideal.
Ryan
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential
information, privileged material (including material protected by the
solicitor-client or other applicable privileges), or constitute non-public
information. Any use of this information by anyone other than the intended
recipient is prohibited. If you have received this transmission in error,
please immediately reply to the sender and delete this information from your
system. Use, dissemination, distribution, or reproduction of this transmission
by unintended recipients is not authorized and may be unlawful.
