are these xml files in the solr add xml format? When you post using curl, I guess it opens as many http connections as there are files. if you can write a small program to post all these files in one request, you should be able to get better perf.
the following can be the pseudo-code open connection write "<root>" for each file write filecontent write "</root>" close connection On Fri, Jul 10, 2009 at 10:23 AM, Francis Yakin<fya...@liquid.com> wrote: > > I have about 1000 folders, each folder consist 2581 xml files. Total of xml > files is ~ 2.6 millions > > I developed perl script, inside my script it's executed this cmd: > > curl http://localhost:7001/solr/update --data-binary "@0039000.xml" -H > 'Content-type:text/plain; charset=utf-8' > > It tooks me about 4 1/2 hrs to load and commit. > > I would like to know the advantages using curl to posting/add/update the xml > files to solr comparing with using WebService::Solr module? > > Is using WebService::Solr faster? > > The XML files are local on the Solr Master box, so I posting it locally( not > using wan or lan). > > Any input will be much appreciated. > > Thanks > > Francis > > > -- ----------------------------------------------------- Noble Paul | Principal Engineer| AOL | http://aol.com
