Re: Solrj ContentStreamUpdateRequest Slow

2010-08-23 Thread Chris Hostetter
: ContentStreamUpdateRequest req = new : ContentStreamUpdateRequest("/update/extract"); : : System.out.println("setting params..."); : req.setParam("stream.url", fileName); : req.setParam("literal.content_id", solrId); ContentStreamUpdateRequest exists so that you can stream content

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-20 Thread Lance Norskog
There are no unit tests for stream.file or stream.url. Tests in org.apache.solr.handler.TestCSVLoader.filename:loadLocal() intercept them and do its own thing, feeding a local file instead of the stream.file parameter. I see no proof that stream.file/stream.url should work in SolrJ or in EmbeddedSo

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-19 Thread Tod
On 8/19/2010 1:45 AM, Lance Norskog wrote: 'stream.url' is just a simple parameter. You should be able to just add it directly. I agree (code excluding imports): public class CommonTest { public static void main(String[] args) { System.out.println("main..."); try { String fileNam

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-18 Thread Lance Norskog
'stream.url' is just a simple parameter. You should be able to just add it directly. On Wed, Aug 18, 2010 at 5:35 AM, Tod wrote: > On 8/16/2010 6:12 PM, Chris Hostetter wrote: >> >> : > I think your problem may be that StreamingUpdateSolrServer buffers up >> : > commands and sends them in batches

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-18 Thread Tod
On 8/16/2010 6:12 PM, Chris Hostetter wrote: : > I think your problem may be that StreamingUpdateSolrServer buffers up : > commands and sends them in batches in a background thread. if you want to : > send individual updates in real time (and time them) you should just use : > CommonsHttpSolrSer

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-16 Thread Chris Hostetter
: > I think your problem may be that StreamingUpdateSolrServer buffers up : > commands and sends them in batches in a background thread. if you want to : > send individual updates in real time (and time them) you should just use : > CommonsHttpSolrServer : : My goal is to batch updates. My cont

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-13 Thread Tod
On 8/12/2010 8:02 PM, Chris Hostetter wrote: : It returns in around a second. When I execute the attached code it takes just : over three minutes. The optimal for me would be able get closer to the : performance I'm seeing with curl using Solrj. I think your problem may be that StreamingUpdate

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-12 Thread Chris Hostetter
: It returns in around a second. When I execute the attached code it takes just : over three minutes. The optimal for me would be able get closer to the : performance I'm seeing with curl using Solrj. I think your problem may be that StreamingUpdateSolrServer buffers up commands and sends them

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-06 Thread Tod
On 8/4/2010 11:11 PM, jayendra patil wrote: ContentStreamUpdateRequest seems to read the file contents and transfer it over http, which slows down the indexing. Try Using StreamingUpdateSolrServer with stream.file param @ http://wiki.apache.org/solr/SolrPerformanceFactors#Embedded_vs_HTTP_Post

Re: Solrj ContentStreamUpdateRequest Slow

2010-08-04 Thread jayendra patil
ContentStreamUpdateRequest seems to read the file contents and transfer it over http, which slows down the indexing. Try Using StreamingUpdateSolrServer with stream.file param @ http://wiki.apache.org/solr/SolrPerformanceFactors#Embedded_vs_HTTP_Post e.g. SolrServer server = new StreamingUpdateS

Solrj ContentStreamUpdateRequest Slow

2010-08-04 Thread Tod
I'm running a slight variation of the example code referenced below and it takes a real long time to finally execute. In fact it hangs for a long time at solr.request(up) before finally executing. Is there anything I can look at or tweak to improve performance? I am also indexing a local pdf