Re: Streaming large updates with SolrJ

2014-07-03 Thread Joe Gresock
Thanks Hoss, that's a good explanation. I don't have much experience with the non-sugar parts of the API, so this was a good summary. I suppose I can at least help out the client heap size this way. On Wed, Jul 2, 2014 at 10:14 PM, Chris Hostetter wrote: > > : Now that I think about it, thoug

Re: Streaming large updates with SolrJ

2014-07-02 Thread Chris Hostetter
: Now that I think about it, though, is there a way to use the Update Xml : messages with something akin to the cloud solr server? I only see examples : posting to actual Solr instances, but we really need to be able to take : advantage of the zookeepers to send our updates to the appropriate ser

Re: Streaming large updates with SolrJ

2014-06-29 Thread Mikhail Khludnev
Joe, Looking at CloudSolrServer we can see how challenging it could be. I can suggest just a clue - you can extend BinaryResponseParser and make it lazy, that's what you need. Then, you can set to for LBHttpSolrServer, and pass it into CloudSolrServer. Wish you a good journey! On Sun, Jun 29, 20

Re: Streaming large updates with SolrJ

2014-06-29 Thread Joe Gresock
Now that I think about it, though, is there a way to use the Update Xml messages with something akin to the cloud solr server? I only see examples posting to actual Solr instances, but we really need to be able to take advantage of the zookeepers to send our updates to the appropriate servers. Th

Re: Streaming large updates with SolrJ

2014-06-28 Thread Joe Gresock
Yeah, I think that's what I'll have to do, Mikhail. I was just testing the waters to see if there was a way to do it with SolrJ. On Sat, Jun 28, 2014 at 4:11 PM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Joe, > if the heap is so tight, couldn't you post > http://wiki.apache.org/so

Re: Streaming large updates with SolrJ

2014-06-28 Thread Mikhail Khludnev
Joe, if the heap is so tight, couldn't you post http://wiki.apache.org/solr/UpdateXmlMessages by own optimized code? On Sat, Jun 28, 2014 at 3:13 AM, Joe Gresock wrote: > Is there a standard way to stream updates to Solr using SolrJ? > Specifically, we have some atomic updates for large field

Re: Streaming large updates with SolrJ

2014-06-28 Thread Ahmet Arslan
Hi Joe, I think it is not possible with SolrJ/SolrInputDocument. Please vote/watch the issue : https://issues.apache.org/jira/browse/SOLR-6199 Ahmet On Saturday, June 28, 2014 2:13 AM, Joe Gresock wrote: Is there a standard way to stream updates to Solr using SolrJ? Specifically, we have some

Streaming large updates with SolrJ

2014-06-27 Thread Joe Gresock
Is there a standard way to stream updates to Solr using SolrJ? Specifically, we have some atomic updates for large field values (hundreds of MB) we'd like to send. We're currently sending partial updates using SolrInputDocument objects, but we'd love to be able to keep less on the heap in our cli