Re: YAML update request handler

2008-02-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
Without breaking the existing stuff we can add another interface BinaryQueryResponse extends QueryResponseWriter{ public void write(OutputStream out, SolrQueryRequest request, SolrQueryResponse response) throws IOException; } and in the SolrDispatchFilter do something like this QueryResponseWrite

Re: YAML update request handler

2008-02-22 Thread Grant Ingersoll
The DispatchFilter could probably be modified to have the option of using the ServletOutputStream instead of the Writer. It would take some doing to maintain the proper compatibility, but it can be done, I think. Maybe we could have a /binary path or something along those lines and SolrJ

Re: YAML update request handler

2008-02-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
The API forbids use of any non-text format. The QueryResponseWriter's write() method can take only a Writer. So we cannot write any binary stream into that. --Noble On Fri, Feb 22, 2008 at 12:30 AM, Walter Underwood <[EMAIL PROTECTED]> wrote: > Python marshal format is worth a try. It is binary

Re: YAML update request handler

2008-02-22 Thread Grant Ingersoll
See https://issues.apache.org/jira/browse/SOLR-476 On Feb 22, 2008, at 5:17 AM, Noble Paul നോബിള്‍ नोब्ळ् wrote: The SolrJ client is designed with the ResponseParser as an abstract class (which is good). But I have no means to plugin my custom ResponseParser class. Add a setter method . setR

Re: YAML update request handler

2008-02-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
The SolrJ client is designed with the ResponseParser as an abstract class (which is good). But I have no means to plugin my custom ResponseParser class. Add a setter method . setResponseParser(ResponseParser parser) and have a lazy initialization of Responseparser . if(_processor == null) _process

Re: YAML update request handler

2008-02-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
For the case where we use Solrj (we control both ends) It is best to resort to a custom binary format. It works fastest and with least cost /bandwidth . We can use a custom object serialization/deserialization mechanism (java standard serialization is verbose ) which is lightweight . I can create

Re: YAML update request handler

2008-02-21 Thread Walter Underwood
Python marshal format is worth a try. It is binary and can represent the same data as JSON. It should be a good fit to Solr. We benchmarked that against XML several years ago and it was 2X faster. Of course, XML parsers are a lot faster now. wunder On 2/21/08 10:50 AM, "Grant Ingersoll" <[EMAIL

Re: YAML update request handler

2008-02-21 Thread Grant Ingersoll
XML can be a problem when it is really lengthy (lots of results, large results) such that a binary format could be useful in certain cases where we control both ends of the pipe (i.e. SolrJ.) I've seen apps that deal with really large files wrapped in XML where the XML parsing takes a sign

Re: YAML update request handler

2008-02-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
hi, The format over the wire is not of great significance because it gets unmarshalled into the corresponding language object as soon as it comes out of the wire. I would say XML/JSON should meet 99% of the requirements because all the platforms come with an unmarshaller for both of these. But,If

Re: YAML update request handler

2008-02-20 Thread alexander lind
On Feb 20, 2008, at 9:31 AM, Doug Steigerwald wrote: A few months back I wrote a YAML update request handler to see if we could post documents faster than with XMl. We did see some small speed improvements (didn't write down the numbers), but the hacked together code was probably making it