Hi, We are currently working on a project where we are making heavy use of /export in Solr in order to stream data back. We have an index with about 16 fields that are all docvalues fields and any number of them may be requested to be streamed in results. Our index has ~450 million documents spread across 10 shards.
We are creating a CloudSolrStream and when we call CloudSolrStream.open() we see that call being slower than we had hoped. For some queries, that call can take 800 ms. What we found interesting was that doing the same request repeatedly resulted in the same time of 800 ms, which seems to indicate that /export does not take advantage of caching or there is something else at play. I’m starting to dig through the code to better understand, but I wanted to reach out to see what sort of expectations we should have here and if there is anything we can do to increase performance of these requests. We are currently using Solr 5, but we’ve also tried with Solr 7 and seen similar results. If I can provide any additional information, please let me know. Thank you! Justin
