I have found that it could be likely due to the hashJoin in the streaming expression, as this will store all tuples in memory?
I have more than 12 million in the collections which I am querying, in 1 shard. The index size of the collection is 45 GB. Physical RAM of server: 384 GB Java Heap: 22 GB Typical search latency: 2 to 4 seconds Regards, Edwin On 7 July 2017 at 16:46, Jan Høydahl <jan....@cominvent.com> wrote: > You have not told us how many documents you have, how many shards, how big > the docs are, physical RAM, Java heap, what typical search latency is etc. > > If you have tried to squeeze too many docs into a single node it might get > overloaded faster, thus sharding would help. > If you return too much content (large fields that you won’t use) that may > lower the max QPS for a node, so check that. > If you are not using DocValues, faceting etc will take too much memory, > but since you use streaming I guess you use Docvalues. > There are products that you can put in front of Solr that can do rate > limiting for you, such as https://getkong.org/ <https://getkong.org/> > > You really need to debug what is the bottleneck in your case and try to > fix that. > > Can you share your key numbers here so we can do a qualified guess? > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > > 2. jul. 2017 kl. 09.00 skrev Zheng Lin Edwin Yeo <edwinye...@gmail.com>: > > > > Hi, > > > > I'm currently facing the issue whereby the Solr crashed when I have > issued > > too many queries with error or those with high memory usage, like JSON > > facet or Streaming expressions. > > > > What could be the issue here? > > > > I'm using Solr 6.5.1 > > > > Regards, > > Edwin > >
