Our solr index (Solr 3.4) has over 100 million docuemnts. We frequently fire one type of query on this index to get documents, do some processing and dump in another index.
Query is of the form - *((keyword1 AND keyword2...) OR (keyword3 AND keyword4...) OR ...) AND date:[date1 TO *]* No. of keywords can be in the range of 100 - 1000. We are adding sort parameter *'date asc'*. The keyword part of the query changes very rarely but date part always changes. Now there are mainly 2 problems, 1) Query takes too much time. 2) Sometimes when 'numFound' is very large for a query, It gives OOM error (I guess this is because of sort). We are not using any type of caching yet. Will caching be helpful to solve these problems? If yes, what type of cache or caching configuration is suitable to start with?
