On Sat, 2017-10-14 at 16:52 -0700, Sundeep T wrote: > In our scale environment, we see that the deep paging queries using > cursormark are running really slow. When we traced out the calls, we > see that the second query which queries the individual id's of > matched pages is sending the q param that is already sent by the > first query again.
Use fq instead of q, so that it will be cached for subsequent calls. > If we remove the q param and directly query for ids, the query runs > really fast. Sure. But that requires you to know the IDs to be matched. [Solr-request with heavy q resulting in 40M hits] > This query results in a second query due to solr implementation of > deep paging like below. In this query, we already know the ids to be > matched. > > So, there is no reason to pass the q param again. We tried manually > executing the below query without the q param and just passing the > ids alone and that executes in 50ms. You only have the first 50 IDs from the first query. You cannot page forward without knowing the IDs that follow, which if why you need to state the q again (or convert it to fq). It is not a bug in Solr, it is a design decision. In order to provide pagination without recomputing the result set, you would need a guaranteed server-side state. Solr does not implement that pattern and thanks for that. - Toke Eskildsen, Royal Danish Library
