Pinging again. Anyone has ideas on this? Thanks
On Sat, Oct 14, 2017 at 4:52 PM, Sundeep T <sundeep....@gmail.com> wrote:
> Hello,
>
> In our scale environment, we see that the deep paging queries using
> cursormark are running really slow. When we traced out the calls, we see
> that the second query which queries the individual id's of matched pages is
> sending the q param that is already sent by the first query again. If we
> remove the q param and directly query for ids, the query runs really fast.
>
> For example, the initial pagination query is like this with q param on
> timestamp field -
>
> 2017-10-14 12:20:51.647 UTC INFO (qtp331844619-393343)
> [core='x:c6e422fc3054c475-core-1'] org.apache.solr.core.SolrCore.
> Request@2304 [c6e422fc3054c475-core-1] webapp=/solr path=/select
> params={distrib=false&df=text&paginatedQuery=true&fl=id&
> shards.purpose=4&start=0&fsv=true&sort=timestamp+desc+,id+asc&shard.url=
> http://ops-data-solr-svc-1.rattle.svc.cluster.local:80/solr/
> c6e422fc3054c475-core-1&*rows=50*&version=2&
> *q=(timestamp:["2017-10-13T18:42:36Z"+TO+"2017-10-13T21:09:00Z"])*&
> shards.tolerant=true&*cursorMark=**&NOW=1507928978918&isShard=
> true&timeAllowed=-1&wt=javabin&trackingId=d5eff5476247487555b7413214648}
> hits=40294067 status=0 QTime=12727
>
> This query results in a second query due to solr implementation of deep
> paging like below. In this query, we already know the ids to be matched.
> So, there is no reason to pass the q param again. We tried manually
> executing the below query without the q param and just passing the ids
> alone and that executes in 50ms. So, this looks like a bug that Solr is
> passing in the q param again. Any ideas if there is workaround for this
> problem we can use?
>
> 2017-10-14 12:21:09.193 UTC INFO (qtp331844619-742579)
> [core='x:6d63f95961c46475-core-1'] org.apache.solr.core.SolrCore.
> Request@2304 [6d63f95961c46475-core-1] webapp=/solr path=/select
> params={distrib=false&df=text&paginatedQuery=true&fl=*,[
> removedocvaluesuffix]&shards.purpose=64&shard.url=http://
> ops-data-solr-svc-1.rattle.svc.cluster.local:80/solr/
> 6d63f95961c46475-core-1&rows=50&version=2&
> *q=(timestamp:["2017-10-14T08:50:16.340Z"+TO+"2017-10-14T19:19:50Z"])*&
> shards.tolerant=true&NOW=1507983581099&ids=00f037832e571941ed46ddd1959205
> 02,145c82e3eaa7678564b9e520822a3de1,09633cfabc6c830dfb44e04c313ba6b4,
> 0032a76ed4ea01207c2891070348ea39,1b5179ee23fe3e17236da37d6b8d991f,
> 04ee42e481b2a657bd3bb3c9f91b5ed5,2a910cf8a259925046a0c9fb5ee013c3,
> 1d1d607b03c18ec59c14c2f9ca0ab47f,034e775c96633dae7e629a1d37da86e6,
> 2759ca26d449d5df9f41689aa8ed3bac,16995a57699a7bb56d5018fe145028ce,
> 0509d16399e679470ffc07a8af22a918,1797ab6e0174c65bf2f6b650b3538407,
> 11c804ec4ae153a31929abe8613de336,11d20ed5dc0cf3d71f57aefc4e4b3ee2,
> 0135baecd2d3ae819909a0c021bbd48b,224b0671196fd141196b15add2e49b91,
> 271088227cf81e3641130d3bd5de8cc6,01f266b9c130239a06b00e45bda277a0,
> 1438bed6ffd956f1c49d765b942f1988,2fc9fef6500124b1b48218169a7cf974,
> 2d85d00593847398bf09e168bb3a190c,10e1c2803df1db3d47e525d3bd8a1868,
> 28b6d72729e79da3ad65ac3879740133,14be34af9995721b358b3fdb0bcb18d7,
> 1f2e0867bd495b8a332b8c8bd8ce2508,12cf1a1c07d9b9550ece4079b15f7583,
> 022cd0b3eef93cd9a6389c0394cf3899,11aa3132e00a96df6a49540612b91c8f,
> 0ff348e0433c9e751f1475db6dcab213,2b48279c9ff833f43a910edfa455a31d,
> 241e002d744ff0215155f214166fdd49,0fee30860c82d9a24bb8389317cd772c,
> 07f04d380832f514b0575866958eebaa,20b0efa5d88e2a9950fa4fd8ba930455,
> 14a9cadb7c75274bfc028bb9ae31236b,1829730aa4ee4750eb242266830b576b,
> 1ad5012e83bd271cf00b0c70ea86a856,0af4247d057bd833753e1f7bef959fc4,
> 0a09767d81cb351ab1598987022b6955,2f166fae9ca809642b8e20cea3020c24,
> 2c4d900575d8594a040c94751af59cb5,03f1c46a004a4e3b995295b512e1e324,
> 2c2aae83afc7426424c7de5301f8c692,034baf21ac1db436a7f3f2cf2cc668b0,
> 1dda29d03fb8611f8de80b90685fd9ee,0632292ab704dcaa606440cb1fee017b,
> 0fbd68f293c6964458a93f3034348625,2cdff46ab2e4d44b42f3381d5e3250b7,
> 1b2c90dce4a51b5e5c344fc2f9ab431d&isShard=true&timeAllowed=-
> 1&wt=javabin&trackingId=d5eff5476247487555b80c9ac7b82} status=0
> QTime=18136
>
> Thanks
> Sundeep
>
