On 6/29/2011 11:27 AM, Shawn Heisey wrote:
On 6/29/2011 9:17 AM, Yonik Seeley wrote:
Hmmm, you could comment out the query and filter caches on both 1.4.1
and 3.2
and then run some of the queries to see if you can figure out which
are slower?
Do any of the queries have stopwords in fields where you now index
those? If so, that could entirely account for the difference.
The query cache warms very quickly, it's the filter cache that's
taking forever. I'm not intimately familiar with what is being put in
our filter queries by our webapp, but I'd be a little surprised if
there are stopwords there. A quick grep through solr logs (when I've
turned it up to INFO) for the really common ones didn't reveal any.
People do type them in fairly frequently, but they go into q= ... fq
values are constructed internally, not from what a user types, and as
far as I know, they involve fields that have never had stopwords removed.
I should add that this happens only after the index has had at least a
few hundred queries, when deletes are committed. The delete process
runs every ten minutes, and checks for document presence before issuing
the delete, which avoids unnecessary commits.
Just now, three of the six shards had documents deleted, and they took
29.07, 27.57, and 28.66 seconds to warm. The 1.4.1 counterpart to the
29.07 second one only took 4.78 seconds, and it did twice as many
autowarm queries. I know it's not my single *:* sorted warming query
(firstSearcher and newSearcher), because on solr startup with either
version, warm time is 0.01 seconds. I have useColdSearcher set to false.
Thanks,
Shawn
