Re: Number of fields in qf & fq

2015-11-20 Thread Scott Stults
Steve, Another thing debugQuery will give you is a breakdown of how much each field contributed to the final score of each hit. That's going to give you a nice shopping list of qf to weed out. k/r, Scott On Fri, Nov 20, 2015 at 9:26 AM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > H

Re: Number of fields in qf & fq

2015-11-20 Thread Mikhail Khludnev
Hello Steve, debugQuery=true shows whether it's facets or query, whether it's query parsing or searching (prepare vs process), cache statistics can tell about its' efficiency; sometimes a problem is obvious from request parameters. Simple sampling with jconsole or even by jstack can point on a smo

Re: Number of fields in qf & fq

2015-11-20 Thread Steven White
Thanks Erick. The 1500 fields is a design that I inherited. I'm trying to figure out why it was done as such and what it will take to fix it. What about my other question: how does one go about debugging performance issues in Solr to find out where time is mostly spent? How do I know my Solr pa

Re: Number of fields in qf & fq

2015-11-19 Thread Erick Erickson
An fq is still a single entry in your filterCache so from that perspective it's the same. And to create that entry, you're still using all the underlying fields to search, so they have to be loaded just like they would be in a q clause. But really, the fundamental question here is why your design

Re: Number of fields in qf & fq

2015-11-19 Thread Walter Underwood
The implementation for fq has changed from 4.x to 5.x, so I’ll let someone else answer that in detail. In 4.x, the result of each filter query can be cached. After that, they are quite fast. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Nov 19,

Re: Number of fields in qf & fq

2015-11-19 Thread Steven White
Thanks Walter. I see your point. Does this apply to fq as will? Also, how does one go about debugging performance issues in Solr to find out where time is mostly spent? Steve On Thu, Nov 19, 2015 at 6:54 PM, Walter Underwood wrote: > With one field in qf for a single-term query, Solr is fetc

Re: Number of fields in qf & fq

2015-11-19 Thread Walter Underwood
With one field in qf for a single-term query, Solr is fetching one posting list. With 1500 fields, it is fetching 1500 posting lists. It could easily be 1500 times slower. It might be even slower than that, because we can’t guarantee that: a) every algorithm in Solr is linear, b) that all those