Re: Request Highlighting only for the final set of rows

2017-08-18 Thread Nawab Zada Asad Iqbal
Actually, part of me is thinking that there are valid use cases for having fl and hl.fl with different values. e.g, receive name etc. in “clean” form in fl field and receive both name and address in html formatted form (by specifying in hl.fl) On Fri, Aug 18, 2017 at 10:57 AM, Nawab Zada Asad Iqb

Re: Request Highlighting only for the final set of rows

2017-08-18 Thread Nawab Zada Asad Iqbal
Actually, i realize that it is an incorrect use on my part to pass only id+score in fl and specify more fields in the hl.fl fields. This was somehow supported in older versions but the new behavior is actually a performance improvement for the scenario when user is asking for only ids. Nawab On

Re: Request Highlighting only for the final set of rows

2017-08-18 Thread Nawab Zada Asad Iqbal
Thanks Erick for the pointing to better option. I will explore that. After your email, I found that if i have specified 'fl=*' in the query then it is doing the right thing (a 2 pass process). However, my queries had 'fl=id+score' (or sometimes fl=id&fl=score), in both of these cases I found that t

Re: Request Highlighting only for the final set of rows

2017-08-18 Thread Erick Erickson
I don't think you're reading it correctly. First of all, if you're going to do be doing deep paging you should be using cusorMark, see: https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results. Second, it's a two-pass process if you don't use cursormark. The first pass gets the candi

Request Highlighting only for the final set of rows

2017-08-17 Thread Nawab Zada Asad Iqbal
Hi, In a multi-node solr installation (without SolrCloud), during a paging scenario (e.g., start=1000, rows=200), the primary node asks for 1200 rows from each shard. If highlighting is ON, then the primary node is asking for highlighting all the 1200 results from each shard, which doesn't scale w