Re: Solr LTR with high rerankDocs

2017-08-11 Thread Sebastian Klemke
Hi On Do, 2017-08-10 at 08:30 -0700, Erick Erickson wrote: > I have to confess that I know very little about the mechanics of LTR, but > I can talk a little bit about compression. > > When a stored values is retrieved for a document it is read from the > *.fdt file which is a compressed, verbatim

Re: Solr LTR with high rerankDocs

2017-08-10 Thread Erick Erickson
I have to confess that I know very little about the mechanics of LTR, but I can talk a little bit about compression. When a stored values is retrieved for a document it is read from the *.fdt file which is a compressed, verbatim copy of the field. DocValues can bypass this stored data and read dir

Solr LTR with high rerankDocs

2017-08-10 Thread Sebastian Klemke
Hi, we're currently experimenting with LTR reranking on large rerank windows (rerankDocs=1000+). On a >500M documents SolrCloud collection, we were only able to get sub-second response times with FieldValueFeature. Therefore we created a custom feature extractor that matches field values with cons