Salman, It also depends on the size of your documents. Re-analyzing 20 fields of 500 bytes each will be a lot faster than re-analyzing 20 fields with 50 KB each.
Otis ---- Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ ----- Original Message ---- > From: Grant Ingersoll <gsing...@apache.org> > To: solr-user@lucene.apache.org > Sent: Wed, January 26, 2011 10:44:09 AM > Subject: Re: Highlighting with/without Term Vectors > > > On Jan 24, 2011, at 2:42 PM, Salman Akram wrote: > > > Hi, > > > > Does anyone have any benchmarks how much highlighting speeds up with Term > > Vectors (compared to without it)? e.g. if highlighting on 20 documents take > > 1 sec with Term Vectors any idea how long it will take without them? > > > > I need to know since the index used for highlighting has a TVF file of > > around 450GB (approx 65% of total index size) so I am trying to see whether > > the decreasing the index size by dropping TVF would be more helpful for > > performance (less RAM, should be good for I/O too I guess) or keeping it is > > still better? > > > > I know the best way is try it out but indexing takes a very long time so > > trying to see whether its even worthy or not. > > > Try testing on a smaller set. In general, you are saving the process of >re-analyzing the content, so, to some extent it is going to be dependent on >how >fast your analyzer chain is. At the size you are at, I don't know if storing >TVs is worth it.
