Put Lucene generator on a diet?

[Andrew Stevens]

Looking through the Lucene block's search generator recently, it occurred to 
me that a fair amount of the code in there was redundant - all the stuff for 
breaking the hits up into pages, and only returning one page full of actual 
hits, seems to me to be duplicating the FilterTransformer.  In fact, in the 
site I've been working on most recently, we've been using just that 
configuration - set the hits/page count on the generator to -1 (so it 
returns everything) and add the filter transformer into the pipeline after 
it to do the paging.

So I'm wondering, are there good reasons for the search generator to include 
the same functionality, or would there be any interest in a patch that 
strips it out?  The only possiblity that's occurred to me so far is perhaps 
it has this in there for performance?  The number of hits isn't likely to be 
an issue in my particular case (less than a few hundred pages in total on 
this site), but I guess it wouldn't be too good if Cocoon had to stream (and 
maybe cache) several million hits.  On the other hand, I can't imagine 
anyone would ever page through all of those anyway, so perhaps just having a 
configurable upper limit on the hit count would be sufficient?


Andrew.