: I tried this "poor mans" cursor approach out ad-hoc, but I get OOM. Pretty : sure this is because you need all uniqueKey-values in FieldCache in order to : be able to sort on it. We do not have memory for that - and never will. Our : uniqueKey field is not DocValue. : Just out of curiosity : * Will I have the same OOM problem using the CURSOR-feature in later Solrs? : * Will the "poor mans" cursor approach still be efficient if my uniqueKey was : DocValued, knowing that all values for uniqueKey (the DocValue file) cannot : fit in memory (OS file cache)?
If you are getting an OOM from the sort because of the FieldCache size, then it doesn't matter if you use cursorMark or the poor mans approach mentioned: neither apoorach ever has a chance if you can't sort the docs in the first place. If DocValues on your uniqueKey field let you sort on it w/o OOM (by leveraing the OS's FS cache) then either cursorMakr or the poor mans cursor using fq should both be equally efficient. Bottom line: it's not the cursor or the fq that's ever going to be a performance problem for you in terms of using RAM -- it's just a question of sorting. -Hoss http://www.lucidworks.com/
