uschindler commented on PR #15823: URL: https://github.com/apache/lucene/pull/15823#issuecomment-4149959326
> > This is how it's used in the disj scorer, so only allow to initialize. Later calling addAll would lead to trouble. > > So basically a factory method like: produce a PQ with that size and take the elements from that stream. > > AddAll should not be allowed when heap is already populated because code may misuse it and fail suddenly. > > > @uschindler - I don't completely follow your comment about addAll only working on a newly-constructed/empty heap. Can you elaborate a bit? I believe both of the addAll methods (existing and this proposed one) should work correctly on a heap that already contains some elements, but I might be missing something? Or maybe I'm misunderstanding your point? Oh wait, after re-reading, maybe you're describing a different option that would appropriately size a new heap for a provided stream? One that would handle the overflow case by essentially allocating a bigger PQ? > > I am trying to understand this too :) Ok let me explain this: The problem is that addAll() does not use the same contract like add(). Normally PQs in Lucene are always used with overflow - and this is also how it differs from the JDK provided one. So I'd expect that addAll works identical to a repeated call to add, just faster. Actually here the addAll call is not using the important overflowing that add uses. And therefor it is completely useless for the "normal" PQ usage like in TopcDocs scoring or similar. addAll in its current form is only useful to the end user if you know how many items are already in the queue and that size is large enough to hold all added elements. Therefor the only use case I see for it when you create an empty PQ with appropiate size and ad all elements to it. If you already have some elements in it and call addAll() it behaves not as expected. So my proposal was to use the current addAll not as a public method, but more allow to create a new PQ prepopulated with the given elements. So my proposal was to add a separate factory method that returns a new PQ with the elemnts given as stream and not offer to use addAll later, unless it is fixed to handle overflow. So in conclusion there are 3 options to solve this: - keep the current method but clearly renam eit to addAllNoOverflow(), documenting that it adds all elements from the stream., but fails once the queue is full. - don't expose addAll at all on the PQ instance and only add a factory method using the stream to populate a newly created instance with initial elements - fix addAll() to behave like repeated add with overflow. I have some code written down already that calls uses a conventional loop with `stream.iterator()`, adds all elements until it hits AIOOBE. Once the AIOObE was hit, it rebalances the heap and adds the remaining elements with add(). Is this now understandable what my problem is? PQ is a central and important public class in Lucene and it should not have methods that are similarily named but behave differently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
