uschindler commented on PR #15823:
URL: https://github.com/apache/lucene/pull/15823#issuecomment-4149959326

   > > This is how it's used in the disj scorer, so only allow to initialize. 
Later calling addAll would lead to trouble.
   > > So basically a factory method like: produce a PQ with that size and take 
the elements from that stream.
   > > AddAll should not be allowed when heap is already populated because code 
may misuse it and fail suddenly.
   > 
   > > @uschindler - I don't completely follow your comment about addAll only 
working on a newly-constructed/empty heap. Can you elaborate a bit? I believe 
both of the addAll methods (existing and this proposed one) should work 
correctly on a heap that already contains some elements, but I might be missing 
something? Or maybe I'm misunderstanding your point? Oh wait, after re-reading, 
maybe you're describing a different option that would appropriately size a new 
heap for a provided stream? One that would handle the overflow case by 
essentially allocating a bigger PQ?
   > 
   > I am trying to understand this too :)
   
   Ok let me explain this: The problem is that addAll() does not use the same 
contract like add(). Normally PQs in Lucene are always used with overflow - and 
this is also how it differs from the JDK provided one. So I'd expect that 
addAll works identical to a repeated call to add, just faster.
   
   Actually here the addAll call is not using the important overflowing that 
add uses. And therefor it is completely useless for the "normal" PQ usage like 
in TopcDocs scoring or similar. addAll in its current form is only useful to 
the end user if you know how many items are already in the queue and that size 
is large enough to hold all added elements. Therefor the only use case I see 
for it when you create an empty PQ with appropiate size and ad all elements to 
it. If you already have some elements in it and call addAll() it behaves not as 
expected.
   
   So my proposal was to use the current addAll not as a public method, but 
more allow to create a new PQ prepopulated with the given elements. So my 
proposal was to add a separate factory method that returns a new PQ with the 
elemnts given as stream and not offer to use addAll later, unless it is fixed 
to handle overflow.
   
   So in conclusion there are 3 options to solve this:
   - keep the current method but clearly renam eit to addAllNoOverflow(), 
documenting that it adds all elements from the stream., but fails once the 
queue is full.
   - don't expose addAll at all on the PQ instance and only add a factory 
method using the stream to populate a newly created instance with initial 
elements
   - fix addAll() to behave like repeated add with overflow. I have some code 
written down already that calls uses a conventional loop with 
`stream.iterator()`, adds all elements until it hits AIOOBE. Once the AIOObE 
was hit, it rebalances the heap and adds the remaining elements with add().
   
   Is this now understandable what my problem is? PQ is a central and important 
public class in Lucene and it should not have methods that are similarily named 
but behave differently.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to