uschindler commented on issue #15079: URL: https://github.com/apache/lucene/issues/15079#issuecomment-3197507601
> [@uschindler](https://github.com/uschindler) I think the inlining breaks because of [@jpountz](https://github.com/jpountz) recent change to introduce "type pollution". > > Specifically this line: > > // Use ByteBuffersDirectory instead of MMapDirectory to have multiple IndexInput sub-classes used by queries > try (Directory dir = new ByteBuffersDirectory()) { > https://github.com/mikemccand/luceneutil/blob/ea81b5d7f6e36bb04ffe7212f7e7129f9f6edc16/src/main/perf/TypePolluter.java#L64-L65 That's fine, but is an issue. The problem here is more different *other* IndexInputs, because: > Now during indexing, as soon as 1GB file is encountered, MMapDirectory introduces a third implementation which makes everything go megamorphic. > > I'd suggest: > > * Remove the two IndexInput implementations in MMapDirectory: there's no longer a 32-bit ByteBuffer limit at play. That's not an issue anymore. MMapDirectory only uses the alternate implementation if the file is larger than 16 Gigabytes (due to fragmentation issues). This won't ever happen with default merging policy unless you force merge to mone segment. So basically you will only see one implementation. You can easily see this in the types seen in the profile/heap dump. > * Fix ByteBuffersDirectory (or rename it?) to no longer use ByteBuffers either. This should remove another implementation: it can just be mmap over an anonymous mapping. If we do this, then actually Adrien would need to replace the pollution code by some other IndexInput impl (like NIOFSDir). Actually he used ByteBuffersDirectory to actually pollute, so by aligning the polluted impl by another one the pollution goes away and we no longer see what we want to see. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
