[
https://issues.apache.org/jira/browse/LUCENE-10158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler updated LUCENE-10158:
-----------------------------------
Description:
While creating the new MMapDirectory using Project Panama in the recent OpenJDK
versions (not yet released, incubation only), I stumbled on our testing
framework, which wraps many objects with AssertingXY. The problem with that is,
mmap in project panama only works when the java.nio.files.Path is owned by the
default file system provider. During testing we wrap it often with custom
implementations emulating Windows or track open file handles.
If you pass such a wrapped Path to the NIO2 Panama APIs, it will fail with
exception, because it can't refer to file channel internal methods from it. In
the final version of Panama this may go away and we can provide our own wrapper
for memory mapping, but this is problematic with current testing.
My plan is to release versions of MMapDirectory version 2 with different
implementations of the Panama APIs for easy pluggin into Lucene, Solr,
Elasticsearch by just adding a JAR file that fits your JDK version. To run
tests, unfortunately the MMapDir impl must "unrwap" the Path wrappers added by
the test system.
To help with that and to make it more general, in my pull requests (e.g.
https://github.com/apache/lucene/pull/177), I added a new interface
{{org.apache.lucene.util.Unwrappable}} that allows to unwrap external code to
get the "original" Path implementation. The same interface could be applied to
many other Lucene/Test classes that needs unwrapping sometimes (e.g. around
Directory or Queries), but for now it is only implemented for Test's
FilterPath. The interface needs to be part of Lucene core and is used by
production code to unwrap any test-framework FilterPath (or similar) wrappers.
MMapDirectory version 2 uses it to get the original Path to be passed to
MemorySegment.mapFile().
I'd like to get this into Lucene 9.0. It does not hurt, it is just an
interface, which is implemented by test classes, ready for extension to other
classes. It also provides the unwrapper method, which is generic.
was:
While creating the new MMapDirectory using Project Panama in the recent OpenJDK
versions (not yet released, incubation only), I stumbled on our testing
framework, which wraps many objects with AssertingXY. The problem with that is,
mmap in project panama only works when the java.nio.files.Path is owned by the
default file system provider. During testing we wrap it often with custom
implementations emulating Windows or track open file handles.
If you pass such a wrapped Path to the NIO2 Panama APIs, it will fail with
exception, because it can't refer to file channel internal methods from it. In
the final version of Panama this may go away and we can provide our own wrapper
for memory mapping, but this is problematic with current testing.
My plan is to release versions of MMapDirectory version 2 with different
implementations of the Panama APIs for easy pluggin into Lucene, Solr,
Elasticsearch by just adding a JAR file that fits your JDK version. To run
tests, unfortunately the MMapDir impl must "unrwap" the Path wrappers added by
the test system.
To help with that and to make it more general, in my pull requests (e.g.
https://github.com/apache/lucene/pull/177), I added a new interface
{{org.apache.lucene.util.Unwrappable}} that allows to unwrap external code to
get the "original" Path implementation. The same interface could be applied to
many other Lucene/Test classes that needs unwrapping sometimes (e.g. around
Directory or Queries), but for now it is only implemented for Test's
FilterPath. The interface is part of Lucene core and just used by the test to
supply unwrapping.
MMapDirectory version 2 uses it to get the original Path to be passed to
MemorySegment.mapFile().
I'd like to get this into Lucene 9.0. It does not hurt, it is just an
interface, which is implemented by test classes, ready for extension to other
classes. It also provides the unwrapper method, which is generic.
> Add a new interface Unwrappable to the utils package to ease migration to new
> MMAPDirectory and its testing
> -----------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-10158
> URL: https://issues.apache.org/jira/browse/LUCENE-10158
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/other, general/test
> Affects Versions: main (9.0)
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Priority: Major
> Fix For: main (9.0)
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While creating the new MMapDirectory using Project Panama in the recent
> OpenJDK versions (not yet released, incubation only), I stumbled on our
> testing framework, which wraps many objects with AssertingXY. The problem
> with that is, mmap in project panama only works when the java.nio.files.Path
> is owned by the default file system provider. During testing we wrap it often
> with custom implementations emulating Windows or track open file handles.
> If you pass such a wrapped Path to the NIO2 Panama APIs, it will fail with
> exception, because it can't refer to file channel internal methods from it.
> In the final version of Panama this may go away and we can provide our own
> wrapper for memory mapping, but this is problematic with current testing.
> My plan is to release versions of MMapDirectory version 2 with different
> implementations of the Panama APIs for easy pluggin into Lucene, Solr,
> Elasticsearch by just adding a JAR file that fits your JDK version. To run
> tests, unfortunately the MMapDir impl must "unrwap" the Path wrappers added
> by the test system.
> To help with that and to make it more general, in my pull requests (e.g.
> https://github.com/apache/lucene/pull/177), I added a new interface
> {{org.apache.lucene.util.Unwrappable}} that allows to unwrap external code to
> get the "original" Path implementation. The same interface could be applied
> to many other Lucene/Test classes that needs unwrapping sometimes (e.g.
> around Directory or Queries), but for now it is only implemented for Test's
> FilterPath. The interface needs to be part of Lucene core and is used by
> production code to unwrap any test-framework FilterPath (or similar)
> wrappers. MMapDirectory version 2 uses it to get the original Path to be
> passed to MemorySegment.mapFile().
> I'd like to get this into Lucene 9.0. It does not hurt, it is just an
> interface, which is implemented by test classes, ready for extension to other
> classes. It also provides the unwrapper method, which is generic.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
