[
https://issues.apache.org/jira/browse/LUCENE-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Weiss updated LUCENE-10255:
---------------------------------
Description:
I've experimented a bit trying to move the code to the JMS. It is _surprisingly
difficult_... A PoC that almost passes all checks is here:
https://github.com/dweiss/lucene/tree/jms
Here are my conclusions so far:
* The JMS and gradle add a lot of complexity (this applies to any higher-level
tooling, including IDEs, I think). For starters, -modules have to be JARs. The
effect of this is that what was previously a set of directories from
dependencies now has to be a JAR. What was previously an incremental update of
a single .class file now ripples throughout the build recreating module JARs
(ZIPs!)... I didn't realize it at first, but it's a costly thing to do. I'm not
even sure how IDEs handle this issue.- (not true)
* A Java module contains metadata (such as the module version or main class)
that is completely detached from any source file. These things live in a class
bytecode of the compiled module-info; interestingly, there is no source-level
way to specify it - these class attributes are injected by the 'jar' tool.
Gradle has some fancy on-the-fly asm conversion filter that injects it.
* Dependencies between modules will effectively live in two places: in gradle
build files and in module-info files. -And they can go out of sync, although
it's probably easy to catch (since javac would complain about missing classes
during compilation, even if they're in module path).- (with separate module and
classpath configurations there is a possibility to verify the consistency).
* Probably the biggest challenge (not covered in the PoC) are with our custom
javadoc and ecj linter tasks - they see the module-info.java and can't cope
with it. At the same time, there is no easy way to exclude that one particular
file: ecj would have to accept a full set of sources (command argument limit
will be a problem), javac can accept a full set of java sources (external file)
but then it doesn't copy doc-files properly anymore (this is probably easier to
fix).
* There are differences at runtime that are hard to anticipate - for example
resource lookups via class loader no longer work (I fixed this in Luke).
* We will have to rethink the long-term strategy of how white-box tests work.
There are some guidelines here but all of them have some cons (IDEs being
confused).
https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
* it's pretty much impossible to exclude transitive dependencies from modules
we depend on - if they're not compile-time only (static) requirements, they
will have to be present on module path.
* supporting modules may or may not work in your IDE.
was:
I've experimented a bit trying to move the code to the JMS. It is _surprisingly
difficult_... A PoC that almost passes all checks is here:
https://github.com/dweiss/lucene/tree/jms
Here are my conclusions so far:
* The JMS and gradle add a lot of complexity (this applies to any higher-level
tooling, including IDEs, I think). For starters, modules have to be JARs. The
effect of this is that what was previously a set of directories from
dependencies now has to be a JAR. What was previously an incremental update of
a single .class file now ripples throughout the build recreating module JARs
(ZIPs!)... I didn't realize it at first, but it's a costly thing to do. I'm not
even sure how IDEs handle this issue.
* A Java module contains metadata (such as the module version or main class)
that is completely detached from any source file. These things live in a class
bytecode of the compiled module-info; interestingly, there is no source-level
way to specify it - these class attributes are injected by the 'jar' tool.
Gradle has some fancy on-the-fly asm conversion filter that injects it.
* Dependencies between modules will effectively live in two places: in gradle
build files and in module-info files. And they can go out of sync, although
it's probably easy to catch (since javac would complain about missing classes
during compilation, even if they're in module path).
* Probably the biggest challenge (not covered in the PoC) are with our custom
javadoc and ecj linter tasks - they see the module-info.java and can't cope
with it. At the same time, there is no easy way to exclude that one particular
file: ecj would have to accept a full set of sources (command argument limit
will be a problem), javac can accept a full set of java sources (external file)
but then it doesn't copy doc-files properly anymore (this is probably easier to
fix).
* There are differences at runtime that are hard to anticipate - for example
resource lookups via class loader no longer work (I fixed this in Luke).
* We will have to rethink the long-term strategy of how white-box tests work.
There are some guidelines here but all of them have some cons (IDEs being
confused).
https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
* it's pretty much impossible to exclude transitive dependencies from modules
we depend on - if they're not compile-time only (static) requirements, they
will have to be present on module path.
After poking a bit and trying it out I have to say I have mixed feelings about
moving to the JMS. On the one hand, many things are great - the module path,
module descriptors and access modes. On the other hand, the tooling tricks
required to make it all work make you shiver.
If anybody wants to play/ improve things on that experimental branch (I
converted Luke to a full module - it works), please be my guest. I have to sit
on this and think whether it's something I really like or not.
> Fully embrace the java module system
> ------------------------------------
>
> Key: LUCENE-10255
> URL: https://issues.apache.org/jira/browse/LUCENE-10255
> Project: Lucene - Core
> Issue Type: New Feature
> Reporter: Dawid Weiss
> Priority: Major
> Attachments: screenshot-1.png, screenshot-2.png
>
> Time Spent: 21h 50m
> Remaining Estimate: 0h
>
> I've experimented a bit trying to move the code to the JMS. It is
> _surprisingly difficult_... A PoC that almost passes all checks is here:
> https://github.com/dweiss/lucene/tree/jms
> Here are my conclusions so far:
> * The JMS and gradle add a lot of complexity (this applies to any
> higher-level tooling, including IDEs, I think). For starters, -modules have
> to be JARs. The effect of this is that what was previously a set of
> directories from dependencies now has to be a JAR. What was previously an
> incremental update of a single .class file now ripples throughout the build
> recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a
> costly thing to do. I'm not even sure how IDEs handle this issue.- (not true)
> * A Java module contains metadata (such as the module version or main class)
> that is completely detached from any source file. These things live in a
> class bytecode of the compiled module-info; interestingly, there is no
> source-level way to specify it - these class attributes are injected by the
> 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that
> injects it.
> * Dependencies between modules will effectively live in two places: in gradle
> build files and in module-info files. -And they can go out of sync, although
> it's probably easy to catch (since javac would complain about missing classes
> during compilation, even if they're in module path).- (with separate module
> and classpath configurations there is a possibility to verify the
> consistency).
> * Probably the biggest challenge (not covered in the PoC) are with our custom
> javadoc and ecj linter tasks - they see the module-info.java and can't cope
> with it. At the same time, there is no easy way to exclude that one
> particular file: ecj would have to accept a full set of sources (command
> argument limit will be a problem), javac can accept a full set of java
> sources (external file) but then it doesn't copy doc-files properly anymore
> (this is probably easier to fix).
> * There are differences at runtime that are hard to anticipate - for example
> resource lookups via class loader no longer work (I fixed this in Luke).
> * We will have to rethink the long-term strategy of how white-box tests work.
> There are some guidelines here but all of them have some cons (IDEs being
> confused).
> https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
> * it's pretty much impossible to exclude transitive dependencies from modules
> we depend on - if they're not compile-time only (static) requirements, they
> will have to be present on module path.
> * supporting modules may or may not work in your IDE.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
