Re: [Log4j] Plans for modularization

2017-05-04 Thread Matt Sicker
I was thinking a little bit about the idea behind the SPI module. One aspect I was hoping for, though it may be unnecessary, was having a nice clean way to separate the implementation of the implementation so to say. The way I see it, there are a few major components of a logging system: * Logging

Re: [Log4j] Plans for modularization

2017-05-02 Thread Mikael Ståldal
So let's have a look at Git branch LOG4J2-1889. On Tue, May 2, 2017 at 5:45 PM, Matt Sicker wrote: > Google has a serious case of NIH syndrome which is why I tend to avoid > their Java libraries. ;) > > Splitting log4j-kafka sounds like a good first step. Hopefully we can > figure out a manageab

Re: [Log4j] Plans for modularization

2017-05-02 Thread Matt Sicker
Google has a serious case of NIH syndrome which is why I tend to avoid their Java libraries. ;) Splitting log4j-kafka sounds like a good first step. Hopefully we can figure out a manageable way to handle the docs over time. On 2 May 2017 at 03:19, Mikael Ståldal wrote: > Sounds good. > > I thin

Re: [Log4j] Plans for modularization

2017-05-02 Thread Mikael Ståldal
Sounds good. I think a first step would be for us to cooperate on completing the separation of log4j-kafka that I started, and then use that as a template for others. On Sun, Apr 30, 2017 at 10:23 PM, Matt Sicker wrote: > Just to make sure this sounds reasonable, here's my idea for a roadmap: >

Re: [Log4j] Plans for modularization

2017-05-02 Thread Mikael Ståldal
Google does not use Git, I think they use some internally developed VCS. On Sun, Apr 30, 2017 at 6:56 PM, Matt Sicker wrote: > I doubt Google uses Git then. > > One of the main goals of separating repositories is to make release > management easier so we can RERO more! Though part of the problem

Re: [Log4j] Plans for modularization

2017-04-30 Thread Gary Gregory
+1 to the split plan. A l'attaque! Gary On Apr 30, 2017 1:23 PM, "Matt Sicker" wrote: > Just to make sure this sounds reasonable, here's my idea for a roadmap: > > 2.9: separate out more modules (not necessarily into their own repo or > repos yet) so that log4j-core contains minimal dependencie

Re: [Log4j] Plans for modularization

2017-04-30 Thread Matt Sicker
That's why I'd like to separate the two goals. It's possible that the difference between log4j-core and a divided core+spi would be minimal in which case I'd change gears with the goal of stabilising a public plugin and configuration SPI. On 30 April 2017 at 15:41, Ralph Goers wrote: > I am fine

Re: [Log4j] Plans for modularization

2017-04-30 Thread Ralph Goers
I am fine with your goals for 2.9. I still have my doubts that separating out an spi can be done. Ralph > On Apr 30, 2017, at 1:23 PM, Matt Sicker wrote: > > Just to make sure this sounds reasonable, here's my idea for a roadmap: > > 2.9: separate out more modules (not necessarily into their

Re: [Log4j] Plans for modularization

2017-04-30 Thread Matt Sicker
Just to make sure this sounds reasonable, here's my idea for a roadmap: 2.9: separate out more modules (not necessarily into their own repo or repos yet) so that log4j-core contains minimal dependencies. 2.10: log4j-core-spi or whatever the name, making it easier for other modules to release on th

Re: [Log4j] Plans for modularization

2017-04-30 Thread Matt Sicker
I doubt Google uses Git then. One of the main goals of separating repositories is to make release management easier so we can RERO more! Though part of the problem there is that for some reason, the release process requires running all the tests and whatnot at least 3 times or more. On 30 April 2

Re: [Log4j] Plans for modularization

2017-04-30 Thread Gary Gregory
Maybe more than one repo isn't such a good idea? I hear Google uses a single repo for all their code... Gary On Apr 30, 2017 9:41 AM, "Matt Sicker" wrote: > I've noticed just with the Scala repo that integrating various repositories > into a single coherent website is not so easy anymore (and i

Re: [Log4j] Plans for modularization

2017-04-30 Thread Matt Sicker
I've noticed just with the Scala repo that integrating various repositories into a single coherent website is not so easy anymore (and it wasn't really all that easy in the first place). While it may be possible to manage each repository's website individually and use symlinks in the svn repo to ke

Re: [Log4j] Plans for modularization

2017-04-25 Thread Matt Sicker
Ideally, the two will align, just like the OSGi modules (which tend to directly correspond with maven modules since that's how they're normally assembled). On 25 April 2017 at 13:39, Gary Gregory wrote: > We are going to have to prefix "module" with "Java" or "Maven" in > discussions and documen

Re: [Log4j] Plans for modularization

2017-04-25 Thread Gary Gregory
We are going to have to prefix "module" with "Java" or "Maven" in discussions and documentation to avoid confusion from now on... Gary On Apr 25, 2017 10:59 AM, "Matt Sicker" wrote: > If you browse around the Java 9 javadocs, you'll see that they split things > up by module there as well. With

Re: [Log4j] Plans for modularization

2017-04-25 Thread Matt Sicker
If you browse around the Java 9 javadocs, you'll see that they split things up by module there as well. With that in mind, hopefully it's not too complicated to support. What I really want to see is inter-module links (both Java modules and Maven modules that is) work properly. On 25 April 2017 at

Re: [Log4j] Plans for modularization

2017-04-25 Thread Ralph Goers
Ouch. This is where it gets messy. Currently, the javadoc is built independently for each module. I’m not sure how to aggregate them all together but I’m sure Java 9 must be doing that with all the modules they are supporting. Ralph > On Apr 25, 2017, at 7:03 AM, Mikael Ståldal wrote: > > Wh

Re: [Log4j] Plans for modularization

2017-04-25 Thread Mikael Ståldal
I have started to extract KafkaAppender to its own module, please review Git branch LOG4J2-1889. This can be a template on how to do it for other plugins. On Tue, Apr 25, 2017 at 5:14 PM, Gary Gregory wrote: > To be honest here, it feels to me like we are splitting hair. Maybe we > could proceed

Re: [Log4j] Plans for modularization

2017-04-25 Thread Gary Gregory
To be honest here, it feels to me like we are splitting hair. Maybe we could proceed with m12n and then do an SPI. It also feels odd to call it SPI if it has a boatload of "stuff" in it like our config code. Maybe we leave -core as in and pull out JRE-based appenders and layouts into a -basic mod

Re: [Log4j] Plans for modularization

2017-04-25 Thread Matt Sicker
I'm also thinking that since classes and packages may have to be moved around a little bit, making plugin refactoring for custom plugins as simple as possible would be great. A search/replace for package imports should hopefully be enough. On 25 April 2017 at 10:03, Matt Sicker wrote: > No, it w

Re: [Log4j] Plans for modularization

2017-04-25 Thread Matt Sicker
No, it would define the interfaces (and abstract classes so that we don't need to introduce Appender2, Appender3, etc., without having default methods available) along with the core configuration and plugin system. Essentially all the parts of log4j-core that we want to create a well-defined servic

Re: [Log4j] Plans for modularization

2017-04-25 Thread Gary Gregory
Would SPI only contain interfaces? Gary On Apr 25, 2017 7:36 AM, "Mikael Ståldal" wrote: > The goal of log4j-spi is that plugins can depend on log4j-spi (and > log4j-api), not on log4j-core. > > On Tue, Apr 25, 2017 at 4:33 PM, Gary Gregory > wrote: > > > If my app depends on the BOM module, d

Re: [Log4j] Plans for modularization

2017-04-25 Thread Matt Sicker
Gary, I don't think you'd want everything from the BOM since it has both log4j-slf4j-impl and log4j-to-slf4j. I suppose a more elaborate explanation of log4j-spi is that it would also need to include the core configuration classes and plugin processing and whatnot. It wouldn't itself provide any p

Re: [Log4j] Plans for modularization

2017-04-25 Thread Mikael Ståldal
The goal of log4j-spi is that plugins can depend on log4j-spi (and log4j-api), not on log4j-core. On Tue, Apr 25, 2017 at 4:33 PM, Gary Gregory wrote: > If my app depends on the BOM module, do I get everything so I can > assembly:single them into a zip with my app? > > Gary > > On Apr 25, 2017 7

Re: [Log4j] Plans for modularization

2017-04-25 Thread Gary Gregory
If my app depends on the BOM module, do I get everything so I can assembly:single them into a zip with my app? Gary On Apr 25, 2017 7:09 AM, "Matt Sicker" wrote: > I also agree that modules should really stick to required dependencies. > > As for only requiring log4j-core, the idea here is that

Re: [Log4j] Plans for modularization

2017-04-25 Thread Remko Popma
About log4j-core automatically pulling in api and spi dependencies: bear in mind that not everyone uses Gradle or Maven... I suspect many enterprise projects don't. Does spi need to be a module? I agree on the need to have a clear Service Provider Interface which defines interfaces, abstract class

Re: [Log4j] Plans for modularization

2017-04-25 Thread Matt Sicker
I also agree that modules should really stick to required dependencies. As for only requiring log4j-core, the idea here is that you could still just do: compile 'org.apache.logging.log4j:log4j-core:2.+' or whatever the equivalent is for your build system, and you'll still get log4j-api and anyth

Re: [Log4j] Plans for modularization

2017-04-25 Thread Mikael Ståldal
I think we should aim for all modules having only required dependencies. We should not add any new optional dependencies, and all new modules should only have required dependencies. Maybe we won't be able to get rid of all optional dependencies in log4j-core right away, though. Maybe we will rele

Re: [Log4j] Plans for modularization

2017-04-25 Thread Matt Sicker
log4j-spi is to make a well-defined API for plugins including a semantically versioned Appender, Filter, Layout, etc., interfaces and abstract classes, along with the annotations and whatnot. It'd be treated similarly to log4j-api in that regard. On 25 April 2017 at 09:03, Gary Gregory wrote: >

Re: [Log4j] Plans for modularization

2017-04-25 Thread Gary Gregory
We should have consensus on the big picture here... are we all Ok with the idea of all modules only having _required_ dependencies? Gary On Apr 25, 2017 6:57 AM, "Remko Popma" wrote: > Nice analysis Mikael! > > I'm a bit fuzzy on log4j-spi, what was that for again? The list says "core > will de

Re: [Log4j] Plans for modularization

2017-04-25 Thread Mikael Ståldal
When adding new modules to the main repo, does each module need its own site directory? On Tue, Apr 25, 2017 at 4:02 PM, Mikael Ståldal wrote: > Yes, they should stay in the main repo, at least for the time being. > > On Tue, Apr 25, 2017 at 3:56 PM, Gary Gregory > wrote: > >> And all of that s

Re: [Log4j] Plans for modularization

2017-04-25 Thread Mikael Ståldal
Yes, they should stay in the main repo, at least for the time being. On Tue, Apr 25, 2017 at 3:56 PM, Gary Gregory wrote: > And all of that should stay in the same repo IMO. > > Gary > > On Apr 25, 2017 2:51 AM, "Mikael Ståldal" > wrote: > > > I guess that log4-core will become: > > > >- lo

Re: [Log4j] Plans for modularization

2017-04-25 Thread Remko Popma
Nice analysis Mikael! I'm a bit fuzzy on log4j-spi, what was that for again? The list says "core will depend on spi" but I think it's worth making an effort to ensure that basic (file) logging behavior only needs core+API... Why does spi need to be separated from core? My first thought about

Re: [Log4j] Plans for modularization

2017-04-25 Thread Gary Gregory
And all of that should stay in the same repo IMO. Gary On Apr 25, 2017 2:51 AM, "Mikael Ståldal" wrote: > I guess that log4-core will become: > >- log4j-core (will depend on log4j-spi) >- log4j-spi >- log4j-csv >- log4j-xml (XmlLayout) >- log4j-json (JsonLayout) >- log4j

Re: [Log4j] Plans for modularization

2017-04-25 Thread Mikael Ståldal
I guess that log4-core will become: - log4j-core (will depend on log4j-spi) - log4j-spi - log4j-csv - log4j-xml (XmlLayout) - log4j-json (JsonLayout) - log4j-yaml (YamlLayout) - log4j-kafka - log4j-smtp - log4j-jms - log4j-jdbc (or can this be kept in log4j-core?)

Re: [Log4j] Plans for modularization

2017-04-25 Thread Mikael Ståldal
I don't think it will be more difficult to figure out which of the modules to use, than to figure out which optional 3rd party dependencies they need to include today. On Mon, Apr 24, 2017 at 7:43 PM, Remko Popma wrote: > How many new modules are we talking about, concretely? > > Matt mentioned

Re: [Log4j] Plans for modularization

2017-04-24 Thread Matt Sicker
It'd probably be confusing to make a log4j-core-all that only works with Java 7 and 8, too, so it'd have to be doable somehow. I wonder how shaded/shadowed applications are supposed to work in Java 9? On 24 April 2017 at 14:12, Ralph Goers wrote: > Yeah, I have no idea how you would build the mo

Re: [Log4j] Plans for modularization

2017-04-24 Thread Ralph Goers
Yeah, I have no idea how you would build the module-info.class file. module-info.java is compiled by the compiler and it performs checks to make sure the things being exported are really there. So you would need to have a maven module that unpacks all the classes and then compiles the module-in

Re: [Log4j] Plans for modularization

2017-04-24 Thread Matt Sicker
It doesn't sound like there's a proper way to export a Java 9 module from two separate JARs, though I'm not clear on the details. If we package up the modules into a sort of log4j-core-all artifact, that'd be an assembly detail for distribution. On 24 April 2017 at 13:19, Remko Popma wrote: > I

Re: [Log4j] Plans for modularization

2017-04-24 Thread Remko Popma
I must be missing something, I don't see the problem. Users wouldn't have both the combined jar and the single-module jars on the classpath together, so there should not be an issue... For example, assume our current log4j-core jar is split up into * log4j-core-slim * log4j-core-appender-db *

Re: [Log4j] Plans for modularization

2017-04-24 Thread Ralph Goers
In theory I agree with this but the devil is in the details. For example, we want to be able to say that Log4j requires NO third party dependencies but that we support and take advantage of features provided by third-party software. This means that even though the third-party jars will now beco

Re: [Log4j] Plans for modularization

2017-04-24 Thread Matt Sicker
The general idea I had was that if you include log4j-core in your dependencies, you'll get log4j-api, log4j-spi, and most of the existing functionality you already get access to in log4j-core currently. We'd split out the bits that have extra dependencies for the most part which generally revolves

Re: [Log4j] Plans for modularization

2017-04-24 Thread Ralph Goers
Guess what? If I am understanding Stephen correctly uber jars are not going to work as you can’t have multiple modules that export the same package. Ralph > On Apr 24, 2017, at 10:43 AM, Remko Popma wrote: > > How many new modules are we talking about, concretely? > > Matt mentioned the Stack

Re: [Log4j] Plans for modularization

2017-04-24 Thread Remko Popma
How many new modules are we talking about, concretely? Matt mentioned the StackOverflow questions about transitive dependencies etc, but I imagine splitting log4j-core into 5 or more new modules will also cause confusion... It won't be trivial for users to figure out which of the many modules they

Re: [Log4j] Plans for modularization

2017-04-24 Thread Matt Sicker
I agree with Ralph here. I'm sure we'll figure out rather quickly which modules are easy to put into rarely updated repositories. On 24 April 2017 at 11:39, Ralph Goers wrote: > I would prefer a hybrid approach. First things should be moved to > separate modules. Then, if they don’t seem to be

Re: [Log4j] Plans for modularization

2017-04-24 Thread Ralph Goers
I would prefer a hybrid approach. First things should be moved to separate modules. Then, if they don’t seem to be modified frequently they can be moved to a separate repo. For example, I think it would be OK for the Flume Appender to be in a separate repo. It hasn’t changed in quite a while an

Re: [Log4j] Plans for modularization

2017-04-24 Thread Gary Gregory
On Apr 24, 2017 2:38 AM, "Mikael Ståldal" wrote: I fully agree with Matt's both proposals. I'm skeptic to creating more repositories (than we already have) though. I think that we should start by splitting out modules from log4j-core and keep those modules in the main repository with synchronize

Re: [Log4j] Plans for modularization

2017-04-24 Thread Mikael Ståldal
I guess that log4j-core would include other Log4j implementation stuff also (that plugins does not need to depend on). On Mon, Apr 24, 2017 at 4:48 PM, Matt Sicker wrote: > That would be the goal. Then we can have a log4j-core which has all the > main plugins (file-related mostly) and other log4

Re: [Log4j] Plans for modularization

2017-04-24 Thread Matt Sicker
That would be the goal. Then we can have a log4j-core which has all the main plugins (file-related mostly) and other log4j addons for the 3rd party dependency ones. On 24 April 2017 at 09:47, Mikael Ståldal wrote: > I guess this means that a plugin only needs to depend on log4j-spi (and > probab

Re: [Log4j] Plans for modularization

2017-04-24 Thread Mikael Ståldal
I guess this means that a plugin only needs to depend on log4j-spi (and probably on log4j-api), not on log4j-core or anything else log4j-*. That would be good. On Mon, Apr 24, 2017 at 4:44 PM, Matt Sicker wrote: > Keeping configuration/plugin processing code inside log4j-spi should > probably b

Re: [Log4j] Plans for modularization

2017-04-24 Thread Matt Sicker
Keeping configuration/plugin processing code inside log4j-spi should probably be marked clearly which classes are public APIs and which aren't then. I'm not sure how many classes would need to be moved around, but that will require some experimentation to figure out anyways. On 24 April 2017 at 04

Re: [Log4j] Plans for modularization

2017-04-24 Thread Mikael Ståldal
I fully agree with Matt's both proposals. I'm skeptic to creating more repositories (than we already have) though. I think that we should start by splitting out modules from log4j-core and keep those modules in the main repository with synchronized versioning and releases, at least for the 2.9 rel

Re: [Log4j] Plans for modularization

2017-04-24 Thread Apache
We have to do something. I am about 3/4 of the way creating java9 modules and I am not happy with core at all. Far too many classes have to be exported as you can only name packages, not classes. I think all the configuration stuff will need to stay in core, which means the optional dependency

[Log4j] Plans for modularization

2017-04-24 Thread Matt Sicker
I think I brought this topic up like 3 years ago when I was working on initial OSGi support, but now that we have 3 more years worth of code additions and optional features, I think this might be a more appropriate time to discuss it again in light of experience. Building log4j-core itself already