We can certainly set up CI to build on upstream snapshot releases or even nightly jobs.
For improving the build time of core, that will require parallelizing tests like in api. On Fri, Jun 11, 2021 at 13:48 Volkan Yazıcı <volkan.yaz...@gmail.com> wrote: > I agree with all the points (both pros and cons) you have shared. > > A small "core" will speed up the development and release cycle > significantly. Most of the time I skip a full Maven "verify", which is > taking more than half an hour on my machine. > > Regarding CI, I believe there will be a huge win too due to faster feedback > cycles. Also note that it is possible to trigger GitHub Actions workflows > from another repository change. Hence the consistency check across > repositories you have described can easily be set up. > > On Fri, Jun 11, 2021 at 7:45 PM Ralph Goers <ralph.go...@dslextreme.com> > wrote: > > > Not sure what you want a recap on. I see a few issues on both sides of > the > > coin: > > > > For splitting: > > 1. Many of the current modules rarely get modified. Creating a new > version > > of > > them with each release just so the release number matches seems silly. > > 2. Yes, creating a release is time consuming. When I am nearing the time > to > > perform a release I: > > a. Review PRs and Jira issues looking for ones that are either > critical, > > serious > > or easy to fix and try to address them. This typically involves > > running the > > build several times. > > b. Perform all the required release updates. > > c. Build the web site locally and inspect it. > > d. Follow the guide to actually perform the release. > > The prep time in step a is typically what takes the most time due to how > > many > > times I run the full build. This is where the most time savings would be > > since > > most of the changes occur in log4j-api and log4j-core, although the most > > time > > consuming part of the build is the log4j-core unit tests. > > 3. This might encourage everyone to run a full build for every commit. > > > > Against splitting: > > 1. Confusion over where a particular component can be found. > > 2. Although unlikely, it is possible a change in log4j-api or log4j-core > > could > > break something in one of the other modules and we wouldn’t find out > > immediately as we do today. This could be solved by setting up good > > Jenkins > > pipelines that cause all the sub projects to rebuild whenever the main > > module is built. > > > > Ralph > > > > > On Jun 11, 2021, at 10:12 AM, Gary Gregory <garydgreg...@gmail.com> > > wrote: > > > > > > I am on 100% board with spliting up modules/artifacts to reduce the > "suck > > > the world in" issue. Splitting into separate repos/projects I have zero > > > interest in, I only see drawbacks. Version confusion, BOM POM or not, > > weird > > > dependeny issues, more fiddling with IDEs to make sure you can get > stuff > > > done, bleh. > > > > > > I know Ralph has been quite generous with him time in release > management > > > and has mentioned numerous time that he feels the current repository > size > > > leads to long release candiate build times. But I see this as a drive > to > > > break up the repository wrong headed. When I run any of the various > Maven > > > builds I use, some builds rebuilds various pieces needlessly, compiling > > > code when no recompilation is needed for example, running tests more > than > > > needed, and other Maven oddities. If the tooling is a problem, then > lets > > > look at that first. > > > > > > Maybe we could get a recap from Ralph? > > > > > > Gary > > > > > > > > > On Fri, Jun 11, 2021, 10:42 Volkan Yazıcı <volkan.yaz...@gmail.com> > > wrote: > > > > > >> Gary, I thought you were on board with breaking up into separate > > projects. > > >> What do you propose instead? > > >> > > >> On Fri, Jun 11, 2021 at 12:22 PM Gary Gregory <garydgreg...@gmail.com > > > > >> wrote: > > >> > > >>> From my POV, this will create an incomprehensible mess of versions > for > > >>> users. Big bummer. > > >>> > > >>> Gary > > >>> > > >>> On Fri, Jun 11, 2021, 04:29 Volkan Yazıcı <volkan.yaz...@gmail.com> > > >> wrote: > > >>> > > >>>> To start the discussion for breaking up the modules into separate > > >>> projects, > > >>>> I propose the following structure: > > >>>> > > >>>> *logging-log4j* > > >>>> log4j-1.2-api > > >>>> log4j-api > > >>>> log4j-bom > > >>>> log4j-core > > >>>> log4j-core-its > > >>>> log4j-gctests > > >>>> log4j-iostreams > > >>>> log4j-layout-template-json > > >>>> log4j-perf > > >>>> log4j-plugins > > >>>> > > >>>> *logging-log4j-appender* > > >>>> log4j-cassandra > > >>>> log4j-couchdb > > >>>> log4j-csv > > >>>> log4j-flume-ng > > >>>> log4j-mongodb3 > > >>>> log4j-mongodb4 > > >>>> log4j-smtp > > >>>> > > >>>> *logging-log4j-appender-jdbc* > > >>>> log4j-jdbc (?) > > >>>> log4j-jdbc-dbcp2 (?) > > >>>> log4j-jpa > > >>>> > > >>>> *logging-log4j-appender-queue* > > >>>> log4j-jeromq > > >>>> log4j-jms > > >>>> log4j-kafka > > >>>> log4j-redis > > >>>> > > >>>> *logging-log4j-binding* > > >>>> log4j-jcl > > >>>> log4j-jpl > > >>>> log4j-jul > > >>>> log4j-liquibase > > >>>> log4j-slf4j18-impl > > >>>> log4j-slf4j-impl > > >>>> log4j-to-slf4j > > >>>> > > >>>> *logging-log4j-container* > > >>>> log4j-docker > > >>>> log4j-kubernetes > > >>>> > > >>>> *logging-log4j-gui* > > >>>> log4j-jmx-gui > > >>>> > > >>>> *logging-log4j-jee* > > >>>> log4j-appserver > > >>>> log4j-taglib > > >>>> log4j-web > > >>>> > > >>>> *logging-log4j-layout-jackson* > > >>>> log4j-layout-jackson > > >>>> log4j-layout-jackson-json > > >>>> log4j-layout-jackson-xml > > >>>> log4j-layout-jackson-yaml > > >>>> > > >>>> *logging-log4j-osgi* > > >>>> log4j-osgi > > >>>> > > >>>> *logging-log4j-spring* > > >>>> log4j-spring-boot > > >>>> log4j-spring-cloud-config > > >>>> > > >>>> Note that above breakdown contains every available module in master, > > >>> except > > >>>> log4j-samples, which has the following sub-modules: > > >>>> > > >>>> log4j-samples-loggerProperties (contains 2 very trivial example > > >> classes, > > >>>> log4j-core [tests] contains more comprehensive alternatives) > > >>>> log4j-samples-flume-remote (flume-specific) > > >>>> log4j-samples-flume-embedded (flume-specific) > > >>>> log4j-samples-flume-common (flume-specific) > > >>>> log4j-samples-configuration (contains 3 very trivial example > classes, > > >>>> log4j-core [tests] contains more comprehensive alternatives) > > >>>> > > >>>> I propose removing log4j-samples module and, if necessary, moving > > >>>> Flume-related parts to the log4j-flume-ng. > > >>>> > > >>>> I have kept log4j-layout-template-json in the logging-log4j project, > > >>> since > > >>>> it has no external dependencies. This said, I am okay with moving it > > >> to a > > >>>> separate logging-log4j-layout project. > > >>>> > > >>>> Please share your remarks. Eventually, I want to translate this > into a > > >>> JIRA > > >>>> ticket. > > >>>> > > >>>> Kind regards. > > >>>> > > >>>> On Thu, Jun 10, 2021 at 5:43 PM Ralph Goers < > > >> ralph.go...@dslextreme.com> > > >>>> wrote: > > >>>> > > >>>>> Gary, > > >>>>> > > >>>>> I am -1 to almost every item on your list. > > >>>>> > > >>>>> When I said break up I meant mostly moving most everything outside > of > > >>>>> log4j-core, > > >>>>> log4j-api, and log4j-plugins into separate repos such as > > >>>>> logging-log4j-nosql, > > >>>>> logging-log4j-pubsub, etc. These would not require groupId or > > >>> artifactId > > >>>>> changes > > >>>>> although the versioning would potentially be out of synch with the > > >> main > > >>>>> releases > > >>>>> as each would be on its own release cycle. This would greatly > > >> simplify > > >>>>> releasing > > >>>>> core but it would require careful though on what the versions would > > >> be > > >>>> for > > >>>>> the > > >>>>> “extra” projects. > > >>>>> > > >>>>> As for further breaking up core, that should revolve primarily > around > > >>>>> reducing the > > >>>>> dependencies listed in module-info.java to the bare minimum. > > >>>>> > > >>>>> There will be no log4j3. We cannot change groupIds, artifactIds, or > > >>>>> package names, > > >>>>> other than what has been done to support JPMS. A world in which a > > >>> log4j2 > > >>>>> and > > >>>>> log4j3 try to co-exist would be an unmitigated disaster. Commons > can > > >>>>> happily get > > >>>>> away with that. Log4j cannot. > > >>>>> > > >>>>> If an application had both log4j2 and log4j3 jars present they > would > > >>> end > > >>>>> up with > > >>>>> multiple LoggerContexts, multiple Configurations, and multiple > > >> Appender > > >>>>> Managers > > >>>>> where today there is only a single one. That would mean two > instances > > >>> of > > >>>>> the same > > >>>>> configuration file would be active at once. So when it is time to > > >>>> rollover > > >>>>> it would > > >>>>> be performed twice instead of once as a simple example. For this > > >> reason > > >>>> we > > >>>>> CANNOT break backward compatibility. > > >>>>> > > >>>>> However, we are talking about runtime backward compatibility. The > > >>> Plugin > > >>>>> system > > >>>>> was changed internally in 3.0 so that plugins compiled with 3.0 use > > >>>>> ServiceLoader > > >>>>> instead of the data file. However, it will still find and use 2.x > > >>> plugins > > >>>>> when they are > > >>>>> present and can be located. > > >>>>> > > >>>>> The meaning of this should be clear. It is 3.0 because to take > > >>> advantage > > >>>>> of its > > >>>>> features you may have to make code changes. But it will tolerate > code > > >>>> that > > >>>>> was > > >>>>> compiled for 2.x. > > >>>>> > > >>>>> The reasons why it is 3.0 and not a simple upgrade to 2.x are: > > >>>>> 1. Plugins must be compiled with 3.0 to use the ServiceLoader > > >>> packaging. > > >>>>> 2. It requires Java 11. We still need to support Java 8. > > >>>>> 3. It fully supports JPMS. Release 2.x does not. It is likely that > > >>>>> applications with > > >>>>> multiple Module Layers might not be able to find all the plugins. > > >>> Adding > > >>>>> full JPMS > > >>>>> support to 2.x simply isn’t possible. > > >>>>> 4. It will be introducing new DI injection features not present in > > >> 2.x. > > >>>>> > > >>>>> Major releases do not imply that you are completely breaking > backward > > >>>>> compatibility. > > >>>>> They imply that some kind of compatibility is broken, which we are > > >>> doing > > >>>>> by > > >>>>> requiring coding changes to Plugins to compile with 3.0. This means > > >> we > > >>>>> need to > > >>>>> leave in any classes or methods that existing plugins might have > > >> used. > > >>> It > > >>>>> means > > >>>>> we have to continue using our own Supplier unless it can be > verified > > >>> that > > >>>>> an > > >>>>> application using the Supplier in 2.x can run with 3.0 even if it > is > > >>>>> converted to > > >>>>> java.util.function.Supplier. I have no idea if the code the > compiler > > >>>>> generates > > >>>>> for lambdas actually implements the declared interface or not. > > >>>>> > > >>>>> Log4j 3.0 is a major change. But that doesn’t mean we can screw our > > >>> users > > >>>>> by > > >>>>> breaking everything. > > >>>>> > > >>>>> Ralph > > >>>>> > > >>>>> > > >>>>> > > >>>>>> On Jun 10, 2021, at 6:32 AM, Gary Gregory <garydgreg...@gmail.com > > > > >>>>> wrote: > > >>>>>> > > >>>>>> These are all IMOs of course, YMMV: > > >>>>>> - What's the rush to 3.0? > > >>>>>> > > >>>>>> - I'm all for breaking up the core and other artifacts into more > > >>>>>> artifacts based on their dependencies requirement such that > > >> depending > > >>>>>> on Log4j 3 does not "suck in the world", for example, I should be > > >>> able > > >>>>>> to depend on a currently non-existent "log4j3-console" and only > > >> bring > > >>>>>> in a tiny bit of code (API, a tiny core, and no plugin system). I > > >> did > > >>>>>> a fair amount of breaking up of various artifacts a while back. > > >>>>>> > > >>>>>> - 3.0 is a MAJOR release that gives us the opportunity to drop > > >>>>>> deprecated APIs and code like our custom functional interfaces: > > >>>>>> Supplier and so on. If we do not clean up, then there is no point > > >> in > > >>> a > > >>>>>> major release. Basically, I expect to break binary and source > > >>>>>> compatibility. > > >>>>>> > > >>>>>> - 3.0 must be in a new package namespace and new Maven > > >> coordinates. I > > >>>>>> MUST be able to run Log4j 1, 2, and 3 in the same class loader. I > > >> can > > >>>>>> already run Log4j 1 and 2 side by side which is good. > > >>>>>> > > >>>>>> Gary > > >>>>>> > > >>>>>> On Thu, Jun 10, 2021 at 8:47 AM Volkan Yazıcı < > > >>> volkan.yaz...@gmail.com > > >>>>> > > >>>>> wrote: > > >>>>>>> > > >>>>>>> Ralph, count me in for such a change. I really want to have > > >> separate > > >>>>>>> sub-projects for such modules. This will extremely speed up > > >>>>> build/release > > >>>>>>> times too, which is nowadays of uttermost importance to my peace > > >> of > > >>>> mind > > >>>>>>> while developing. > > >>>>>>> > > >>>>>>> This said, I am reluctant about such a major change when we are > > >> this > > >>>>> close > > >>>>>>> to the 3.0.0 release. I guess this would definitely postpone the > > >>> 3.0.0 > > >>>>>>> release to 2022. This will probably break the backward > > >> compatibility > > >>>> at > > >>>>>>> least for the artifact groupId, am I wrong? Not to mention that > > >> the > > >>>>> entire > > >>>>>>> website needs to be adapted to this multi-project setup too. Is > > >>> there > > >>>>>>> anything else? > > >>>>>>> > > >>>>>>> On Thu, Jun 3, 2021 at 5:31 PM Ralph Goers < > > >>>> ralph.go...@dslextreme.com> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> Yeah - I have proposed moving all these extra integrations to a > > >>>>> separate > > >>>>>>>> repo but > > >>>>>>>> I’ve never gotten consensus. I’d prefer to have a project like > > >>>>>>>> log4j-pubsub where > > >>>>>>>> things like JMS, JeroMQ, etc can go live, log4j-nosql for all > the > > >>>> nosql > > >>>>>>>> modules, etc. > > >>>>>>>> The problem seems to be that some people believe that we would > > >> have > > >>>> to > > >>>>> cut > > >>>>>>>> a > > >>>>>>>> release of those every time we do a log4j release. > > >>>>>>>> > > >>>>>>>> If we were to do that 3.0 would be the right time. > > >>>>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > >
