FWIW, my experience with GitHub actions over at Apache Commons is extremely positive.
Gary On Fri, Jun 11, 2021, 15:21 Matt Sicker <boa...@gmail.com> wrote: > We can certainly set up CI to build on upstream snapshot releases or even > nightly jobs. > > For improving the build time of core, that will require parallelizing tests > like in api. > > On Fri, Jun 11, 2021 at 13:48 Volkan Yazıcı <volkan.yaz...@gmail.com> > wrote: > > > I agree with all the points (both pros and cons) you have shared. > > > > A small "core" will speed up the development and release cycle > > significantly. Most of the time I skip a full Maven "verify", which is > > taking more than half an hour on my machine. > > > > Regarding CI, I believe there will be a huge win too due to faster > feedback > > cycles. Also note that it is possible to trigger GitHub Actions workflows > > from another repository change. Hence the consistency check across > > repositories you have described can easily be set up. > > > > On Fri, Jun 11, 2021 at 7:45 PM Ralph Goers <ralph.go...@dslextreme.com> > > wrote: > > > > > Not sure what you want a recap on. I see a few issues on both sides of > > the > > > coin: > > > > > > For splitting: > > > 1. Many of the current modules rarely get modified. Creating a new > > version > > > of > > > them with each release just so the release number matches seems silly. > > > 2. Yes, creating a release is time consuming. When I am nearing the > time > > to > > > perform a release I: > > > a. Review PRs and Jira issues looking for ones that are either > > critical, > > > serious > > > or easy to fix and try to address them. This typically involves > > > running the > > > build several times. > > > b. Perform all the required release updates. > > > c. Build the web site locally and inspect it. > > > d. Follow the guide to actually perform the release. > > > The prep time in step a is typically what takes the most time due to > how > > > many > > > times I run the full build. This is where the most time savings would > be > > > since > > > most of the changes occur in log4j-api and log4j-core, although the > most > > > time > > > consuming part of the build is the log4j-core unit tests. > > > 3. This might encourage everyone to run a full build for every commit. > > > > > > Against splitting: > > > 1. Confusion over where a particular component can be found. > > > 2. Although unlikely, it is possible a change in log4j-api or > log4j-core > > > could > > > break something in one of the other modules and we wouldn’t find out > > > immediately as we do today. This could be solved by setting up good > > > Jenkins > > > pipelines that cause all the sub projects to rebuild whenever the main > > > module is built. > > > > > > Ralph > > > > > > > On Jun 11, 2021, at 10:12 AM, Gary Gregory <garydgreg...@gmail.com> > > > wrote: > > > > > > > > I am on 100% board with spliting up modules/artifacts to reduce the > > "suck > > > > the world in" issue. Splitting into separate repos/projects I have > zero > > > > interest in, I only see drawbacks. Version confusion, BOM POM or not, > > > weird > > > > dependeny issues, more fiddling with IDEs to make sure you can get > > stuff > > > > done, bleh. > > > > > > > > I know Ralph has been quite generous with him time in release > > management > > > > and has mentioned numerous time that he feels the current repository > > size > > > > leads to long release candiate build times. But I see this as a > drive > > to > > > > break up the repository wrong headed. When I run any of the various > > Maven > > > > builds I use, some builds rebuilds various pieces needlessly, > compiling > > > > code when no recompilation is needed for example, running tests more > > than > > > > needed, and other Maven oddities. If the tooling is a problem, then > > lets > > > > look at that first. > > > > > > > > Maybe we could get a recap from Ralph? > > > > > > > > Gary > > > > > > > > > > > > On Fri, Jun 11, 2021, 10:42 Volkan Yazıcı <volkan.yaz...@gmail.com> > > > wrote: > > > > > > > >> Gary, I thought you were on board with breaking up into separate > > > projects. > > > >> What do you propose instead? > > > >> > > > >> On Fri, Jun 11, 2021 at 12:22 PM Gary Gregory < > garydgreg...@gmail.com > > > > > > >> wrote: > > > >> > > > >>> From my POV, this will create an incomprehensible mess of versions > > for > > > >>> users. Big bummer. > > > >>> > > > >>> Gary > > > >>> > > > >>> On Fri, Jun 11, 2021, 04:29 Volkan Yazıcı <volkan.yaz...@gmail.com > > > > > >> wrote: > > > >>> > > > >>>> To start the discussion for breaking up the modules into separate > > > >>> projects, > > > >>>> I propose the following structure: > > > >>>> > > > >>>> *logging-log4j* > > > >>>> log4j-1.2-api > > > >>>> log4j-api > > > >>>> log4j-bom > > > >>>> log4j-core > > > >>>> log4j-core-its > > > >>>> log4j-gctests > > > >>>> log4j-iostreams > > > >>>> log4j-layout-template-json > > > >>>> log4j-perf > > > >>>> log4j-plugins > > > >>>> > > > >>>> *logging-log4j-appender* > > > >>>> log4j-cassandra > > > >>>> log4j-couchdb > > > >>>> log4j-csv > > > >>>> log4j-flume-ng > > > >>>> log4j-mongodb3 > > > >>>> log4j-mongodb4 > > > >>>> log4j-smtp > > > >>>> > > > >>>> *logging-log4j-appender-jdbc* > > > >>>> log4j-jdbc (?) > > > >>>> log4j-jdbc-dbcp2 (?) > > > >>>> log4j-jpa > > > >>>> > > > >>>> *logging-log4j-appender-queue* > > > >>>> log4j-jeromq > > > >>>> log4j-jms > > > >>>> log4j-kafka > > > >>>> log4j-redis > > > >>>> > > > >>>> *logging-log4j-binding* > > > >>>> log4j-jcl > > > >>>> log4j-jpl > > > >>>> log4j-jul > > > >>>> log4j-liquibase > > > >>>> log4j-slf4j18-impl > > > >>>> log4j-slf4j-impl > > > >>>> log4j-to-slf4j > > > >>>> > > > >>>> *logging-log4j-container* > > > >>>> log4j-docker > > > >>>> log4j-kubernetes > > > >>>> > > > >>>> *logging-log4j-gui* > > > >>>> log4j-jmx-gui > > > >>>> > > > >>>> *logging-log4j-jee* > > > >>>> log4j-appserver > > > >>>> log4j-taglib > > > >>>> log4j-web > > > >>>> > > > >>>> *logging-log4j-layout-jackson* > > > >>>> log4j-layout-jackson > > > >>>> log4j-layout-jackson-json > > > >>>> log4j-layout-jackson-xml > > > >>>> log4j-layout-jackson-yaml > > > >>>> > > > >>>> *logging-log4j-osgi* > > > >>>> log4j-osgi > > > >>>> > > > >>>> *logging-log4j-spring* > > > >>>> log4j-spring-boot > > > >>>> log4j-spring-cloud-config > > > >>>> > > > >>>> Note that above breakdown contains every available module in > master, > > > >>> except > > > >>>> log4j-samples, which has the following sub-modules: > > > >>>> > > > >>>> log4j-samples-loggerProperties (contains 2 very trivial example > > > >> classes, > > > >>>> log4j-core [tests] contains more comprehensive alternatives) > > > >>>> log4j-samples-flume-remote (flume-specific) > > > >>>> log4j-samples-flume-embedded (flume-specific) > > > >>>> log4j-samples-flume-common (flume-specific) > > > >>>> log4j-samples-configuration (contains 3 very trivial example > > classes, > > > >>>> log4j-core [tests] contains more comprehensive alternatives) > > > >>>> > > > >>>> I propose removing log4j-samples module and, if necessary, moving > > > >>>> Flume-related parts to the log4j-flume-ng. > > > >>>> > > > >>>> I have kept log4j-layout-template-json in the logging-log4j > project, > > > >>> since > > > >>>> it has no external dependencies. This said, I am okay with moving > it > > > >> to a > > > >>>> separate logging-log4j-layout project. > > > >>>> > > > >>>> Please share your remarks. Eventually, I want to translate this > > into a > > > >>> JIRA > > > >>>> ticket. > > > >>>> > > > >>>> Kind regards. > > > >>>> > > > >>>> On Thu, Jun 10, 2021 at 5:43 PM Ralph Goers < > > > >> ralph.go...@dslextreme.com> > > > >>>> wrote: > > > >>>> > > > >>>>> Gary, > > > >>>>> > > > >>>>> I am -1 to almost every item on your list. > > > >>>>> > > > >>>>> When I said break up I meant mostly moving most everything > outside > > of > > > >>>>> log4j-core, > > > >>>>> log4j-api, and log4j-plugins into separate repos such as > > > >>>>> logging-log4j-nosql, > > > >>>>> logging-log4j-pubsub, etc. These would not require groupId or > > > >>> artifactId > > > >>>>> changes > > > >>>>> although the versioning would potentially be out of synch with > the > > > >> main > > > >>>>> releases > > > >>>>> as each would be on its own release cycle. This would greatly > > > >> simplify > > > >>>>> releasing > > > >>>>> core but it would require careful though on what the versions > would > > > >> be > > > >>>> for > > > >>>>> the > > > >>>>> “extra” projects. > > > >>>>> > > > >>>>> As for further breaking up core, that should revolve primarily > > around > > > >>>>> reducing the > > > >>>>> dependencies listed in module-info.java to the bare minimum. > > > >>>>> > > > >>>>> There will be no log4j3. We cannot change groupIds, artifactIds, > or > > > >>>>> package names, > > > >>>>> other than what has been done to support JPMS. A world in which a > > > >>> log4j2 > > > >>>>> and > > > >>>>> log4j3 try to co-exist would be an unmitigated disaster. Commons > > can > > > >>>>> happily get > > > >>>>> away with that. Log4j cannot. > > > >>>>> > > > >>>>> If an application had both log4j2 and log4j3 jars present they > > would > > > >>> end > > > >>>>> up with > > > >>>>> multiple LoggerContexts, multiple Configurations, and multiple > > > >> Appender > > > >>>>> Managers > > > >>>>> where today there is only a single one. That would mean two > > instances > > > >>> of > > > >>>>> the same > > > >>>>> configuration file would be active at once. So when it is time to > > > >>>> rollover > > > >>>>> it would > > > >>>>> be performed twice instead of once as a simple example. For this > > > >> reason > > > >>>> we > > > >>>>> CANNOT break backward compatibility. > > > >>>>> > > > >>>>> However, we are talking about runtime backward compatibility. The > > > >>> Plugin > > > >>>>> system > > > >>>>> was changed internally in 3.0 so that plugins compiled with 3.0 > use > > > >>>>> ServiceLoader > > > >>>>> instead of the data file. However, it will still find and use 2.x > > > >>> plugins > > > >>>>> when they are > > > >>>>> present and can be located. > > > >>>>> > > > >>>>> The meaning of this should be clear. It is 3.0 because to take > > > >>> advantage > > > >>>>> of its > > > >>>>> features you may have to make code changes. But it will tolerate > > code > > > >>>> that > > > >>>>> was > > > >>>>> compiled for 2.x. > > > >>>>> > > > >>>>> The reasons why it is 3.0 and not a simple upgrade to 2.x are: > > > >>>>> 1. Plugins must be compiled with 3.0 to use the ServiceLoader > > > >>> packaging. > > > >>>>> 2. It requires Java 11. We still need to support Java 8. > > > >>>>> 3. It fully supports JPMS. Release 2.x does not. It is likely > that > > > >>>>> applications with > > > >>>>> multiple Module Layers might not be able to find all the plugins. > > > >>> Adding > > > >>>>> full JPMS > > > >>>>> support to 2.x simply isn’t possible. > > > >>>>> 4. It will be introducing new DI injection features not present > in > > > >> 2.x. > > > >>>>> > > > >>>>> Major releases do not imply that you are completely breaking > > backward > > > >>>>> compatibility. > > > >>>>> They imply that some kind of compatibility is broken, which we > are > > > >>> doing > > > >>>>> by > > > >>>>> requiring coding changes to Plugins to compile with 3.0. This > means > > > >> we > > > >>>>> need to > > > >>>>> leave in any classes or methods that existing plugins might have > > > >> used. > > > >>> It > > > >>>>> means > > > >>>>> we have to continue using our own Supplier unless it can be > > verified > > > >>> that > > > >>>>> an > > > >>>>> application using the Supplier in 2.x can run with 3.0 even if it > > is > > > >>>>> converted to > > > >>>>> java.util.function.Supplier. I have no idea if the code the > > compiler > > > >>>>> generates > > > >>>>> for lambdas actually implements the declared interface or not. > > > >>>>> > > > >>>>> Log4j 3.0 is a major change. But that doesn’t mean we can screw > our > > > >>> users > > > >>>>> by > > > >>>>> breaking everything. > > > >>>>> > > > >>>>> Ralph > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>>> On Jun 10, 2021, at 6:32 AM, Gary Gregory < > garydgreg...@gmail.com > > > > > > >>>>> wrote: > > > >>>>>> > > > >>>>>> These are all IMOs of course, YMMV: > > > >>>>>> - What's the rush to 3.0? > > > >>>>>> > > > >>>>>> - I'm all for breaking up the core and other artifacts into more > > > >>>>>> artifacts based on their dependencies requirement such that > > > >> depending > > > >>>>>> on Log4j 3 does not "suck in the world", for example, I should > be > > > >>> able > > > >>>>>> to depend on a currently non-existent "log4j3-console" and only > > > >> bring > > > >>>>>> in a tiny bit of code (API, a tiny core, and no plugin system). > I > > > >> did > > > >>>>>> a fair amount of breaking up of various artifacts a while back. > > > >>>>>> > > > >>>>>> - 3.0 is a MAJOR release that gives us the opportunity to drop > > > >>>>>> deprecated APIs and code like our custom functional interfaces: > > > >>>>>> Supplier and so on. If we do not clean up, then there is no > point > > > >> in > > > >>> a > > > >>>>>> major release. Basically, I expect to break binary and source > > > >>>>>> compatibility. > > > >>>>>> > > > >>>>>> - 3.0 must be in a new package namespace and new Maven > > > >> coordinates. I > > > >>>>>> MUST be able to run Log4j 1, 2, and 3 in the same class loader. > I > > > >> can > > > >>>>>> already run Log4j 1 and 2 side by side which is good. > > > >>>>>> > > > >>>>>> Gary > > > >>>>>> > > > >>>>>> On Thu, Jun 10, 2021 at 8:47 AM Volkan Yazıcı < > > > >>> volkan.yaz...@gmail.com > > > >>>>> > > > >>>>> wrote: > > > >>>>>>> > > > >>>>>>> Ralph, count me in for such a change. I really want to have > > > >> separate > > > >>>>>>> sub-projects for such modules. This will extremely speed up > > > >>>>> build/release > > > >>>>>>> times too, which is nowadays of uttermost importance to my > peace > > > >> of > > > >>>> mind > > > >>>>>>> while developing. > > > >>>>>>> > > > >>>>>>> This said, I am reluctant about such a major change when we are > > > >> this > > > >>>>> close > > > >>>>>>> to the 3.0.0 release. I guess this would definitely postpone > the > > > >>> 3.0.0 > > > >>>>>>> release to 2022. This will probably break the backward > > > >> compatibility > > > >>>> at > > > >>>>>>> least for the artifact groupId, am I wrong? Not to mention that > > > >> the > > > >>>>> entire > > > >>>>>>> website needs to be adapted to this multi-project setup too. Is > > > >>> there > > > >>>>>>> anything else? > > > >>>>>>> > > > >>>>>>> On Thu, Jun 3, 2021 at 5:31 PM Ralph Goers < > > > >>>> ralph.go...@dslextreme.com> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> Yeah - I have proposed moving all these extra integrations to > a > > > >>>>> separate > > > >>>>>>>> repo but > > > >>>>>>>> I’ve never gotten consensus. I’d prefer to have a project like > > > >>>>>>>> log4j-pubsub where > > > >>>>>>>> things like JMS, JeroMQ, etc can go live, log4j-nosql for all > > the > > > >>>> nosql > > > >>>>>>>> modules, etc. > > > >>>>>>>> The problem seems to be that some people believe that we would > > > >> have > > > >>>> to > > > >>>>> cut > > > >>>>>>>> a > > > >>>>>>>> release of those every time we do a log4j release. > > > >>>>>>>> > > > >>>>>>>> If we were to do that 3.0 would be the right time. > > > >>>>>>>> > > > >>>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>> > > > >>> > > > >> > > > > > > > > > > > >
