FWIW, Some of this is borne out by looking at the stats at https://repository.apache.org/#central-stat <https://repository.apache.org/#central-stat>. The good news is that the downloads are 3 times what they were a year ago. But you can see that the usage of these extra components is 1000 times smaller than log4j-core.
Ralph > On Jan 21, 2018, at 9:35 PM, Ralph Goers <ralph.go...@dslextreme.com> wrote: > > Remko, > > I am all for speeding up the build. I have been complaining about this for > well over a year and it keeps getting worse with every release. The compile > of log4j-core takes less than 20 seconds on the machine provided by my > employer. On the same machine the tests take 11 minutes. On this machine the > whole build takes about 16 minutes. This is a bit frustrating as I perform a > complete build at least once before I commit 95% of my changes. > > My personal machine, which is where I run the release builds, is over 6 years > old now and is considerably slower. On that machine a “normal” build > currently takes . In addition, the release build does perform more work and > runs through the build twice (although I am not sure it runs the tests twice > anymore) so it is considerably longer. Also with every release we add new > unit tests. It wasn’t too long ago that I was marveling that we had over 900 > unit tests. We now have over 2,400. In addition, each release also requires > building the web site, which can take a long time as well. > > While I suspect about 1/3 of the time of a “normal” build could be cut by > executing some stuff in parallel that “fix” will only last for a short while > as we keep adding components related to third party projects (Mongo, > Cassandra, DBCP2, etc) that most likely don’t change much once they are done. > I can’t remember the tag library module required changes. Many of them only > change when the third party project changes (such as Mongo 2 vs Mongo 3). On > that note I really don’t want to see Mongo 2 and Mongo 3 projects as part of > the Log4j main build. > > In addition to speeding up the build I would prefer to have the main log4j > releases follow the 80/20 rule - they should contain only the stuff 80% of > our users will use. I would venture a guess that at least 80% only use the > File and Console appenders. > > While we may not have as firm of a plugin API as Maven does, almost > everything a user should need to create an Appender, Layout, Lookup or Filter > should be as backward compatible as the Log4j API. If we can’t guarantee > backward compatibility for those components we have a big problem as users > are creating these kinds of components all the time. We cannot be breaking > them. So from that standpoint we should have no problem moving those kinds of > plugins out of the main build. > > Ralph > > > >> On Jan 21, 2018, at 5:51 PM, Remko Popma <remko.po...@gmail.com> wrote: >> >> Log4j2 is not like Maven. Maven (I assume) has a plugin API. Log4j2 modules >> depend on the internals of log4j-core. >> >> I agree with Gary that not being able to verify that a core change doesn’t >> break any modules when they live outside the main project is a valid >> concern. >> >> Why are we talking about modules and repos when the issue is that it takes >> too long to do a release? Maven reports most modules as taking a handful of >> seconds to build. >> >> I keep thinking: >> >> 1. We’re running the tests twice during a release. Let’s change this. >> Logically, once should be enough. What prevents us from doing this? >> >> 2. Core tests take unnecessarily long. By running some tests in parallel and >> some other tests without forking I have hope we can run all core tests in >> ~15 minutes. >> >> Won’t those two changes be much more impactful in reducing the release time >> than any amount of module or repo reshuffling? >> >> Remko >> >> (Shameless plug) Every java main() method deserves http://picocli.info >> >>> On Jan 22, 2018, at 8:14, Ralph Goers <ralph.go...@dslextreme.com> wrote: >>> >>> I am very much against the idea of a single repo. Yes, you can have >>> multiple projects in the repo but I am not sure how that can be sanely >>> released. I much prefer the model that Maven has taken. They are now using >>> gitbox [1] which seems to allow GitHub to be the primary repo. Every Maven >>> plugin is individually released. Scroll down the link below to the Maven >>> section and you can see all the plugin repos. >>> >>> The upside to this is that it are: >>> 1. It is far easier to perform releases of the individual components. >>> 2. It is much easier to accept plugin contributions. >>> The downsides are: >>> 1. A page like https://maven.apache.org/plugins/ >>> <https://maven.apache.org/plugins/> is needed to keep track of the plugin >>> versions. >>> 2. It could make sense to have log4j-parent and log4j-bom projects. The >>> first to help keep the builds similar and the second to help customers pick >>> up the latest versions. >>> >>> Ralph >>> >>> [1] https://gitbox.apache.org/repos/asf >>> <https://gitbox.apache.org/repos/asf> >>> >>>> On Jan 21, 2018, at 8:55 AM, Gary Gregory <garydgreg...@gmail.com> wrote: >>>> >>>> Hi All, >>>> >>>> I am writing this to in an effort to understand how we can manage and grow >>>> Log4j. I use the term 'grow' not in the 'bigger is better' sense but more >>>> in the maturing sense. >>>> >>>> I am prompted to write this by Ralph's comment that log4j-core and the main >>>> repo is too big and that releases take too long make, partially prompted by >>>> my addition of a new module called log4j-jdbc-dbcp2 which currently >>>> contains one small class with a dependency on Apache Commons DBCP 2. >>>> >>>> I would like to express my gratitude at Ralph's efforts to revitalized >>>> Log4j and performing most release management duties. Thank you Ralph! >>>> >>>> I can see two main orthogonal issues: >>>> - The size of the git repo. >>>> - The size of the log4j-core module. >>>> >>>> Ralph has proposed that new modules like log4j-jdbc-dbcp2 be moved to >>>> another 'plugin' repo. >>>> >>>> I really dislike this idea: >>>> - The plugin repo has never been released. Not that one releases a repo but >>>> you get the point. >>>> - How do you keep things in sync between repos and code when we have no >>>> official 'core' SPI. >>>> >>>> For my money, we should keep _everything_ in one repo. Good enough for >>>> Google, so good enough for me. What we release out of that repo is a >>>> different story and what I would like to discuss next. >>>> >>>> This is not the same as Maven plugins IMO but the case could be made for it >>>> I suppose where a lot/most plugins live in their own repos. It is not the >>>> same as with Maven IMO because our plugins rely on log4j-core and it's >>>> guts, for which we only make loose compatibility guarantees -- as opposed >>>> to log4j-core where we are strict(er). Maven OTOH, has a API for plugin >>>> auteurs. >>>> >>>> For example, log4j-jdbc-dbcp2 replies on the guts of log4j-core and we have >>>> no 'official' core SPI, so splitting it off into a separate repo would >>>> greatly increase the risk of it falling out of sync. It is just so much >>>> more easier to maintain when it is all in one repo. >>>> >>>> My proposal is to: >>>> >>>> - Put everything back into one repo (Chainsaw too?) >>>> - Define a core SPI for plugin writers where we make some statement about >>>> BC, more than the casual 'we try not break stuff.' >>>> - Defining what Log4j project 'components' we have and release based on >>>> that. For example, today, all of the main Log4j repo is one component with >>>> many modules. Chainsaw would be another component of the Log4j project. >>>> Maybe we need to redefine components: API, File, JDBC and so on. A >>>> component can have one of more module >>>> - Got for there. >>>> >>>> Thoughts? Help me flush this out? >>>> >>>> Thank you for reading! >>>> Gary >>> >> > > >
