Re: [VOTE] Release Apache Log4j 2.21.1 RC1
+1 Checked sigs, hashes, reproducibility, and commits compared to `rel/2.21.0` tag. On Fri, Oct 20, 2023 at 11:20 PM Piotr P. Karwasz wrote: > This is a vote to release the Apache Log4j 2.21.1. > > Website: https://logging-log4j2.staged.apache.org/log4j > GitHub: https://github.com/apache/logging-log4j2 > Commit: e613e9ed71279bb52753a4df810d61c11389df81 > Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j > Nexus: > https://repository.apache.org/content/repositories/orgapachelogging-1209 > Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0 > > Please download, test, and cast your votes on this mailing list. > > [ ] +1, release the artifacts > [ ] -1, don't release, because... > > This vote is open for 72 hours and will pass unless getting a > net negative vote count. All votes are welcome and we encourage > everyone to test the release, but only the Logging Services PMC > votes are officially counted. > > == Countdown > > > https://www.timeanddate.com/countdown/party?iso=20231023T2320&p0=1449&font=cursive > > == Release Notes > > This patch release contains only the fix of a `log4j-jcl` bug that > prevents it from connecting with `commons-logging`. > > The Log4j 2.21.1 API, as well as the other artifacts, maintains binary > compatibility with the previous release. > > Apache Log4j 2.21.1 requires Java 8 to run. > The build requires JDK 11 and generates reproducible binaries. > > For complete information on Apache Log4j 2, including instructions on > how to submit bug reports, patches, get support, or suggestions for > improvement, see http://logging.apache.org/log4j/2.x/[the Apache Log4j > 2 website]. > > > === Fixed > > * Fixes the Apache Commons Logging (JCL) bridge: `log4j-jcl`. (#1865) > > Piotr >
[log4j] Annotation processing alternatives
Good point Matt! Annotation processing has always been an unpleasant magic
to work with from a developer perspective for the reasons you shared. What
are our alternatives? Can't we offer a `META-INF/log4j.factories`
functionality similar to Spring Boot? If so,
1. What are its cons/pros for users?
2. What are its implementation requirements (for the Log4j crew) roughly?
On Fri, Oct 20, 2023 at 7:40 PM Matt Sicker wrote:
> Snippet from the JDK 21 announcement email that pertains to us. While the
> annotation processor was initially developed to try to be implicit and
> magic, as we’ve seen both in our build and in IDEs, annotation processing
> is typically either disabled by default (in IDEs) or sometimes requires
> manual setup anyways (like in our build). It sounds like starting in Java
> 22, implicit annotation processing will no longer be available. We’ll need
> to ensure our docs about this mention that you need to enable annotation
> processing in your build, too.
>
> > Begin forwarded message:
> >
> > From: David Delabassee
> > Subject: JDK 21 Is Now GA, a New VS Code Extension, and an Annotation
> Processing Heads-up
> > Date: October 20, 2023 at 4:39:07 AM CDT
> > To: "dev@logging.apache.org"
> > Reply-To: dev@logging.apache.org
> > List-Id:
> >
> > ## Heads-Up - JDK 22: Implicit Annotation Processing Behavior Change
> >
> > As discussed in the July 2023 Quality Outreach update [8], starting in
> JDK 21 javac emits a note if _implicit_ annotation processing is being
> used, that is, if one or more annotation processors are found and run from
> the class path when no explicit annotation processing configuration options
> are used.
> >
> > The note is reported since, quoting from the note text: "A future
> release of javac may disable annotation processing unless at least one
> processor is specified by name (-processor), or a search path is specified
> (--processor-path, --processor-module-path), or annotation processing is
> enabled explicitly (-proc:only, -proc:full)."
> >
> > That future version of javac has arrived in JDK 22 b19+ with JDK-8306819
> ("Consider disabling the compiler's default active annotation processing").
> In the situation where a note was emitted in JDK 21, in JDK 22 no note is
> emitted, and annotation processors are *not* run. To restore the previous
> behavior with respect to running annotation processors, add the
> '-proc:full' javac option.
> >
> > Feedback on the annotation processing policy change can be sent to
> compiler-dev [9].
> >
> > [8]
> https://mail.openjdk.org/pipermail/quality-discuss/2023-July/001122.html
> > [9] https://mail.openjdk.org/mailman/listinfo/compiler-dev
>
>
[log4j] Project URLs per major version
Currently, we have the following folder structure in the `logging-log4j-site` repository for the Log4j project: - `log4j-1.2.17` – the website generated by the last Log4j 1 release, i.e, `1.2.17` - `log4j-1.2` – symlinks to `log4j-1.2.17` - `1.x` – symlinks to `log4j-1.2.17` - ... - `log4j-2.19.0` – the website generated by the `2.19.0` release - `log4j-2.20.0` – the website generated by the `2.20.0` release - `log4j-2.21.0` – the website generated by the `2.21.0` release - `2.x` – symlinks to `log4j-2.21.0` (the latest Log4j 2 release) I propose keeping existing folders as is (hence, *no changes to the existing URLs!*) and applying following practices: 1. *Add a `latest` symlink* pointing to the latest stable version. Today it will point to `2.x`, sometime in the future it will point to `3.x`. - This will make following URLs possible: https://logging.apache.org/log4j/latest - This convention is practiced by several other projects, e.g., Spring Boot: https://docs.spring.io/spring-boot/docs/current 2. *Redirect root to `latest`*, i.e., `logging.apache.org/log4j` will redirect to `logging.apache.org/log4j/latest` 3. *Don't create a new folder for every release, but override the `2.x` folder.* - This is okay, since we keep backward compatibility in minor+patch releases and explicitly provide version for features that are added later on (e.g., "starting from 2.17.0 Log4j provides X...") - We can set up CI jobs to periodically populate `1.x`, `2.x`, `3.x`, `4.x`, etc. websites and avoid the need to generate the website once and for all. - We will stop polluting the folder structure. As a matter of fact, I already implemented this convention for some projects. All the following URLs work: - `logging-parent` - logging.apache.org/logging-parent - logging.apache.org/logging-parent/latest - logging.apache.org/logging-parent/10.x - `logging-log4j-kotlin` - logging.apache.org/log4j/kotlin - logging.apache.org/log4j/kotlin/latest - logging.apache.org/log4j/kotlin/1.x - `logging-log4j-scala` - logging.apache.org/log4j/scala - logging.apache.org/log4j/scala/latest - logging.apache.org/log4j/scala/13.x - `logging-log4j-tools` - logging.apache.org/log4j/tools - logging.apache.org/log4j/tools/latest - logging.apache.org/log4j/tools/0.x - `logging-log4j-transform` - logging.apache.org/log4j/transform - logging.apache.org/log4j/transform/latest - logging.apache.org/log4j/transform/0.x Unless there are objections, I will update the CI in this direction and we all will enjoy automatically populated Log4j websites.
Re: [log4j] Annotation processing alternatives
Let me expand on this question with further context as it relates to why the
annotation processor exists in the first place. Back when the initial plugin
system was developed by Ralph, we had an option to specify a comma-separated
list of packages to scan for plugins (note that scanning the entire classpath
was and still is a poor way to implement this), and we had a default package in
the code to scan for log4j-core plugins. Everything still leads back to this
general need for discovering the list of classes and their corresponding plugin
metadata (primarily the plugin name which is used for configuring things like
`…` or `…` rather than
`…` as well as
the category/namespace) so that various configurable classes are instantiated
from the parsed configuration data. This can be sort of compared to use of
things like type discriminators in JSON where types are all resolved against
plugin names rather than arbitrary class names.
To alleviate this scan, we started with a cache file that dumped out metadata
about plugins discovered from reflection-based classpath scanning (via a class
in log4j-core with a main method). Later on, we added an annotation processor
that did the same thing as this bundled program. In 3.x, we replaced this cache
file with generated Java code implementing a ServiceLoader-loaded class that is
almost exactly the same thing as what the cache file did before (serialize and
deserialize a list of PluginEntry instances through some format supported by
core Java).
So now we come to your question about a factories file. That’s an interesting
idea, though it’s extremely similar to what we’re doing here with ServiceLoader
(though the split between META-INF/services/ files and module-info.java
provides/uses entries is confusing). Moving away from ServiceLoader to a custom
mechanism might be more flexible, though I’m not very knowledgeable about how
Java modules work with them.
If you examine the output of a generated plugin service file from the
annotation processor, you may see that it’s something that you can easily write
by hand, but it’s tedious as it’s replicating information from the plugin
annotations into PluginEntry instances.
—
Matt Sicker
> On Oct 22, 2023, at 14:35, Volkan Yazıcı wrote:
>
> Good point Matt! Annotation processing has always been an unpleasant magic
> to work with from a developer perspective for the reasons you shared. What
> are our alternatives? Can't we offer a `META-INF/log4j.factories`
> functionality similar to Spring Boot? If so,
>
> 1. What are its cons/pros for users?
> 2. What are its implementation requirements (for the Log4j crew) roughly?
>
>
> On Fri, Oct 20, 2023 at 7:40 PM Matt Sicker wrote:
>
>> Snippet from the JDK 21 announcement email that pertains to us. While the
>> annotation processor was initially developed to try to be implicit and
>> magic, as we’ve seen both in our build and in IDEs, annotation processing
>> is typically either disabled by default (in IDEs) or sometimes requires
>> manual setup anyways (like in our build). It sounds like starting in Java
>> 22, implicit annotation processing will no longer be available. We’ll need
>> to ensure our docs about this mention that you need to enable annotation
>> processing in your build, too.
>>
>>> Begin forwarded message:
>>>
>>> From: David Delabassee
>>> Subject: JDK 21 Is Now GA, a New VS Code Extension, and an Annotation
>> Processing Heads-up
>>> Date: October 20, 2023 at 4:39:07 AM CDT
>>> To: "dev@logging.apache.org"
>>> Reply-To: dev@logging.apache.org
>>> List-Id:
>>>
>>> ## Heads-Up - JDK 22: Implicit Annotation Processing Behavior Change
>>>
>>> As discussed in the July 2023 Quality Outreach update [8], starting in
>> JDK 21 javac emits a note if _implicit_ annotation processing is being
>> used, that is, if one or more annotation processors are found and run from
>> the class path when no explicit annotation processing configuration options
>> are used.
>>>
>>> The note is reported since, quoting from the note text: "A future
>> release of javac may disable annotation processing unless at least one
>> processor is specified by name (-processor), or a search path is specified
>> (--processor-path, --processor-module-path), or annotation processing is
>> enabled explicitly (-proc:only, -proc:full)."
>>>
>>> That future version of javac has arrived in JDK 22 b19+ with JDK-8306819
>> ("Consider disabling the compiler's default active annotation processing").
>> In the situation where a note was emitted in JDK 21, in JDK 22 no note is
>> emitted, and annotation processors are *not* run. To restore the previous
>> behavior with respect to running annotation processors, add the
>> '-proc:full' javac option.
>>>
>>> Feedback on the annotation processing policy change can be sent to
>> compiler-dev [9].
>>>
>>> [8]
>> https://mail.openjdk.org/pipermail/quality-discuss/2023-July/001122.html
>>> [9] https://mail.openjdk.org/mailman/listin
Re: [log4j] Project URLs per major version
This sounds pretty cool Volkan, if I didn't get anything I pretty much like it. One question - you wrote: "we all will enjoy automatically populated Log4j websites." Does this mean we will see our website is updated automatically by CI so we can see in example /log4j/dev? Or what else is "automatically updated"? On Sun, Oct 22, 2023, at 22:20, Volkan Yazıcı wrote: > Currently, we have the following folder structure in the > `logging-log4j-site` repository for the Log4j project: > >- `log4j-1.2.17` – the website generated by the last Log4j 1 release, >i.e, `1.2.17` >- `log4j-1.2` – symlinks to `log4j-1.2.17` >- `1.x` – symlinks to `log4j-1.2.17` >- ... >- `log4j-2.19.0` – the website generated by the `2.19.0` release >- `log4j-2.20.0` – the website generated by the `2.20.0` release >- `log4j-2.21.0` – the website generated by the `2.21.0` release >- `2.x` – symlinks to `log4j-2.21.0` (the latest Log4j 2 release) > > I propose keeping existing folders as is (hence, *no changes to the > existing URLs!*) and applying following practices: > >1. *Add a `latest` symlink* pointing to the latest stable version. Today >it will point to `2.x`, sometime in the future it will point to `3.x`. > - This will make following URLs possible: > https://logging.apache.org/log4j/latest > - This convention is practiced by several other projects, e.g., > Spring Boot: https://docs.spring.io/spring-boot/docs/current >2. *Redirect root to `latest`*, i.e., `logging.apache.org/log4j` will >redirect to `logging.apache.org/log4j/latest` >3. *Don't create a new folder for every release, but override the `2.x` >folder.* > - This is okay, since we keep backward compatibility in minor+patch > releases and explicitly provide version for features that are added > later > on (e.g., "starting from 2.17.0 Log4j provides X...") > - We can set up CI jobs to periodically populate `1.x`, `2.x`, `3.x`, > `4.x`, etc. websites and avoid the need to generate the website once and > for all. > - We will stop polluting the folder structure. > > As a matter of fact, I already implemented this convention for some > projects. All the following URLs work: > >- `logging-parent` > - logging.apache.org/logging-parent > - logging.apache.org/logging-parent/latest > - logging.apache.org/logging-parent/10.x >- `logging-log4j-kotlin` > - logging.apache.org/log4j/kotlin > - logging.apache.org/log4j/kotlin/latest > - logging.apache.org/log4j/kotlin/1.x >- `logging-log4j-scala` > - logging.apache.org/log4j/scala > - logging.apache.org/log4j/scala/latest > - logging.apache.org/log4j/scala/13.x >- `logging-log4j-tools` > - logging.apache.org/log4j/tools > - logging.apache.org/log4j/tools/latest > - logging.apache.org/log4j/tools/0.x >- `logging-log4j-transform` > - logging.apache.org/log4j/transform > - logging.apache.org/log4j/transform/latest > - logging.apache.org/log4j/transform/0.x > > Unless there are objections, I will update the CI in this direction and we > all will enjoy automatically populated Log4j websites.
Re: [log4j] Annotation processing alternatives
Hi Matt, On Sun, 22 Oct 2023 at 22:49, Matt Sicker wrote: > So now we come to your question about a factories file. That’s an interesting > idea, though it’s extremely similar to what we’re doing here with > ServiceLoader (though the split between META-INF/services/ files and > module-info.java provides/uses entries is confusing). Moving away from > ServiceLoader to a custom mechanism might be more flexible, though I’m not > very knowledgeable about how Java modules work with them. I think that `ServiceLoader` is the only standard feature that is supported by both OSGi and JPMS. If we move away from it, we'll need some custom code to support it (or `opens`, etc). I wouldn't go that way. Piotr
Re: [log4j] Project URLs per major version
Hi Volkan, On Sun, 22 Oct 2023 at 22:20, Volkan Yazıcı wrote: >3. *Don't create a new folder for every release, but override the `2.x` >folder.* > - This is okay, since we keep backward compatibility in minor+patch > releases and explicitly provide version for features that are added > later > on (e.g., "starting from 2.17.0 Log4j provides X...") > - We can set up CI jobs to periodically populate `1.x`, `2.x`, `3.x`, > `4.x`, etc. websites and avoid the need to generate the website once and > for all. > - We will stop polluting the folder structure. +1 on keeping just `1.x`, `2.x`, etc. We just need to keep in mind that not all public methods have a correct `@since` Javadoc tag. We would need to fix that. Piotr
Re: JDK 21 Is Now GA, a New VS Code Extension, and an Annotation Processing Heads-up
Yeah I think it’s mainly a documentation thing for 3.x.
—
Matt Sicker
> On Oct 20, 2023, at 18:06, Ralph Goers wrote:
>
> I don’t think this is a problem. Only users of Log4j 3.x should be using Java
> 17 and up by the time this makes it to an LTS release. Log4j 3.x has put the
> annotation processor in its own jar. When using JPMS the annotation processor
> has to be explicitly specified so we really had no choice but to do this. I
> suspect this isn’t documented as well as it should be though.
>
> Ralph
>
>> On Oct 20, 2023, at 10:38 AM, Matt Sicker wrote:
>>
>> Snippet from the JDK 21 announcement email that pertains to us. While the
>> annotation processor was initially developed to try to be implicit and
>> magic, as we’ve seen both in our build and in IDEs, annotation processing is
>> typically either disabled by default (in IDEs) or sometimes requires manual
>> setup anyways (like in our build). It sounds like starting in Java 22,
>> implicit annotation processing will no longer be available. We’ll need to
>> ensure our docs about this mention that you need to enable annotation
>> processing in your build, too.
>>
>>> Begin forwarded message:
>>>
>>> From: David Delabassee
>>> Subject: JDK 21 Is Now GA, a New VS Code Extension, and an Annotation
>>> Processing Heads-up
>>> Date: October 20, 2023 at 4:39:07 AM CDT
>>> To: "dev@logging.apache.org"
>>> Reply-To: dev@logging.apache.org
>>> List-Id:
>>>
>>> ## Heads-Up - JDK 22: Implicit Annotation Processing Behavior Change
>>>
>>> As discussed in the July 2023 Quality Outreach update [8], starting in JDK
>>> 21 javac emits a note if _implicit_ annotation processing is being used,
>>> that is, if one or more annotation processors are found and run from the
>>> class path when no explicit annotation processing configuration options are
>>> used.
>>>
>>> The note is reported since, quoting from the note text: "A future release
>>> of javac may disable annotation processing unless at least one processor is
>>> specified by name (-processor), or a search path is specified
>>> (--processor-path, --processor-module-path), or annotation processing is
>>> enabled explicitly (-proc:only, -proc:full)."
>>>
>>> That future version of javac has arrived in JDK 22 b19+ with JDK-8306819
>>> ("Consider disabling the compiler's default active annotation processing").
>>> In the situation where a note was emitted in JDK 21, in JDK 22 no note is
>>> emitted, and annotation processors are *not* run. To restore the previous
>>> behavior with respect to running annotation processors, add the
>>> '-proc:full' javac option.
>>>
>>> Feedback on the annotation processing policy change can be sent to
>>> compiler-dev [9].
>>>
>>> [8] https://mail.openjdk.org/pipermail/quality-discuss/2023-July/001122.html
>>> [9] https://mail.openjdk.org/mailman/listinfo/compiler-dev
>>
>
Re: Staging sites and repo convention
It has been a long thread and I want to capture the result: *there are no objections to Piotr's proposal, right?* If not, please say so. To avoid misunderstanding, I want to repeat certain points one more time: 1. All existing logging.apache.org URLs will remain as is – no changes there. 2. Instead of using logging.*staged.*apache.org*/foo*, we will use logging*-foo.staged.*apache.org for staging websites. 3. Log4j Scala, Kotlin, Tools, and Transformation website content will be moved from `logging-log4j-site` repository to `logging-log4j-scala`, `logging-log4j-kotlin`, `logging-log4j-tools`, and `logging-log4j-transformation` repositories, respectively. On Thu, Oct 19, 2023 at 10:03 AM Piotr P. Karwasz wrote: > Hi, > > Since now we have a fast release process It might happen (and it > already did) that the voting periods for releases will not be > disjoint. > > That is why I would like to introduce a convention on the procedure to > stage websites and Nexus repositories. > > For websites I would propose: > > 1. Every Git code repository uses a different staging domain name. > E.g. `logging-log4j2` would set: > > staging: > profile: log4j2 > > which will result in a https://logging-log4j2.staged.apache.org URI. > For the `logging-log4j-site` website repo this will also entail that > it will have multiple staging branches. > 2. The `asf-staging` should not be protected. Before staging a website > the Release Manager would perform: > > git reset --hard origin/asf-site > git push -f > > hence ensuring that moving changes from the staging branch to > `asf-site` will be usually a fast-forward and a simple cherry-pick > `origin/asf-site..asf-staging` at worst. > > For the staging Nexus repo I would propose using a comment to close > the repo in the format: > > `` version `` RC1 > > For example Volkan used "`logging-parent` version `10.2.0` RC`" on the > 1204 repo and we can easily guess what that repo contains. ;-) > > Piotr > > PS: Maybe we could drop the `*-site` Git repositories except > `logging-site` and move their content to an `asf-site/asf-staging` > branch of the corresponding code repo. >
Re: Staging sites and repo convention
Hi Volkan, On Sun, 22 Oct 2023 at 23:47, Volkan Yazıcı wrote: >2. Instead of using logging.*staged.*apache.org*/foo*, we will use >logging*-foo.staged.*apache.org for staging websites. >3. Log4j Scala, Kotlin, Tools, and Transformation website content will >be moved from `logging-log4j-site` repository to `logging-log4j-scala`, >`logging-log4j-kotlin`, `logging-log4j-tools`, and >`logging-log4j-transformation` repositories, respectively. If we implement 3, 2 will become optional: as long as each project is on a separate Git repo/branch staging should be easy. Piotr
Re: Staging sites and repo convention
On Sun, Oct 22, 2023, at 21:54, Volkan Yazıcı wrote: > It has been a long thread and I want to capture the result: *there are no > objections to Piotr's proposal, right?* If not, please say so. I am not objecting, but I would like to point out that logging.s.a.o is in bad shape today, and I don't know why (in detail). I'd love to have this working again before we move on to a different system. If you know how to fix the system in a way that works with the original proposal in one go, I am okay with it, too. However, I am afraid we are touching too many wheels at once and navigating in a bad state. > > To avoid misunderstanding, I want to repeat certain points one more time: > >1. All existing logging.apache.org URLs will remain as is – no changes >there. >2. Instead of using logging.*staged.*apache.org*/foo*, we will use >logging*-foo.staged.*apache.org for staging websites. >3. Log4j Scala, Kotlin, Tools, and Transformation website content will >be moved from `logging-log4j-site` repository to `logging-log4j-scala`, >`logging-log4j-kotlin`, `logging-log4j-tools`, and >`logging-log4j-transformation` repositories, respectively. > > > On Thu, Oct 19, 2023 at 10:03 AM Piotr P. Karwasz > wrote: > >> Hi, >> >> Since now we have a fast release process It might happen (and it >> already did) that the voting periods for releases will not be >> disjoint. >> >> That is why I would like to introduce a convention on the procedure to >> stage websites and Nexus repositories. >> >> For websites I would propose: >> >> 1. Every Git code repository uses a different staging domain name. >> E.g. `logging-log4j2` would set: >> >> staging: >> profile: log4j2 >> >> which will result in a https://logging-log4j2.staged.apache.org URI. >> For the `logging-log4j-site` website repo this will also entail that >> it will have multiple staging branches. >> 2. The `asf-staging` should not be protected. Before staging a website >> the Release Manager would perform: >> >> git reset --hard origin/asf-site >> git push -f >> >> hence ensuring that moving changes from the staging branch to >> `asf-site` will be usually a fast-forward and a simple cherry-pick >> `origin/asf-site..asf-staging` at worst. >> >> For the staging Nexus repo I would propose using a comment to close >> the repo in the format: >> >> `` version `` RC1 >> >> For example Volkan used "`logging-parent` version `10.2.0` RC`" on the >> 1204 repo and we can easily guess what that repo contains. ;-) >> >> Piotr >> >> PS: Maybe we could drop the `*-site` Git repositories except >> `logging-site` and move their content to an `asf-site/asf-staging` >> branch of the corresponding code repo. >>
