Re: [log4j] Using `DateTimeFormatter` by default

2024-10-14 Thread Ralph Goers
Before I can weigh in one way or another I would need to know what the 
performance difference is between DateTimeFormatter and the Fixed and Fast 
classes, at least where they are working properly. Any such test should include 
the caching otherwise you would be comparing worst case performance.

Ralph

> On Oct 14, 2024, at 12:30 PM, Volkan Yazıcı  wrote:
> 
> *Abstract:* Log4j contains two custom date & time formatting classes. All
> our competitors (Logback
> ,
> Tinylog
> ,
> etc.) use Java's `DateTimeFormatter` and I know of no user complaints about
> their performance issues. Shall we switch to `DateTimeFormatter` as the
> default in the next minor release and make the custom implementations
> opt-in?
> 
> Log4j ships two custom date & time formatting classes:
> 
>   - `FixedDateFormat`
>  - Supports a hardcoded list of formats
>  - Bug: Conflates `n` directive with `S`
>  - Bug: It cannot calculate DST correctly for all time zones
>  
>   - `FastDateFormat`
>  - Copied from Commons Lang in 2015
> 
> I recently ran a test comparing the output of these with
> `DateTimeFormatter` – the difference is *big*! I can spend months fixing
> these issues, but... Do we really need them?
> 
>   1. No other competitors have such an optimization (both Logback
>   
> 
>and Tinylog
>   
> 
> use
>   `DateTimeFormatter`)
>   2. Date & time formatting is very fragile (DST is more voodoo than
>   science)
>   3. If date & time formatting during logging is found to be the
>   bottleneck of an application, shouldn't they instead encode it differently,
>   e.g., encoding epoch numbers?
>   4. `DateTimeFormatter` performance will only get better over time
> 
> Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
> the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
> 
> Note that the caching optimization we have for the last formatted timestamp
> will stay untouched.



Re: Successor to mailing lists

2024-10-14 Thread Ralph Goers
Maybe we just need to start contributing to PonyMail to improve the UI to 
eliminate actually needing the email delivered to our accounts.

I am only 1/4 serious about this. There has to be a better solution.

Ralph

> On Oct 14, 2024, at 10:25 AM, Matt Sicker  wrote:
> 
> I didn’t get the original email in this thread once again, so I think I’d 
> support trying somewhere else to host discussions. Besides archiving those 
> messages into a mailing list, it would be great if the solution provided 
> allowed for email interactivity (e.g., you can reply to the notification of a 
> message and it’s added to the thread appropriately; this is how GitHub 
> notification emails typically work).
> 
>> On Oct 10, 2024, at 05:40, Christian Grobmeier  wrote:
>> 
>> Hi
>> 
>> I am generally open to such experiments. I would start with the easiest 
>> parts, such as users@, and see where it goes. 
>> 
>> I would advise against mirroring it to users@ behind the scenes, as it may 
>> cause privacy problems (we need user consensus to mirror it). When a user 
>> uses GitHub, they know what to expect.
>> 
>> As for Discourse, many use that now, but I find it very overwhelming and 
>> stressful. I prefer the clean Github discussions approach.
>> 
>> I haven't checked against ASF policies but feel positive about this move for 
>> the arguments you have given
>> 
>> Kind regards
>> Christian
>> 
>> 
>> 
>> On Thu, Oct 10, 2024, at 10:58, Volkan Yazıcı wrote:
>>> *Abstract:* Modern email system security measures make it practically
>>> impossible for mailing lists to work – many subscribers don't get all
>>> emails. This not only hinders communication, but blocks an inclusive
>>> one. *Shall
>>> we, as Logging Services, experiment with alternatives?*
>>> 
>>> *Motivation #1: mailing lists technically don't work*
>>> 
>>> Several widely-used email providers (GMail, Yahoo, iCloud, etc.) have in
>>> the last couple of years enabled new measures (DMARC, SPF, DKIM, etc.) to
>>> verify the authenticity of emails. In short, these measures enrich email
>>> content with checksums and signatures capturing its authenticity. When a
>>> mailing list system (e.g., ezmlm, mailman) receives such an email, it
>>> performs several changes on its content (adds information about the mailing
>>> list, etc.), and delivers it to all subscribers. When the mail server of a
>>> subscriber receives such tampered mail, and if that mail server happens to
>>> have earlier shared authenticity checks enabled, it discards the email, or
>>> at best, marks it as spam.
>>> 
>>> Ralph, Matt, Piotr stated many times that they don't receive all emails.
>>> Ralph actually stated many ASF mailing list emails end up in his spam 
>>> box
>>> .
>>> Recently we witnessed even Brian Proffitt (VP, Marketing & Publicity) 
>>> suffer
>>> from the same problem
>>> . 
>>> INFRA
>>> is crawling with related tickets: INFRA-24574
>>> , INFRA-24790
>>> , INFRA-24845
>>> , INFRA-24850
>>> , INFRA-24872
>>> , INFRA-25947
>>> , INFRA-26171
>>>  – there are dozens 
>>> more.
>>> 
>>> This technical difficulty is not only known to us. AFAIK, this is one of
>>> the main reasons PSF (Python Software Foundation) decided to switch from
>>> mailing lists to Discourse. Mailman documents several DMARC mitigations
>>> ,
>>> but I think these are workarounds/hacks rather than well-established
>>> solutions.
>>> 
>>> *Motivation #2: ezmlm is dead*
>>> 
>>> ezmlm, the mailing list software ASF uses, is dead – it is neither
>>> developed, nor maintained anymore. (Last official release was in 1997, 
>>> there
>>> is the `ezmlm-idx` add-on, which later on became a successor
>>> ,
>>> which last produced a release in 2014, and so on. Long, dead story.) 
>>> INFRA
>>> maintains a very big, sophisticated set of Perl rules for running ASF 
>>> ezmlm
>>> instances. If you look closely at the INFRA tickets I cited above, some
>>> suggest INFRA to fork ezmlm and fix some long standing bugs, etc. We can
>>> discuss the possibility of migrating from ezmlm to mailman (yet another
>>> mailing list software, but one that is still maintained), whether such a
>>> migration should be practiced ASF-wide or only for Logging Services, 
>>> etc.
>>> But eventually, we will still be using a mailing list

Re: Successor to mailing lists

2024-10-14 Thread Matt Sicker
There’s a very, very small chance I’ll ever remember to visit a website to find 
out about what are essentially emails that could have been sent to me. I have a 
regular habit of reading email nearly every day, but developing new habits is 
unlikely to stick.

> On Oct 14, 2024, at 14:50, Ralph Goers  wrote:
> 
> Maybe we just need to start contributing to PonyMail to improve the UI to 
> eliminate actually needing the email delivered to our accounts.
> 
> I am only 1/4 serious about this. There has to be a better solution.
> 
> Ralph
> 
>> On Oct 14, 2024, at 10:25 AM, Matt Sicker  wrote:
>> 
>> I didn’t get the original email in this thread once again, so I think I’d 
>> support trying somewhere else to host discussions. Besides archiving those 
>> messages into a mailing list, it would be great if the solution provided 
>> allowed for email interactivity (e.g., you can reply to the notification of 
>> a message and it’s added to the thread appropriately; this is how GitHub 
>> notification emails typically work).
>> 
>>> On Oct 10, 2024, at 05:40, Christian Grobmeier  wrote:
>>> 
>>> Hi
>>> 
>>> I am generally open to such experiments. I would start with the easiest 
>>> parts, such as users@, and see where it goes. 
>>> 
>>> I would advise against mirroring it to users@ behind the scenes, as it may 
>>> cause privacy problems (we need user consensus to mirror it). When a user 
>>> uses GitHub, they know what to expect.
>>> 
>>> As for Discourse, many use that now, but I find it very overwhelming and 
>>> stressful. I prefer the clean Github discussions approach.
>>> 
>>> I haven't checked against ASF policies but feel positive about this move 
>>> for the arguments you have given
>>> 
>>> Kind regards
>>> Christian
>>> 
>>> 
>>> 
>>> On Thu, Oct 10, 2024, at 10:58, Volkan Yazıcı wrote:
 *Abstract:* Modern email system security measures make it practically
 impossible for mailing lists to work – many subscribers don't get all
 emails. This not only hinders communication, but blocks an inclusive
 one. *Shall
 we, as Logging Services, experiment with alternatives?*
 
 *Motivation #1: mailing lists technically don't work*
 
 Several widely-used email providers (GMail, Yahoo, iCloud, etc.) have in
 the last couple of years enabled new measures (DMARC, SPF, DKIM, etc.) to
 verify the authenticity of emails. In short, these measures enrich email
 content with checksums and signatures capturing its authenticity. When a
 mailing list system (e.g., ezmlm, mailman) receives such an email, it
 performs several changes on its content (adds information about the mailing
 list, etc.), and delivers it to all subscribers. When the mail server of a
 subscriber receives such tampered mail, and if that mail server happens to
 have earlier shared authenticity checks enabled, it discards the email, or
 at best, marks it as spam.
 
 Ralph, Matt, Piotr stated many times that they don't receive all emails.
 Ralph actually stated many ASF mailing list emails end up in his spam 
 box
 .
 Recently we witnessed even Brian Proffitt (VP, Marketing & Publicity) 
 suffer
 from the same problem
 . 
 INFRA
 is crawling with related tickets: INFRA-24574
 , INFRA-24790
 , INFRA-24845
 , INFRA-24850
 , INFRA-24872
 , INFRA-25947
 , INFRA-26171
  – there are dozens 
 more.
 
 This technical difficulty is not only known to us. AFAIK, this is one of
 the main reasons PSF (Python Software Foundation) decided to switch from
 mailing lists to Discourse. Mailman documents several DMARC mitigations
 ,
 but I think these are workarounds/hacks rather than well-established
 solutions.
 
 *Motivation #2: ezmlm is dead*
 
 ezmlm, the mailing list software ASF uses, is dead – it is neither
 developed, nor maintained anymore. (Last official release was in 1997, 
 there
 is the `ezmlm-idx` add-on, which later on became a successor
 ,
 which last produced a release in 2014, and so on. Long, dead story.) 
 INFRA
 maintains a very big, sophisticated set of Perl rules for running ASF 
 ezmlm
 instances. If you look clo

Re: [log4j] Using `DateTimeFormatter` by default

2024-10-14 Thread Matt Sicker
I think it would make sense to switch to the standard Java one. The custom 
classes mostly predated Java 8, and I’ve seen enough test failures whenever 
daylight saving time starts or ends to know that they’re buggy.

> On Oct 14, 2024, at 14:30, Volkan Yazıcı  wrote:
> 
> *Abstract:* Log4j contains two custom date & time formatting classes. All
> our competitors (Logback
> ,
> Tinylog
> ,
> etc.) use Java's `DateTimeFormatter` and I know of no user complaints about
> their performance issues. Shall we switch to `DateTimeFormatter` as the
> default in the next minor release and make the custom implementations
> opt-in?
> 
> Log4j ships two custom date & time formatting classes:
> 
>   - `FixedDateFormat`
>  - Supports a hardcoded list of formats
>  - Bug: Conflates `n` directive with `S`
>  - Bug: It cannot calculate DST correctly for all time zones
>  
>   - `FastDateFormat`
>  - Copied from Commons Lang in 2015
> 
> I recently ran a test comparing the output of these with
> `DateTimeFormatter` – the difference is *big*! I can spend months fixing
> these issues, but... Do we really need them?
> 
>   1. No other competitors have such an optimization (both Logback
>   
> 
>and Tinylog
>   
> 
> use
>   `DateTimeFormatter`)
>   2. Date & time formatting is very fragile (DST is more voodoo than
>   science)
>   3. If date & time formatting during logging is found to be the
>   bottleneck of an application, shouldn't they instead encode it differently,
>   e.g., encoding epoch numbers?
>   4. `DateTimeFormatter` performance will only get better over time
> 
> Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
> the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
> 
> Note that the caching optimization we have for the last formatted timestamp
> will stay untouched.



Re: [log4j] Using `DateTimeFormatter` by default

2024-10-14 Thread Volkan Yazıcı
Some results using Java 17:


*-MM-dd'T'HH:mm:ss.SSS*DateTimeFormatter  2.844 ± 0.310  ops/ms
FastDateFormat 6.847 ± 1.302  ops/ms
FixedDateFormat   39.497 ± 6.423  ops/ms

*HH:mm:ss.SSS*
DateTimeFormatter  3.881 ± 0.447  ops/ms
FastDateFormat10.862 ± 1.915  ops/ms
FixedDateFormat   39.618 ± 9.318  ops/ms


I used distinct instants within the same day to avoid cache misses in
`FixedDateFormat`'s cache on the `-MM-dd` part, which is updated daily
at midnight. That is why `FixedDF` figures are more or less the same for
both patterns.

On Mon, Oct 14, 2024 at 9:48 PM Ralph Goers 
wrote:

> Before I can weigh in one way or another I would need to know what the
> performance difference is between DateTimeFormatter and the Fixed and Fast
> classes, at least where they are working properly. Any such test should
> include the caching otherwise you would be comparing worst case performance.
>
> Ralph
>
> > On Oct 14, 2024, at 12:30 PM, Volkan Yazıcı 
> wrote:
> >
> > *Abstract:* Log4j contains two custom date & time formatting classes. All
> > our competitors (Logback
> > <
> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
> >,
> > Tinylog
> > <
> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
> >,
> > etc.) use Java's `DateTimeFormatter` and I know of no user complaints
> about
> > their performance issues. Shall we switch to `DateTimeFormatter` as the
> > default in the next minor release and make the custom implementations
> > opt-in?
> >
> > Log4j ships two custom date & time formatting classes:
> >
> >   - `FixedDateFormat`
> >  - Supports a hardcoded list of formats
> >  - Bug: Conflates `n` directive with `S`
> >  - Bug: It cannot calculate DST correctly for all time zones
> >  
> >   - `FastDateFormat`
> >  - Copied from Commons Lang in 2015
> >
> > I recently ran a test comparing the output of these with
> > `DateTimeFormatter` – the difference is *big*! I can spend months fixing
> > these issues, but... Do we really need them?
> >
> >   1. No other competitors have such an optimization (both Logback
> >   <
> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
> >
> >and Tinylog
> >   <
> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
> >
> > use
> >   `DateTimeFormatter`)
> >   2. Date & time formatting is very fragile (DST is more voodoo than
> >   science)
> >   3. If date & time formatting during logging is found to be the
> >   bottleneck of an application, shouldn't they instead encode it
> differently,
> >   e.g., encoding epoch numbers?
> >   4. `DateTimeFormatter` performance will only get better over time
> >
> > Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
> > the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
> >
> > Note that the caching optimization we have for the last formatted
> timestamp
> > will stay untouched.
>
>


Re: [VOTE] Release Apache Log4cxx 1.3.0

2024-10-14 Thread Piotr P. Karwasz
Hi Stephen,

On Tue, 15 Oct 2024 at 06:33, Stephen Webb  wrote:
> Review kit: 
> https://github.com/apache/logging-log4cxx/admin/release-review-instructions.md

There is a small typo, the correct URL is:
https://github.com/apache/logging-log4cxx/blob/master/admin/release-review-instructions.md
For the `validate-release.sh` script:
https://github.com/apache/logging-log4cxx/blob/master/admin/validate-release.sh


[log4j] Using `DateTimeFormatter` by default

2024-10-14 Thread Volkan Yazıcı
*Abstract:* Log4j contains two custom date & time formatting classes. All
our competitors (Logback
,
Tinylog
,
etc.) use Java's `DateTimeFormatter` and I know of no user complaints about
their performance issues. Shall we switch to `DateTimeFormatter` as the
default in the next minor release and make the custom implementations
opt-in?

Log4j ships two custom date & time formatting classes:

   - `FixedDateFormat`
  - Supports a hardcoded list of formats
  - Bug: Conflates `n` directive with `S`
  - Bug: It cannot calculate DST correctly for all time zones
  
   - `FastDateFormat`
  - Copied from Commons Lang in 2015

I recently ran a test comparing the output of these with
`DateTimeFormatter` – the difference is *big*! I can spend months fixing
these issues, but... Do we really need them?

   1. No other competitors have such an optimization (both Logback
   

and Tinylog
   

use
   `DateTimeFormatter`)
   2. Date & time formatting is very fragile (DST is more voodoo than
   science)
   3. If date & time formatting during logging is found to be the
   bottleneck of an application, shouldn't they instead encode it differently,
   e.g., encoding epoch numbers?
   4. `DateTimeFormatter` performance will only get better over time

Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
the `log4j.time.legacyFormatterEnabled` property is provided. Objections?

Note that the caching optimization we have for the last formatted timestamp
will stay untouched.


Re: Successor to mailing lists

2024-10-14 Thread Ralph Goers
I can’t say I agree with that. It didn’t take me very long to get used to using 
Slack.

Ralph

> On Oct 14, 2024, at 1:47 PM, Matt Sicker  wrote:
> 
> There’s a very, very small chance I’ll ever remember to visit a website to 
> find out about what are essentially emails that could have been sent to me. I 
> have a regular habit of reading email nearly every day, but developing new 
> habits is unlikely to stick.
> 
>> On Oct 14, 2024, at 14:50, Ralph Goers  wrote:
>> 
>> Maybe we just need to start contributing to PonyMail to improve the UI to 
>> eliminate actually needing the email delivered to our accounts.
>> 
>> I am only 1/4 serious about this. There has to be a better solution.
>> 
>> Ralph
>> 
>>> On Oct 14, 2024, at 10:25 AM, Matt Sicker  wrote:
>>> 
>>> I didn’t get the original email in this thread once again, so I think I’d 
>>> support trying somewhere else to host discussions. Besides archiving those 
>>> messages into a mailing list, it would be great if the solution provided 
>>> allowed for email interactivity (e.g., you can reply to the notification of 
>>> a message and it’s added to the thread appropriately; this is how GitHub 
>>> notification emails typically work).
>>> 
 On Oct 10, 2024, at 05:40, Christian Grobmeier  
 wrote:
 
 Hi
 
 I am generally open to such experiments. I would start with the easiest 
 parts, such as users@, and see where it goes. 
 
 I would advise against mirroring it to users@ behind the scenes, as it may 
 cause privacy problems (we need user consensus to mirror it). When a user 
 uses GitHub, they know what to expect.
 
 As for Discourse, many use that now, but I find it very overwhelming and 
 stressful. I prefer the clean Github discussions approach.
 
 I haven't checked against ASF policies but feel positive about this move 
 for the arguments you have given
 
 Kind regards
 Christian
 
 
 
 On Thu, Oct 10, 2024, at 10:58, Volkan Yazıcı wrote:
> *Abstract:* Modern email system security measures make it practically
> impossible for mailing lists to work – many subscribers don't get all
> emails. This not only hinders communication, but blocks an inclusive
> one. *Shall
> we, as Logging Services, experiment with alternatives?*
> 
> *Motivation #1: mailing lists technically don't work*
> 
> Several widely-used email providers (GMail, Yahoo, iCloud, etc.) have in
> the last couple of years enabled new measures (DMARC, SPF, DKIM, etc.) to
> verify the authenticity of emails. In short, these measures enrich email
> content with checksums and signatures capturing its authenticity. When a
> mailing list system (e.g., ezmlm, mailman) receives such an email, it
> performs several changes on its content (adds information about the 
> mailing
> list, etc.), and delivers it to all subscribers. When the mail server of a
> subscriber receives such tampered mail, and if that mail server happens to
> have earlier shared authenticity checks enabled, it discards the email, or
> at best, marks it as spam.
> 
> Ralph, Matt, Piotr stated many times that they don't receive all emails.
> Ralph actually stated many ASF mailing list emails end up in his spam 
> box
> .
> Recently we witnessed even Brian Proffitt (VP, Marketing & Publicity) 
> suffer
> from the same problem
> . 
> INFRA
> is crawling with related tickets: INFRA-24574
> , INFRA-24790
> , INFRA-24845
> , INFRA-24850
> , INFRA-24872
> , INFRA-25947
> , INFRA-26171
>  – there are dozens 
> more.
> 
> This technical difficulty is not only known to us. AFAIK, this is one of
> the main reasons PSF (Python Software Foundation) decided to switch from
> mailing lists to Discourse. Mailman documents several DMARC mitigations
> ,
> but I think these are workarounds/hacks rather than well-established
> solutions.
> 
> *Motivation #2: ezmlm is dead*
> 
> ezmlm, the mailing list software ASF uses, is dead – it is neither
> developed, nor maintained anymore. (Last official release was in 1997, 
> there
> is the `ezmlm-idx` add-on, which later on became a successor
> 

Re: Successor to mailing lists

2024-10-14 Thread Matt Sicker
That’s because a lot of other things are also using Slack. On the other hand, I 
had to disable notifications from Slack due to people misusing it to DM me 
instead of sending emails to the Secretary properly (unrelated to Log4j).

> On Oct 14, 2024, at 15:53, Ralph Goers  wrote:
> 
> I can’t say I agree with that. It didn’t take me very long to get used to 
> using Slack.
> 
> Ralph
> 
>> On Oct 14, 2024, at 1:47 PM, Matt Sicker  wrote:
>> 
>> There’s a very, very small chance I’ll ever remember to visit a website to 
>> find out about what are essentially emails that could have been sent to me. 
>> I have a regular habit of reading email nearly every day, but developing new 
>> habits is unlikely to stick.
>> 
>>> On Oct 14, 2024, at 14:50, Ralph Goers  wrote:
>>> 
>>> Maybe we just need to start contributing to PonyMail to improve the UI to 
>>> eliminate actually needing the email delivered to our accounts.
>>> 
>>> I am only 1/4 serious about this. There has to be a better solution.
>>> 
>>> Ralph
>>> 
 On Oct 14, 2024, at 10:25 AM, Matt Sicker  wrote:
 
 I didn’t get the original email in this thread once again, so I think I’d 
 support trying somewhere else to host discussions. Besides archiving those 
 messages into a mailing list, it would be great if the solution provided 
 allowed for email interactivity (e.g., you can reply to the notification 
 of a message and it’s added to the thread appropriately; this is how 
 GitHub notification emails typically work).
 
> On Oct 10, 2024, at 05:40, Christian Grobmeier  
> wrote:
> 
> Hi
> 
> I am generally open to such experiments. I would start with the easiest 
> parts, such as users@, and see where it goes. 
> 
> I would advise against mirroring it to users@ behind the scenes, as it 
> may cause privacy problems (we need user consensus to mirror it). When a 
> user uses GitHub, they know what to expect.
> 
> As for Discourse, many use that now, but I find it very overwhelming and 
> stressful. I prefer the clean Github discussions approach.
> 
> I haven't checked against ASF policies but feel positive about this move 
> for the arguments you have given
> 
> Kind regards
> Christian
> 
> 
> 
> On Thu, Oct 10, 2024, at 10:58, Volkan Yazıcı wrote:
>> *Abstract:* Modern email system security measures make it practically
>> impossible for mailing lists to work – many subscribers don't get all
>> emails. This not only hinders communication, but blocks an inclusive
>> one. *Shall
>> we, as Logging Services, experiment with alternatives?*
>> 
>> *Motivation #1: mailing lists technically don't work*
>> 
>> Several widely-used email providers (GMail, Yahoo, iCloud, etc.) have in
>> the last couple of years enabled new measures (DMARC, SPF, DKIM, etc.) to
>> verify the authenticity of emails. In short, these measures enrich email
>> content with checksums and signatures capturing its authenticity. When a
>> mailing list system (e.g., ezmlm, mailman) receives such an email, it
>> performs several changes on its content (adds information about the 
>> mailing
>> list, etc.), and delivers it to all subscribers. When the mail server of 
>> a
>> subscriber receives such tampered mail, and if that mail server happens 
>> to
>> have earlier shared authenticity checks enabled, it discards the email, 
>> or
>> at best, marks it as spam.
>> 
>> Ralph, Matt, Piotr stated many times that they don't receive all emails.
>> Ralph actually stated many ASF mailing list emails end up in his spam 
>> box
>> .
>> Recently we witnessed even Brian Proffitt (VP, Marketing & Publicity) 
>> suffer
>> from the same problem
>> . 
>> INFRA
>> is crawling with related tickets: INFRA-24574
>> , INFRA-24790
>> , INFRA-24845
>> , INFRA-24850
>> , INFRA-24872
>> , INFRA-25947
>> , INFRA-26171
>>  – there are dozens 
>> more.
>> 
>> This technical difficulty is not only known to us. AFAIK, this is one of
>> the main reasons PSF (Python Software Foundation) decided to switch from
>> mailing lists to Discourse. Mailman documents several DMARC mitigations
>> ,
>> b

Re: [log4j] Using `DateTimeFormatter` by default

2024-10-14 Thread Ralph Goers
Wow.  I didn’t expect them to be that much faster.  That makes me very 
reluctant to be ok with switching.

Ralph



> On Oct 14, 2024, at 1:51 PM, Volkan Yazıcı  wrote:
> 
> Some results using Java 17:
> 
> 
> *-MM-dd'T'HH:mm:ss.SSS*DateTimeFormatter  2.844 ± 0.310  ops/ms
> FastDateFormat 6.847 ± 1.302  ops/ms
> FixedDateFormat   39.497 ± 6.423  ops/ms
> 
> *HH:mm:ss.SSS*
> DateTimeFormatter  3.881 ± 0.447  ops/ms
> FastDateFormat10.862 ± 1.915  ops/ms
> FixedDateFormat   39.618 ± 9.318  ops/ms
> 
> 
> I used distinct instants within the same day to avoid cache misses in
> `FixedDateFormat`'s cache on the `-MM-dd` part, which is updated daily
> at midnight. That is why `FixedDF` figures are more or less the same for
> both patterns.
> 
> On Mon, Oct 14, 2024 at 9:48 PM Ralph Goers 
> wrote:
> 
>> Before I can weigh in one way or another I would need to know what the
>> performance difference is between DateTimeFormatter and the Fixed and Fast
>> classes, at least where they are working properly. Any such test should
>> include the caching otherwise you would be comparing worst case performance.
>> 
>> Ralph
>> 
>>> On Oct 14, 2024, at 12:30 PM, Volkan Yazıcı 
>> wrote:
>>> 
>>> *Abstract:* Log4j contains two custom date & time formatting classes. All
>>> our competitors (Logback
>>> <
>> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
>>> ,
>>> Tinylog
>>> <
>> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
>>> ,
>>> etc.) use Java's `DateTimeFormatter` and I know of no user complaints
>> about
>>> their performance issues. Shall we switch to `DateTimeFormatter` as the
>>> default in the next minor release and make the custom implementations
>>> opt-in?
>>> 
>>> Log4j ships two custom date & time formatting classes:
>>> 
>>>  - `FixedDateFormat`
>>> - Supports a hardcoded list of formats
>>> - Bug: Conflates `n` directive with `S`
>>> - Bug: It cannot calculate DST correctly for all time zones
>>> 
>>>  - `FastDateFormat`
>>> - Copied from Commons Lang in 2015
>>> 
>>> I recently ran a test comparing the output of these with
>>> `DateTimeFormatter` – the difference is *big*! I can spend months fixing
>>> these issues, but... Do we really need them?
>>> 
>>>  1. No other competitors have such an optimization (both Logback
>>>  <
>> https://github.com/qos-ch/logback/blob/master/logback-core/src/main/java/ch/qos/logback/core/util/CachingDateFormatter.java
>>> 
>>>   and Tinylog
>>>  <
>> https://github.com/tinylog-org/tinylog/blob/7590ad150b1523691ae6f26ae24a291b81d804c9/tinylog-api/src/main/java/org/tinylog/runtime/PreciseTimestampFormatter.java
>>> 
>>> use
>>>  `DateTimeFormatter`)
>>>  2. Date & time formatting is very fragile (DST is more voodoo than
>>>  science)
>>>  3. If date & time formatting during logging is found to be the
>>>  bottleneck of an application, shouldn't they instead encode it
>> differently,
>>>  e.g., encoding epoch numbers?
>>>  4. `DateTimeFormatter` performance will only get better over time
>>> 
>>> Hence, I propose switching all layouts to use `DateTimeFormatter`, unless
>>> the `log4j.time.legacyFormatterEnabled` property is provided. Objections?
>>> 
>>> Note that the caching optimization we have for the last formatted
>> timestamp
>>> will stay untouched.
>> 
>> 



Re: Successor to mailing lists

2024-10-14 Thread Xeno Amess
python...sigh.


发件人: Gary Gregory 
发送时间: 星期二, 十月 15, 2024 3:59:10 上午
收件人: Apache Logging Developers List 
主题: Re: Successor to mailing lists

I'll admit that I rely on PonyMail. It seems we (Apache) should have a
professional (paid for is ok) set up for this, not a custom solution.

Gary

On Mon, Oct 14, 2024, 3:51 PM Ralph Goers 
wrote:

> Maybe we just need to start contributing to PonyMail to improve the UI to
> eliminate actually needing the email delivered to our accounts.
>
> I am only 1/4 serious about this. There has to be a better solution.
>
> Ralph
>
> > On Oct 14, 2024, at 10:25 AM, Matt Sicker  wrote:
> >
> > I didn’t get the original email in this thread once again, so I think
> I’d support trying somewhere else to host discussions. Besides archiving
> those messages into a mailing list, it would be great if the solution
> provided allowed for email interactivity (e.g., you can reply to the
> notification of a message and it’s added to the thread appropriately; this
> is how GitHub notification emails typically work).
> >
> >> On Oct 10, 2024, at 05:40, Christian Grobmeier 
> wrote:
> >>
> >> Hi
> >>
> >> I am generally open to such experiments. I would start with the easiest
> parts, such as users@, and see where it goes.
> >>
> >> I would advise against mirroring it to users@ behind the scenes, as it
> may cause privacy problems (we need user consensus to mirror it). When a
> user uses GitHub, they know what to expect.
> >>
> >> As for Discourse, many use that now, but I find it very overwhelming
> and stressful. I prefer the clean Github discussions approach.
> >>
> >> I haven't checked against ASF policies but feel positive about this
> move for the arguments you have given
> >>
> >> Kind regards
> >> Christian
> >>
> >>
> >>
> >> On Thu, Oct 10, 2024, at 10:58, Volkan Yazıcı wrote:
> >>> *Abstract:* Modern email system security measures make it practically
> >>> impossible for mailing lists to work – many subscribers don't get all
> >>> emails. This not only hinders communication, but blocks an inclusive
> >>> one. *Shall
> >>> we, as Logging Services, experiment with alternatives?*
> >>>
> >>> *Motivation #1: mailing lists technically don't work*
> >>>
> >>> Several widely-used email providers (GMail, Yahoo, iCloud, etc.) have
> in
> >>> the last couple of years enabled new measures (DMARC, SPF, DKIM, etc.)
> to
> >>> verify the authenticity of emails. In short, these measures enrich
> email
> >>> content with checksums and signatures capturing its authenticity. When
> a
> >>> mailing list system (e.g., ezmlm, mailman) receives such an email, it
> >>> performs several changes on its content (adds information about the
> mailing
> >>> list, etc.), and delivers it to all subscribers. When the mail server
> of a
> >>> subscriber receives such tampered mail, and if that mail server
> happens to
> >>> have earlier shared authenticity checks enabled, it discards the
> email, or
> >>> at best, marks it as spam.
> >>>
> >>> Ralph, Matt, Piotr stated many times that they don't receive all
> emails.
> >>> Ralph actually stated many ASF mailing list emails end up in his spam
> >>> box
> >>> <
> https://the-asf.slack.com/archives/CBX4TSBQ8/p1728032221080189?thread_ts=1727958807.348019&cid=CBX4TSBQ8
> >.
> >>> Recently we witnessed even Brian Proffitt (VP, Marketing & Publicity)
> >>> suffer
> >>> from the same problem
> >>> .
> >>> INFRA
> >>> is crawling with related tickets: INFRA-24574
> >>> , INFRA-24790
> >>> , INFRA-24845
> >>> , INFRA-24850
> >>> , INFRA-24872
> >>> , INFRA-25947
> >>> , INFRA-26171
> >>>  – there are
> dozens
> >>> more.
> >>>
> >>> This technical difficulty is not only known to us. AFAIK, this is one
> of
> >>> the main reasons PSF (Python Software Foundation) decided to switch
> from
> >>> mailing lists to Discourse. Mailman documents several DMARC mitigations
> >>> <
> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/handlers/docs/dmarc-mitigations.html
> >,
> >>> but I think these are workarounds/hacks rather than well-established
> >>> solutions.
> >>>
> >>> *Motivation #2: ezmlm is dead*
> >>>
> >>> ezmlm, the mailing list software ASF uses, is dead – it is neither
> >>> developed, nor maintained anymore. (Last official release was in 1997,
> >>> there
> >>> is the `ezmlm-idx` add-on, which later on became a successor
> >>> <
> https://untroubled.org/ezmlm/faq/What-is-the-difference-between-ezmlm-and-ezmlm_002didx_003f.html
> >,
> >>> which last produced a release in 2014, and so on. Long, dead story.)
> 

[VOTE] Release Apache Log4cxx 1.3.0

2024-10-14 Thread Stephen Webb
This is a vote to release the Apache Log4cxx 1.3.0.

Website: https://logging.staged.apache.org/log4cxx/1.3.0/changelog.html
GitHub: https://github.com/apache/logging-log4cxx
Commit: 1370eee2c19bdc287b7aa98892af02fd136694e7
Distribution: https://dist.apache.org/repos/dist/dev/logging/log4cxx/
Signing key: C6FC425C96722EFC4E2D02D5D4305EBC16B4A78D
Review kit: 
https://github.com/apache/logging-log4cxx/admin/release-review-instructions.md

Please download, test, and cast your votes on this mailing list.

[ ] +1, release the artifacts
[ ] -1, don't release, because ...

This vote is open for 72 hours and will pass unless getting a net
negative vote count.
All votes are welcome and we encourage everyone to test the release,
but only the
Logging Services PMC votes are officially counted.


Re: Successor to mailing lists

2024-10-14 Thread Gary Gregory
I'll admit that I rely on PonyMail. It seems we (Apache) should have a
professional (paid for is ok) set up for this, not a custom solution.

Gary

On Mon, Oct 14, 2024, 3:51 PM Ralph Goers 
wrote:

> Maybe we just need to start contributing to PonyMail to improve the UI to
> eliminate actually needing the email delivered to our accounts.
>
> I am only 1/4 serious about this. There has to be a better solution.
>
> Ralph
>
> > On Oct 14, 2024, at 10:25 AM, Matt Sicker  wrote:
> >
> > I didn’t get the original email in this thread once again, so I think
> I’d support trying somewhere else to host discussions. Besides archiving
> those messages into a mailing list, it would be great if the solution
> provided allowed for email interactivity (e.g., you can reply to the
> notification of a message and it’s added to the thread appropriately; this
> is how GitHub notification emails typically work).
> >
> >> On Oct 10, 2024, at 05:40, Christian Grobmeier 
> wrote:
> >>
> >> Hi
> >>
> >> I am generally open to such experiments. I would start with the easiest
> parts, such as users@, and see where it goes.
> >>
> >> I would advise against mirroring it to users@ behind the scenes, as it
> may cause privacy problems (we need user consensus to mirror it). When a
> user uses GitHub, they know what to expect.
> >>
> >> As for Discourse, many use that now, but I find it very overwhelming
> and stressful. I prefer the clean Github discussions approach.
> >>
> >> I haven't checked against ASF policies but feel positive about this
> move for the arguments you have given
> >>
> >> Kind regards
> >> Christian
> >>
> >>
> >>
> >> On Thu, Oct 10, 2024, at 10:58, Volkan Yazıcı wrote:
> >>> *Abstract:* Modern email system security measures make it practically
> >>> impossible for mailing lists to work – many subscribers don't get all
> >>> emails. This not only hinders communication, but blocks an inclusive
> >>> one. *Shall
> >>> we, as Logging Services, experiment with alternatives?*
> >>>
> >>> *Motivation #1: mailing lists technically don't work*
> >>>
> >>> Several widely-used email providers (GMail, Yahoo, iCloud, etc.) have
> in
> >>> the last couple of years enabled new measures (DMARC, SPF, DKIM, etc.)
> to
> >>> verify the authenticity of emails. In short, these measures enrich
> email
> >>> content with checksums and signatures capturing its authenticity. When
> a
> >>> mailing list system (e.g., ezmlm, mailman) receives such an email, it
> >>> performs several changes on its content (adds information about the
> mailing
> >>> list, etc.), and delivers it to all subscribers. When the mail server
> of a
> >>> subscriber receives such tampered mail, and if that mail server
> happens to
> >>> have earlier shared authenticity checks enabled, it discards the
> email, or
> >>> at best, marks it as spam.
> >>>
> >>> Ralph, Matt, Piotr stated many times that they don't receive all
> emails.
> >>> Ralph actually stated many ASF mailing list emails end up in his spam
> >>> box
> >>> <
> https://the-asf.slack.com/archives/CBX4TSBQ8/p1728032221080189?thread_ts=1727958807.348019&cid=CBX4TSBQ8
> >.
> >>> Recently we witnessed even Brian Proffitt (VP, Marketing & Publicity)
> >>> suffer
> >>> from the same problem
> >>> .
> >>> INFRA
> >>> is crawling with related tickets: INFRA-24574
> >>> , INFRA-24790
> >>> , INFRA-24845
> >>> , INFRA-24850
> >>> , INFRA-24872
> >>> , INFRA-25947
> >>> , INFRA-26171
> >>>  – there are
> dozens
> >>> more.
> >>>
> >>> This technical difficulty is not only known to us. AFAIK, this is one
> of
> >>> the main reasons PSF (Python Software Foundation) decided to switch
> from
> >>> mailing lists to Discourse. Mailman documents several DMARC mitigations
> >>> <
> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/handlers/docs/dmarc-mitigations.html
> >,
> >>> but I think these are workarounds/hacks rather than well-established
> >>> solutions.
> >>>
> >>> *Motivation #2: ezmlm is dead*
> >>>
> >>> ezmlm, the mailing list software ASF uses, is dead – it is neither
> >>> developed, nor maintained anymore. (Last official release was in 1997,
> >>> there
> >>> is the `ezmlm-idx` add-on, which later on became a successor
> >>> <
> https://untroubled.org/ezmlm/faq/What-is-the-difference-between-ezmlm-and-ezmlm_002didx_003f.html
> >,
> >>> which last produced a release in 2014, and so on. Long, dead story.)
> >>> INFRA
> >>> maintains a very big, sophisticated set of Perl rules for running ASF
> >>> ezmlm
> >>> instances. If you look closely at the INFRA tickets I cited above, some

Re: Subpages of logging.staged.apache.org report error 404

2024-10-14 Thread Stephen Webb
Chris advises that all the subpage directory were empty.  We will have to
push whitespace commits to all the sub repos that publish content.

On Fri, Oct 11, 2024 at 5:56 PM Volkan Yazıcı 
wrote:

> There is an INFRA issue  >,
> we're on it.
>
> On Fri, Oct 11, 2024 at 4:59 AM Stephen Webb  wrote:
>
> > Is anyone able to shed light on what is going on?
> >
> > Thanks
> > Stephen Webb
> >
>


Re: Disable JIRA issue creation

2024-10-14 Thread Jan Friedrich
+1

On 14 October 2024 12:12:10 CEST, "Piotr P. Karwasz"  
wrote:
>Hi all,
>
>What do you think about disabling the creation of new JIRA issues for
>LOG4J2? I confirmed with INFRA that this is technically possible.
>Disabling new issues would prevent JIRA users from filing new reports
>against LOG4J2. Personally I look at JIRA sporadically and only to
>look for references to old issues. I don't expect any new issues, so
>these might be left without any answer for quite some time.
>
>Piotr
>
>BTW: I use a shared filter[1] on JIRA, that filters out problems
>without any activity in the past 2 years. This currently gives a total
>of 178 open issues (113 on GH and 65 on JIRA). Most of them are
>unlikely to be fixed anyway.
>
>[1] https://issues.apache.org/jira/issues/?filter=12351399

Jan

Re: Disable JIRA issue creation

2024-10-14 Thread Volkan Yazıcı
+1

On Mon, Oct 14, 2024 at 12:12 PM Piotr P. Karwasz 
wrote:

> Hi all,
>
> What do you think about disabling the creation of new JIRA issues for
> LOG4J2? I confirmed with INFRA that this is technically possible.
> Disabling new issues would prevent JIRA users from filing new reports
> against LOG4J2. Personally I look at JIRA sporadically and only to
> look for references to old issues. I don't expect any new issues, so
> these might be left without any answer for quite some time.
>
> Piotr
>
> BTW: I use a shared filter[1] on JIRA, that filters out problems
> without any activity in the past 2 years. This currently gives a total
> of 178 open issues (113 on GH and 65 on JIRA). Most of them are
> unlikely to be fixed anyway.
>
> [1] https://issues.apache.org/jira/issues/?filter=12351399
>


Disable JIRA issue creation

2024-10-14 Thread Piotr P. Karwasz
Hi all,

What do you think about disabling the creation of new JIRA issues for
LOG4J2? I confirmed with INFRA that this is technically possible.
Disabling new issues would prevent JIRA users from filing new reports
against LOG4J2. Personally I look at JIRA sporadically and only to
look for references to old issues. I don't expect any new issues, so
these might be left without any answer for quite some time.

Piotr

BTW: I use a shared filter[1] on JIRA, that filters out problems
without any activity in the past 2 years. This currently gives a total
of 178 open issues (113 on GH and 65 on JIRA). Most of them are
unlikely to be fixed anyway.

[1] https://issues.apache.org/jira/issues/?filter=12351399


Re: Disable JIRA issue creation

2024-10-14 Thread Davyd McColl

+1

On 2024-10-14 12:12, Piotr P. Karwasz wrote:

Hi all,

What do you think about disabling the creation of new JIRA issues for
LOG4J2? I confirmed with INFRA that this is technically possible.
Disabling new issues would prevent JIRA users from filing new reports
against LOG4J2. Personally I look at JIRA sporadically and only to
look for references to old issues. I don't expect any new issues, so
these might be left without any answer for quite some time.

Piotr

BTW: I use a shared filter[1] on JIRA, that filters out problems
without any activity in the past 2 years. This currently gives a total
of 178 open issues (113 on GH and 65 on JIRA). Most of them are
unlikely to be fixed anyway.

[1] https://issues.apache.org/jira/issues/?filter=12351399


Unified release procedure in Apache Logging Services projects

2024-10-14 Thread Piotr P. Karwasz
Hi all,

Recently on `security-discuss@community` Christopher Schultz posted a
list[0] of security-related aspects that could be improved in Apache
projects.
After looking at the new Log4cxx release[1] and release review[2]
instructions, that I tried to automatise in PR #414[3], I am wondering
if we can come up with a release and release review procedure that is,
as much as possible, common to all projects.
The advantage of such a procedure would be to relieve PMC members from
remembering (or reading) the steps required to verify a release.

Some ideas for a brainstorming session:

### Staging a release

The Java projects have the most automatised release process:

1. It is started by a push to a `release/` branch.
2. (Java specific) It publishes artifacts to Nexus.
3. It creates all the archives we need for `downloads.apache.org`,
computes their hashes and signatures.
4. It sends those archives to Subversion.

I think that this process could be generalized to all projects. We
only need to replace the Java-specific part with something specific to
each ecosystem. Do you agree?

The current implementation of point 3 is Maven specific: we use a
Groovy script started by a Maven plugin to package the files. Could we
perhaps replace this with something more universal? A shell script
should probably be enough. I have a prototype in [4], which only needs
some massaging to work on MacOS and Windows and some generalization.

### Reviewing a release

The release instructions for Java releases[5] start with some common
steps. We probably agree that the verification of each Logging
Services release should start with:

1. Downloading the artifacts in `downloads.apache.org`.
2. Verifying the hashes.
3. Verifying the signatures.

After these steps the current Java verification procedure directly
starts a build using the downloaded sources. IMHO there is a step
missing (which is crucial for Log4cxx and Log4NET): the integrity and
reproducibility of the source archive should be verified first! Since
the _de facto_ source archive is the Git repository, we need to:

4. Make a (shallow) clone of the appropriate Git commit.
5. Package the source archive. For this step we can reuse the same
script as in "Staging a release"
6. Verify the source archives using the hashes from Subversion.

Currently we justify the absence of these steps by the reproducibility
guarantees of our Java build. We should however consider that not all
PMC members run the reproducibility checks, which are basically only
guaranteed to work on Linux.

### Voting a release

A long dream of mine was to record the names of PMC members that voted
+1 on a release in a cryptographically verifiable way. We could adopt
an (optional) procedure, where PMC members:

1. Reproduce all the artifacts that will be published to `downloads.apache.org`.
2. Sign the local artifacts (not those downloaded from Subversion).
3. Append the signatures to those in Subversion.
4. If the signatures are valid (which not only guarantees that the
signer is who he claims to be, but also that the artifacts are
reproducible) push the commit to Subversion.

What do you think?

Piotr

[0] https://lists.apache.org/thread/4w27qvdxdc30mnt72sj98dhjmz2qq3z3
[1] https://github.com/apache/logging-log4cxx/blob/master/admin/releasing.md
[2] 
https://github.com/apache/logging-log4cxx/blob/master/admin/release-review-instructions.md
[3] https://github.com/apache/logging-log4cxx/pull/414
[4] 
https://github.com/apache/logging-log4cxx/blob/b1dfdf25289db7926db803eaaaf99607ee9e8829/package.sh
[5] 
https://logging.apache.org/logging-parent/release-review-instructions.html#verify


Re: Successor to mailing lists

2024-10-14 Thread Matt Sicker
I didn’t get the original email in this thread once again, so I think I’d 
support trying somewhere else to host discussions. Besides archiving those 
messages into a mailing list, it would be great if the solution provided 
allowed for email interactivity (e.g., you can reply to the notification of a 
message and it’s added to the thread appropriately; this is how GitHub 
notification emails typically work).

> On Oct 10, 2024, at 05:40, Christian Grobmeier  wrote:
> 
> Hi
> 
> I am generally open to such experiments. I would start with the easiest 
> parts, such as users@, and see where it goes. 
> 
> I would advise against mirroring it to users@ behind the scenes, as it may 
> cause privacy problems (we need user consensus to mirror it). When a user 
> uses GitHub, they know what to expect.
> 
> As for Discourse, many use that now, but I find it very overwhelming and 
> stressful. I prefer the clean Github discussions approach.
> 
> I haven't checked against ASF policies but feel positive about this move for 
> the arguments you have given
> 
> Kind regards
> Christian
> 
> 
> 
> On Thu, Oct 10, 2024, at 10:58, Volkan Yazıcı wrote:
>> *Abstract:* Modern email system security measures make it practically
>> impossible for mailing lists to work – many subscribers don't get all
>> emails. This not only hinders communication, but blocks an inclusive
>> one. *Shall
>> we, as Logging Services, experiment with alternatives?*
>> 
>> *Motivation #1: mailing lists technically don't work*
>> 
>> Several widely-used email providers (GMail, Yahoo, iCloud, etc.) have in
>> the last couple of years enabled new measures (DMARC, SPF, DKIM, etc.) to
>> verify the authenticity of emails. In short, these measures enrich email
>> content with checksums and signatures capturing its authenticity. When a
>> mailing list system (e.g., ezmlm, mailman) receives such an email, it
>> performs several changes on its content (adds information about the mailing
>> list, etc.), and delivers it to all subscribers. When the mail server of a
>> subscriber receives such tampered mail, and if that mail server happens to
>> have earlier shared authenticity checks enabled, it discards the email, or
>> at best, marks it as spam.
>> 
>> Ralph, Matt, Piotr stated many times that they don't receive all emails.
>> Ralph actually stated many ASF mailing list emails end up in his spam 
>> box
>> .
>> Recently we witnessed even Brian Proffitt (VP, Marketing & Publicity) 
>> suffer
>> from the same problem
>> . 
>> INFRA
>> is crawling with related tickets: INFRA-24574
>> , INFRA-24790
>> , INFRA-24845
>> , INFRA-24850
>> , INFRA-24872
>> , INFRA-25947
>> , INFRA-26171
>>  – there are dozens 
>> more.
>> 
>> This technical difficulty is not only known to us. AFAIK, this is one of
>> the main reasons PSF (Python Software Foundation) decided to switch from
>> mailing lists to Discourse. Mailman documents several DMARC mitigations
>> ,
>> but I think these are workarounds/hacks rather than well-established
>> solutions.
>> 
>> *Motivation #2: ezmlm is dead*
>> 
>> ezmlm, the mailing list software ASF uses, is dead – it is neither
>> developed, nor maintained anymore. (Last official release was in 1997, 
>> there
>> is the `ezmlm-idx` add-on, which later on became a successor
>> ,
>> which last produced a release in 2014, and so on. Long, dead story.) 
>> INFRA
>> maintains a very big, sophisticated set of Perl rules for running ASF 
>> ezmlm
>> instances. If you look closely at the INFRA tickets I cited above, some
>> suggest INFRA to fork ezmlm and fix some long standing bugs, etc. We can
>> discuss the possibility of migrating from ezmlm to mailman (yet another
>> mailing list software, but one that is still maintained), whether such a
>> migration should be practiced ASF-wide or only for Logging Services, 
>> etc.
>> But eventually, we will still be using a mailing list, and as I tried to
>> explain above, IMO, they just don't work good.
>> 
>> *Proposal #1: Experimenting with GitHub Discussions*
>> 
>> GitHub is our development bread and butter. We use its tickets, PRs,
>> reviews, discussions, CI, security & code quality checks, etc. It works
>> perfectly and components are integrated well, i.e., you can link issues,
>> comments, PR

Re: Unified release procedure in Apache Logging Services projects

2024-10-14 Thread Volkan Yazıcı
-1

   - In "Staging a release", -1 for replacing a platform-independent
   `pom.xml` snippet with OS-specific shell scripts
   - In "Reviewing a release", you say "Since the de facto source archive
   is the Git repository". On the contrary, as much as Legal is concerned, de
   facto source archive is the one distributed with SVN.
   - In "Voting a release", -1 for everything related with signatures. It
   introduces complexity without any _practical_ benefit.

There is the Artifact Distribution Platform project
 of INFRA
– see its Slack channel .
I think it is a better place to improve the release process, ASF-wide.

On Mon, Oct 14, 2024 at 4:31 PM Piotr P. Karwasz 
wrote:

> Hi all,
>
> Recently on `security-discuss@community` Christopher Schultz posted a
> list[0] of security-related aspects that could be improved in Apache
> projects.
> After looking at the new Log4cxx release[1] and release review[2]
> instructions, that I tried to automatise in PR #414[3], I am wondering
> if we can come up with a release and release review procedure that is,
> as much as possible, common to all projects.
> The advantage of such a procedure would be to relieve PMC members from
> remembering (or reading) the steps required to verify a release.
>
> Some ideas for a brainstorming session:
>
> ### Staging a release
>
> The Java projects have the most automatised release process:
>
> 1. It is started by a push to a `release/` branch.
> 2. (Java specific) It publishes artifacts to Nexus.
> 3. It creates all the archives we need for `downloads.apache.org`,
> computes their hashes and signatures.
> 4. It sends those archives to Subversion.
>
> I think that this process could be generalized to all projects. We
> only need to replace the Java-specific part with something specific to
> each ecosystem. Do you agree?
>
> The current implementation of point 3 is Maven specific: we use a
> Groovy script started by a Maven plugin to package the files. Could we
> perhaps replace this with something more universal? A shell script
> should probably be enough. I have a prototype in [4], which only needs
> some massaging to work on MacOS and Windows and some generalization.
>
> ### Reviewing a release
>
> The release instructions for Java releases[5] start with some common
> steps. We probably agree that the verification of each Logging
> Services release should start with:
>
> 1. Downloading the artifacts in `downloads.apache.org`.
> 2. Verifying the hashes.
> 3. Verifying the signatures.
>
> After these steps the current Java verification procedure directly
> starts a build using the downloaded sources. IMHO there is a step
> missing (which is crucial for Log4cxx and Log4NET): the integrity and
> reproducibility of the source archive should be verified first! Since
> the _de facto_ source archive is the Git repository, we need to:
>
> 4. Make a (shallow) clone of the appropriate Git commit.
> 5. Package the source archive. For this step we can reuse the same
> script as in "Staging a release"
> 6. Verify the source archives using the hashes from Subversion.
>
> Currently we justify the absence of these steps by the reproducibility
> guarantees of our Java build. We should however consider that not all
> PMC members run the reproducibility checks, which are basically only
> guaranteed to work on Linux.
>
> ### Voting a release
>
> A long dream of mine was to record the names of PMC members that voted
> +1 on a release in a cryptographically verifiable way. We could adopt
> an (optional) procedure, where PMC members:
>
> 1. Reproduce all the artifacts that will be published to `
> downloads.apache.org`.
> 2. Sign the local artifacts (not those downloaded from Subversion).
> 3. Append the signatures to those in Subversion.
> 4. If the signatures are valid (which not only guarantees that the
> signer is who he claims to be, but also that the artifacts are
> reproducible) push the commit to Subversion.
>
> What do you think?
>
> Piotr
>
> [0] https://lists.apache.org/thread/4w27qvdxdc30mnt72sj98dhjmz2qq3z3
> [1]
> https://github.com/apache/logging-log4cxx/blob/master/admin/releasing.md
> [2]
> https://github.com/apache/logging-log4cxx/blob/master/admin/release-review-instructions.md
> [3] https://github.com/apache/logging-log4cxx/pull/414
> [4]
> https://github.com/apache/logging-log4cxx/blob/b1dfdf25289db7926db803eaaaf99607ee9e8829/package.sh
> [5]
> https://logging.apache.org/logging-parent/release-review-instructions.html#verify
>