Re: Reproducible builds update

2022-01-26 Thread Raymond Augé
Hey Mark, bnd is in ramp down phase targetting a release in Feb so if you do find an issue soon-ish we can work to get it in the release. Ray On Wed, Jan 26, 2022 at 4:05 AM Mark Thomas wrote: > I have made some progress on this over the last few days. The current > status is: > > - Builds are

Re: Reproducible builds update

2022-01-26 Thread Mark Thomas
I have made some progress on this over the last few days. The current status is: - Builds are reproducible (excluding signing of Windows binaries) when using the same OS / Java / Ant combination. - JSign gives us what we need to handling the signing of the Windows binaries. "Just" need to i

Re: Reproducible builds update

2022-01-24 Thread Mark Thomas
On 24/01/2022 11:50, Rainer Jung wrote: I had a little nag about ant.tstamp.now.iso mailed on January 20th. Thanks. Fixed. I switched from ant.tstamp.now.iso to ant.tstamp.now shortly after adding the property but missed a couple of references in that switch. Should be OK now. Mark Cop

Re: Reproducible builds update

2022-01-24 Thread Rainer Jung
I had a little nag about ant.tstamp.now.iso mailed on January 20th. Copying here: Am 15.01.2022 um 12:56 schrieb ma...@apache.org: > This is an automated email from the ASF dual-hosted git repository. > > markt pushed a commit to branch main > in repository https://gitbox.apache.org/repos/asf/to

Re: Reproducible builds update

2022-01-21 Thread Mark Thomas
On 21/01/2022 17:11, Christopher Schultz wrote: Maybe patch ant to work differently? I agree, that is the longer term fix. I'm not too familiar with the Ant code but I didn't see a simple way to fix it. At the moment the short-term fix of renaming or deleting stuff seems to be working. I

Re: Reproducible builds update

2022-01-21 Thread Christopher Schultz
Mark, On 1/20/22 17:56, Mark Thomas wrote: On 20/01/2022 11:02, Mark Thomas wrote: On 19/01/2022 16:56, Rémy Maucherat wrote: On Wed, Jan 19, 2022 at 5:52 PM Emmanuel Bourg wrote: Fortunately a non-reproducible javadoc isn't really important, what matters the most is to have reproducible exe

Re: Reproducible builds update

2022-01-21 Thread Rémy Maucherat
On Thu, Jan 20, 2022 at 11:57 PM Mark Thomas wrote: > > On 20/01/2022 11:02, Mark Thomas wrote: > > On 19/01/2022 16:56, Rémy Maucherat wrote: > >> On Wed, Jan 19, 2022 at 5:52 PM Emmanuel Bourg wrote: > >>> Fortunately a non-reproducible javadoc isn't really important, what > >>> matters the mos

Re: Reproducible builds update

2022-01-20 Thread Mark Thomas
On 20/01/2022 11:02, Mark Thomas wrote: On 19/01/2022 16:56, Rémy Maucherat wrote: On Wed, Jan 19, 2022 at 5:52 PM Emmanuel Bourg wrote: Fortunately a non-reproducible javadoc isn't really important, what matters the most is to have reproducible executable packages. That seems like a reasona

Re: Reproducible builds update

2022-01-20 Thread Mark Thomas
On 19/01/2022 16:56, Rémy Maucherat wrote: On Wed, Jan 19, 2022 at 5:52 PM Emmanuel Bourg wrote: Fortunately a non-reproducible javadoc isn't really important, what matters the most is to have reproducible executable packages. That seems like a reasonable statement ! Ack. I'll look at the J

Re: Reproducible builds update

2022-01-19 Thread Rémy Maucherat
On Wed, Jan 19, 2022 at 5:52 PM Emmanuel Bourg wrote: > Fortunately a non-reproducible javadoc isn't really important, what > matters the most is to have reproducible executable packages. That seems like a reasonable statement ! Rémy -

Re: Reproducible builds update

2022-01-19 Thread Emmanuel Bourg
Le 18/01/2022 à 21:03, Mark Thomas a écrit : I'm wondering about second issue. It would be nice to have a complete fix but if the full docs package isn't reproducible how much of an issue is that? Reproducibility issues with javadoc are quite frequent unfortunately, many Java packages in Deb

Re: Reproducible builds update

2022-01-18 Thread Mark Thomas
On 19/01/2022 07:47, Romain Manni-Bucau wrote: Hi Mark, It is possible to use jdk.javadoc/jdk.compiler as libs and plug in through the Context class forcing a JavaFileManager to force a DocFileFactory in the HtmlConfiguration but it will be way more complex than unzipping/rezipping the jar. Ni

Re: Reproducible builds update

2022-01-18 Thread Romain Manni-Bucau
Hi Mark, It is possible to use jdk.javadoc/jdk.compiler as libs and plug in through the Context class forcing a JavaFileManager to force a DocFileFactory in the HtmlConfiguration but it will be way more complex than unzipping/rezipping the jar. Another option can be a light javaagent patching the

Re: Reproducible builds update

2022-01-18 Thread Christopher Schultz
Mark, On 1/18/22 15:03, Mark Thomas wrote: On 18/01/2022 19:39, Mark Thomas wrote: The first issue looks relatively simple to fix. I don't see an easy fix for the second. My best idea so far is some sort of post-processing for the Javadoc generation that extracts the file from the zip, set

Re: Reproducible builds update

2022-01-18 Thread Mark Thomas
On 18/01/2022 19:39, Mark Thomas wrote: The first issue looks relatively simple to fix. I don't see an easy fix for the second. My best idea so far is some sort of post-processing for the Javadoc generation that extracts the file from the zip, sets the timestamp and then re-zips it. Suggesti

Re: Reproducible builds

2021-03-24 Thread Mark Thomas
On March 24, 2021 3:57:47 PM UTC, Christopher Schultz wrote: >Mark, > >On 3/23/21 09:45, Mark Thomas wrote: >> On 23/03/2021 13:15, Mark Thomas wrote: >> >> >> >>> I did a little more digging. There is an assumption in >JarInputStream >>> that /META-INF/MANIFEST.MF is either first or only pre

Re: Reproducible builds

2021-03-24 Thread Christopher Schultz
Mark, On 3/23/21 09:45, Mark Thomas wrote: On 23/03/2021 13:15, Mark Thomas wrote: I did a little more digging. There is an assumption in JarInputStream that /META-INF/MANIFEST.MF is either first or only preceded by /META-INF/ I suspect I'm going to need to revert the to changes. I'm cu

Re: Reproducible builds

2021-03-23 Thread Mark Thomas
On 23/03/2021 13:15, Mark Thomas wrote: I did a little more digging. There is an assumption in JarInputStream that /META-INF/MANIFEST.MF is either first or only preceded by /META-INF/ I suspect I'm going to need to revert the to changes. I'm currently looking into alternatives. Aha! Th

Re: Reproducible builds

2021-03-23 Thread Mark Thomas
On 22/03/2021 14:53, Konstantin Kolinko wrote: пн, 22 мар. 2021 г. в 17:32, Mark Thomas : On 22/03/2021 13:16, Konstantin Kolinko wrote: сб, 20 мар. 2021 г. в 18:24, Emmanuel Bourg : I got a quick look, I guess you replaced the tasks with to make the timestamp of the zip entries reproduci

Re: Reproducible builds

2021-03-22 Thread Konstantin Kolinko
пн, 22 мар. 2021 г. в 17:32, Mark Thomas : > > On 22/03/2021 13:16, Konstantin Kolinko wrote: > > сб, 20 мар. 2021 г. в 18:24, Emmanuel Bourg : > > >> I got a quick look, I guess you replaced the tasks with to > >> make the timestamp of the zip entries reproducible? I'm not sure this is > >> suff

Re: Reproducible builds

2021-03-22 Thread Mark Thomas
On 22/03/2021 13:16, Konstantin Kolinko wrote: сб, 20 мар. 2021 г. в 18:24, Emmanuel Bourg : 2. The current Windows exe signing process isn't repeatable. There are a few suggestions workarounds at https://reproducible-builds.org/ and I need to discuss these with the provider of the code signing

Re: Reproducible builds

2021-03-22 Thread Konstantin Kolinko
сб, 20 мар. 2021 г. в 18:24, Emmanuel Bourg : > > 2. The current Windows exe signing process isn't repeatable. There are a > > few suggestions workarounds at https://reproducible-builds.org/ and I > > need to discuss these with the provider of the code signing service the > > ASF uses (DigiCert).

Re: Reproducible builds

2021-03-22 Thread Mark Thomas
On 20/03/2021 15:24, Emmanuel Bourg wrote: Le 19/03/2021 à 16:39, Mark Thomas a écrit : Over the last few days I have been looking at making the Tomcat builds (more) reproducible. I have currently reached the stage where sequential builds on my local machine produce identical output. That's a

Re: Reproducible builds

2021-03-20 Thread Emmanuel Bourg
Le 19/03/2021 à 16:39, Mark Thomas a écrit : > Over the last few days I have been looking at making the Tomcat builds > (more) reproducible. I have currently reached the stage where sequential > builds on my local machine produce identical output. That's a great idea. > There are several caveat