Re: last review of Reproducible Builds proposal

2019-11-04 Thread Andreas Sewe
Hi, > but the format of the timestamp is completely different ... doesn't that > matter? I don't think so; at least I don't think the Bnd-LastModified header is meant to be parsed by machines. But I have _removeheaders>Bnd-* (yes, wildcards are supported) in my , so what do I know... > I was c

AW: last review of Reproducible Builds proposal

2019-11-04 Thread Christofer Dutz
: Christofer Dutz Gesendet: Montag, 4. November 2019 18:58:10 An: Maven Developers List Betreff: Re: last review of Reproducible Builds proposal Hi all, so today I made sure the LastModified and the Creator username are omitted and now the Apache PLC4X build had a 100% reproducible build (at least on my

Re: last review of Reproducible Builds proposal

2019-11-04 Thread Christofer Dutz
Hi all, so today I made sure the LastModified and the Creator username are omitted and now the Apache PLC4X build had a 100% reproducible build (at least on my one machine) ... will be checking this out on other machines. Chris Am 04.11.19, 10:12 schrieb "Christofer Dutz" : Hi Andreas,

Re: last review of Reproducible Builds proposal

2019-11-04 Thread Christofer Dutz
Hi Andreas, but the format of the timestamp is completely different ... doesn't that matter? I was currently going for the option number 1 with removing the header. In order to be 100% happy with this, I would prefer a setup where the normal mechanisms are used if no maven.build.outputTimestap i

Re: last review of Reproducible Builds proposal

2019-11-04 Thread Andreas Sewe
Christofer Dutz wrote: > Well I have a new candidate: > > maven-bundle-plugin > > Creates: Bnd-LastModified in the MANIFEST.MF > > Gotta find out a way to either suppress that entry (Would be great if it > could consume the same property) <_removeheaders>Bnd-LastModified Alternat

Re: last review of Reproducible Builds proposal

2019-11-03 Thread Christofer Dutz
Hervé - Mail original - De: "Christofer Dutz" À: "Maven Developers List" Envoyé: Dimanche 3 Novembre 2019 20:16:42 Objet: Re: last review of Reproducible Builds proposal Hi Herve, unfortunately I now have implemented some tooling to comp

Re: last review of Reproducible Builds proposal

2019-11-03 Thread herve . boutemy
- De: "Christofer Dutz" À: "Maven Developers List" Envoyé: Dimanche 3 Novembre 2019 20:16:42 Objet: Re: last review of Reproducible Builds proposal Hi Herve, unfortunately I now have implemented some tooling to compare the stuff produced by the updated reproducible buil

Re: last review of Reproducible Builds proposal

2019-11-03 Thread Christofer Dutz
Hi Herve, unfortunately I now have implemented some tooling to compare the stuff produced by the updated reproducible builds. And it does show quite a number of non binary equal files. Will investigate what's the difference. Chris Am 03.11.19, 11:08 schrieb "Hervé BOUTEMY" : great feed

Re: last review of Reproducible Builds proposal

2019-11-03 Thread Hervé BOUTEMY
great feedback, thank you: this proves the initial set of only 3 plugins is a good basis. For checking the many output artifacts from a build, future buildinfo file [1] should help a lot. I still have many usability concerns for Maven multi-module builds (starting with: should we generate only t

Re: last review of Reproducible Builds proposal

2019-11-01 Thread Christofer Dutz
Hi, so I just updated the versions of the 3 plugins and gave the Apache PLC4X build a little spin ... https://github.com/apache/plc4x/tree/feature/reproducible-builds Did two executions of this: mvn -U -P with-boost,with-java,with-dotnet,with-cpp,with-python,with-proxies,with-sandbox,with -Da

Re: last review of Reproducible Builds proposal

2019-11-01 Thread Hervé BOUTEMY
please provide an example of a visible issue so we can work on it Regards, Hervé Le vendredi 1 novembre 2019, 10:27:36 CET Vladimir Sitnikov a écrit : > OpenJDK8 uses hashmap for manifest entries, so jar files are not really > reproducible there. > > Note: there's run-to-run randomization in j.

Re: last review of Reproducible Builds proposal

2019-11-01 Thread Hervé BOUTEMY
Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit : > Hi Herve, > > thanks for that will try it asap and report any findings back. > > But good to know that there is a difference between JDK major versions and > OSes ... so it would probably be best to stage releases on Linux with

Re: last review of Reproducible Builds proposal

2019-11-01 Thread Vladimir Sitnikov
OpenJDK8 uses hashmap for manifest entries, so jar files are not really reproducible there. Note: there's run-to-run randomization in j.u.HashMap, so the manifest contents is not predictable. Vladimir

Re: last review of Reproducible Builds proposal

2019-11-01 Thread Christofer Dutz
Hi Herve, thanks for that will try it asap and report any findings back. But good to know that there is a difference between JDK major versions and OSes ... so it would probably be best to stage releases on Linux with an OpenJDK of the minimum supported version? Just thinking how to make it pos

Re: last review of Reproducible Builds proposal

2019-11-01 Thread Hervé BOUTEMY
Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit : > Hi all, > > as I can see you're voting on releasing the reproducible build extended > plugin versions. Is there any documentation on how to use this new > feature? > > I had a look at the confluence page, but that seemed like a b

Re: last review of Reproducible Builds proposal

2019-10-31 Thread Christofer Dutz
Hi all, as I can see you're voting on releasing the reproducible build extended plugin versions. Is there any documentation on how to use this new feature? I had a look at the confluence page, but that seemed like a brainstorming session. I would love to add this to the PLC4X build asap. So

Re: last review of Reproducible Builds proposal

2019-10-16 Thread Hervé BOUTEMY
Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a écrit : > Emmanuel Bourg wrote: > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit : > >> last question: now that we seem to fully understand each other, does it > >> mean that you don't need any more "seconds since the epoch" format > >> supp

Re: last review of Reproducible Builds proposal

2019-10-16 Thread Andreas Sewe
Emmanuel Bourg wrote: > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit : >> last question: now that we seem to fully understand each other, does it mean >> that you don't need any more "seconds since the epoch" format support for >> the >> property? > > If Maven supports the SOURCE_DATE_EPOCH env

Re: last review of Reproducible Builds proposal

2019-10-16 Thread Emmanuel Bourg
Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit : > last question: now that we seem to fully understand each other, does it mean > that you don't need any more "seconds since the epoch" format support for the > property? If Maven supports the SOURCE_DATE_EPOCH environment variable I don't think tha

Re: last review of Reproducible Builds proposal

2019-10-15 Thread Michael Osipov
Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY: based on the feedback I got, I updated the proposal: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format

Re: last review of Reproducible Builds proposal

2019-10-15 Thread Michael Osipov
Am 2019-10-16 um 08:35 schrieb Hervé BOUTEMY: last question: now that we seem to fully understand each other, does it mean that you don't need any more "seconds since the epoch" format support for the property? I can add it if you feel it may be useful, since it won't break any logic, even if I

Re: last review of Reproducible Builds proposal

2019-10-15 Thread Hervé BOUTEMY
last question: now that we seem to fully understand each other, does it mean that you don't need any more "seconds since the epoch" format support for the property? I can add it if you feel it may be useful, since it won't break any logic, even if I don't really see the use case given the appro

Re: last review of Reproducible Builds proposal

2019-10-11 Thread Hervé BOUTEMY
> Hope im not asking anything obvious. To me it looked as if the > > timestamp > > has to be manipulated manually. > > > > > > > > Chris > > > > > > > > Holen Sie sich Outlook für Android<http

Re: last review of Reproducible Builds proposal

2019-10-11 Thread Christofer Dutz
r Android<https://aka.ms/ghei36> > > From: Emmanuel Bourg > Sent: Thursday, October 10, 2019 11:50:34 PM > To: dev@maven.apache.org > Subject: Re: last review of Reproducible Builds proposal > > Le 10/10/2019

Re: last review of Reproducible Builds proposal

2019-10-11 Thread Enrico Olivelli
ead? > > Hope im not asking anything obvious. To me it looked as if the timestamp > has to be manipulated manually. > > Chris > > Holen Sie sich Outlook für Android<https://aka.ms/ghei36> > > From: Emmanuel Bourg > Sent: Thursday,

Re: last review of Reproducible Builds proposal

2019-10-10 Thread Christofer Dutz
Sent: Thursday, October 10, 2019 11:50:34 PM To: dev@maven.apache.org Subject: Re: last review of Reproducible Builds proposal Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit : > the only little mis-interpretation is that it is a pure build information, > then I don't see why this property

Re: last review of Reproducible Builds proposal

2019-10-10 Thread Emmanuel Bourg
Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit : > the only little mis-interpretation is that it is a pure build information, > then I don't see why this property would appear in a consumer POM Thank you for the clarification, that makes perfectly sense. And I now see the benefit of using a proper

Re: last review of Reproducible Builds proposal

2019-10-10 Thread Hervé BOUTEMY
Le jeudi 10 octobre 2019 17:17:07 CEST, vous avez écrit : > Hi Hervé, > > >> - We already have maven.build.timestamp [1], so I wonder whether it > >> should be maven.build.outputTimestap, too. Although one may argue that > >> this *output* timestamp is a property of the project being built, not >

Re: last review of Reproducible Builds proposal

2019-10-09 Thread Hervé BOUTEMY
Le mardi 8 octobre 2019, 23:42:55 CEST Mark Derricutt a écrit : > On 6 Oct 2019, at 9:14, Hervé BOUTEMY wrote: > > if anybody cares about the exact value: but > > who really looks at the timestamp of entries in release zips/jars/tar.gz > > honestly? > > I've actually done so in the past trying to

Re: last review of Reproducible Builds proposal

2019-10-08 Thread Mark Derricutt
On 6 Oct 2019, at 9:14, Hervé BOUTEMY wrote: > if anybody cares about the exact value: but > who really looks at the timestamp of entries in release zips/jars/tar.gz > honestly? I've actually done so in the past trying to find differences between two versions of a jar for repair reasons. VERY i

Re: last review of Reproducible Builds proposal

2019-10-07 Thread Emmanuel Bourg
Le 07/10/2019 à 14:40, Andreas Sewe a écrit : > - Can we get something analogous to maven.build.timestamp.format? One > could then write the following to be compatible with env.SOURCE_DATE_EPOCH: > > > > > ${env.SOURCE_DATE_EPOCH} > > ... > > +1, some kind of support for SOURCE_DATE_EP

Re: last review of Reproducible Builds proposal

2019-10-07 Thread Andreas Sewe
Hi, Hervé BOUTEMY wrote: > based on the feedback I got, I updated the proposal: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 > > The archives entries timestamp is now configured with > project.build.outputTimestamp property, in ISO-8601 format > > > > 2019

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Hervé BOUTEMY
Le dimanche 6 octobre 2019, 22:18:59 CEST Emmanuel Bourg a écrit : > Le 06/10/2019 à 20:13, Hervé BOUTEMY a écrit : > > no, it does not add any dependency on developer configuration: > > 2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 == > > 2019-10-05T16:37:42-02:00 > yes but: > > "2019-10-05T

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Romain Manni-Bucau
Small reminder: if you want to be reproducible you must fix the timestamp so whatever zone, whatever format works. It is common to use new Date(1000) in utc but not important at the end. Side note: same applies for most of the env though (locale for ex.). Le dim. 6 oct. 2019 à 22:57, Tibor Digana

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Tibor Digana
Hi Emmanuel, >> The point is, two developers may generate a different pom if the local timezone is used. very well explained! It could not be better: utc time is the same however the text is different and that's important for the stream identity. Karl had a proposal with additional property "form

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Emmanuel Bourg
Le 06/10/2019 à 20:13, Hervé BOUTEMY a écrit : > no, it does not add any dependency on developer configuration: > 2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 == 2019-10-05T16:37:42-02:00 yes but: "2019-10-05T18:37:42Z" != "2019-10-05T20:37:42+02:00" != "2019-10-05T16:37:42-02:00" The poi

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Michael Osipov
Am 2019-10-06 um 20:35 schrieb Hervé BOUTEMY: Le dimanche 6 octobre 2019 20:29:46 CEST, vous avez écrit : Am 2019-10-06 um 20:21 schrieb Hervé BOUTEMY:> Le dimanche 6 octobre 2019 12:24:57 CEST, vous avez écrit : >> Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY: >>> Le samedi 5 octobre 2019,

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Michael Osipov
Am 2019-10-06 um 20:21 schrieb Hervé BOUTEMY:> Le dimanche 6 octobre 2019 12:24:57 CEST, vous avez écrit : >> Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY: >>> Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit : Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY: > Le samedi 5 oct

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Hervé BOUTEMY
Le dimanche 6 octobre 2019, 16:41:27 CEST Emmanuel Bourg a écrit : > Le 06/10/2019 à 09:43, Hervé BOUTEMY a écrit : > > Notice that you can also express a timezone (as digits), as seen in the > > unit tests. > I know but that's not desirable, otherwise the formatted timestamp would > depend on the

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Hervé BOUTEMY
Le dimanche 6 octobre 2019, 12:19:35 CEST Michael Osipov a écrit : > Am 2019-10-05 um 23:15 schrieb Emmanuel Bourg: > > Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit : > >> based on the feedback I got, I updated the proposal: > >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=7468

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Hervé BOUTEMY
Le dimanche 6 octobre 2019, 16:41:27 CEST Emmanuel Bourg a écrit : > Le 06/10/2019 à 09:43, Hervé BOUTEMY a écrit : > > Notice that you can also express a timezone (as digits), as seen in the > > unit tests. > I know but that's not desirable, otherwise the formatted timestamp would > depend on the

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Vladimir Sitnikov
>ISO 8601 is neither aware of zones or >DSTs, just abstract offsets which is a good thing Well. ISO 8601 allows timestamps without "UTC offset" For instance, 2019-10-06T12:30:00 is ISO 8601-compatible timestamp, however it is ambiguous. So I suggest that the property must require Z or +HH:MM part

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Michael Osipov
I don't understand that either. ISO 8601 is neither aware of zones or DSTs, just abstract offsets which is a good thing. The format Hervé has chosen is almost correct (comments) left. The way the value has to be provided can *always* be canonicalized to UTC. I don't see here a big hurdle to sol

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Emmanuel Bourg
Le 06/10/2019 à 09:43, Hervé BOUTEMY a écrit : > Notice that you can also express a timezone (as digits), as seen in the unit > tests. I know but that's not desirable, otherwise the formatted timestamp would depend on the timezone of the developer and that would harm the reproducibility of the p

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Tibor Digana
The difference between this time and UTC is ( Zone Offset + DST offset ). I think here this feature does not need to describe the LOCAL time nothing but the timestamp and that is in UTC. It makes sense that the plugins use particular FORMAT, forinstance platform format of predefined format by the M

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Michael Osipov
I still don't see and issue because the offset is there. If you subtract or add the offset and you have the Zulu time. Can you provide this concrete example? I am quite certain that there was an error on some side. If you case is true, the entire time logic in Java 8 woudn't be able to perfo

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Tibor Digana
Michael, it is the problem with summer time. Do you know what i mean?We had this problem in my company therefore we strictly used Z as UTC and if somebody sent another timezone we sent back an error from REST. You cannot say that you disagree if you do not understand. Pls have it logically explaine

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Michael Osipov
Am 2019-10-06 um 12:25 schrieb Tibor Digana: ISO format was often discussed and this was found as problematic format because you cannot always compute it to UTC due to GMT offset. The offset is not enough. What is required for EXACT computing to UTC is Time zome name but this format does not su

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Tibor Digana
ISO format was often discussed and this was found as problematic format because you cannot always compute it to UTC due to GMT offset. The offset is not enough. What is required for EXACT computing to UTC is Time zome name but this format does not support it. It is exactly the same problem in XML.

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Michael Osipov
Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY: Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit : Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY: Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit : Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY: based on the feedback I got, I u

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Michael Osipov
Am 2019-10-05 um 23:15 schrieb Emmanuel Bourg: Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit : based on the feedback I got, I updated the proposal: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 The archives entries timestamp is now configured with project.build.outpu

Fwd: Re: last review of Reproducible Builds proposal

2019-10-06 Thread Hervé BOUTEMY
Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit : > Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY: > > based on the feedback I got, I updated the proposal: > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 > > > > The archives entries timestamp is now configured

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Hervé BOUTEMY
Le samedi 5 octobre 2019, 23:15:58 CEST Emmanuel Bourg a écrit : > Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit : > > based on the feedback I got, I updated the proposal: > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 > > > > The archives entries timestamp is now con

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Hervé BOUTEMY
Yes, having the timestamp to 0 is something I wanted generally to avoid. But this has opened another question: use what value? Can this be automated? I knew that war files could be a specific use case. Perhaps this plugin requires a specific way of handling reproducibility, even more than the sta

Re: last review of Reproducible Builds proposal

2019-10-06 Thread Hervé BOUTEMY
Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit : > Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY: > > Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit : > >> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY: > >>> based on the feedback I got, I updated the proposal: > >>> https://

Re: last review of Reproducible Builds proposal

2019-10-05 Thread Emmanuel Bourg
Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit : > based on the feedback I got, I updated the proposal: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 > > The archives entries timestamp is now configured with > project.build.outputTimestamp property, in ISO-8601 format

Re: last review of Reproducible Builds proposal

2019-10-05 Thread Michael Osipov
Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY: Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit : Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY: based on the feedback I got, I updated the proposal: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 The archives entr

Re: last review of Reproducible Builds proposal

2019-10-05 Thread Vladimir Sitnikov
>but >who really looks at the timestamp of entries in release zips/jars/tar.gz >honestly? Tomcat when it decides on what to send in the "Last-Modified" header. For instance, current Gradle does not allow to configure the timestamp, and for reproducible builds it always sets the timestamp to 0 or s

Re: last review of Reproducible Builds proposal

2019-10-05 Thread Hervé BOUTEMY
with inheritance from parent, this question disappears: once parent pom has a timestamp value to have a reproducible release, child poms inherit the value and the reproducible feature changing the value in child is just a little improvement to get a timestamp that has more meaningful value, if a

Re: last review of Reproducible Builds proposal

2019-10-05 Thread Tibor Digana
Hi Herve, I want to make sure we understand correctly. So. What you want to achieve with this property is to stick the property to one fixed value when the user has supposed the archive would have same content. And opposite, means that the property would be real when the content of the archive sho

Re: last review of Reproducible Builds proposal

2019-10-05 Thread Michael Osipov
Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY: based on the feedback I got, I updated the proposal: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format

last review of Reproducible Builds proposal

2019-10-05 Thread Hervé BOUTEMY
based on the feedback I got, I updated the proposal: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format 2019-10-02T08:04:00Z The shared components,