Re: RFC v3: Another proposed hash function transition plan

2017-10-02 Thread Jeff King
On Mon, Oct 02, 2017 at 10:18:02AM -0700, Linus Torvalds wrote: > On Mon, Oct 2, 2017 at 7:00 AM, Jason Cooper wrote: > > > > Ahhh, so if I understand you correctly, you'd prefer SHA-256 over > > SHA3-256 because it's more performant for your usecase? Well, that's a > > completely different anim

Re: RFC v3: Another proposed hash function transition plan

2017-10-02 Thread Linus Torvalds
On Mon, Oct 2, 2017 at 7:00 AM, Jason Cooper wrote: > > Ahhh, so if I understand you correctly, you'd prefer SHA-256 over > SHA3-256 because it's more performant for your usecase? Well, that's a > completely different animal that cryptographic suitability. In almost all loads I've seen, zlib inf

Re: RFC v3: Another proposed hash function transition plan

2017-10-02 Thread Brandon Williams
On 10/02, Jason Cooper wrote: > Hi Jonathan, > > On Tue, Sep 26, 2017 at 04:51:58PM -0700, Jonathan Nieder wrote: > > Johannes Schindelin wrote: > > > On Tue, 26 Sep 2017, Jason Cooper wrote: > > >> For my use cases, as a user of git, I have a plan to maintain provable > > >> integrity of existing

Re: RFC v3: Another proposed hash function transition plan

2017-10-02 Thread Jason Cooper
Hi Jonathan, On Tue, Sep 26, 2017 at 04:51:58PM -0700, Jonathan Nieder wrote: > Johannes Schindelin wrote: > > On Tue, 26 Sep 2017, Jason Cooper wrote: > >> For my use cases, as a user of git, I have a plan to maintain provable > >> integrity of existing objects stored in git under sha1 while migr

Re: RFC v3: Another proposed hash function transition plan

2017-10-02 Thread Johannes Schindelin
Hi Joan, On Sun, 1 Oct 2017, Joan Daemen wrote: > On 30/09/17 00:33, Johannes Schindelin wrote: > > > As far as Git is concerned, we not only care about the source code of > > the hash algorithm we use, we need to care even more about what you > > call "executable": ready-to-use, high quality, w

Re: RFC v3: Another proposed hash function transition plan

2017-10-02 Thread Jason Cooper
Hi Johannes, Thanks for the response. Sorry for the delay. Had a large deadline for $dayjob. On Wed, Sep 27, 2017 at 12:11:14AM +0200, Johannes Schindelin wrote: > On Tue, 26 Sep 2017, Jason Cooper wrote: > > On Thu, Sep 14, 2017 at 08:45:35PM +0200, Johannes Schindelin wrote: > > > On Wed, 13

Re: RFC v3: Another proposed hash function transition plan

2017-09-30 Thread Joan Daemen
and an SHA-1, then we have a huge problem on our hands. Ciao, Johannes Begin forwarded message: From: Gilles Van Assche Subject: Re: RFC v3: Another proposed hash function transition plan Date: 30 Sep 2017 22:20:42 CEST To: Joan Daemen , kec...@noekeon.org Dag Joan, About the implementation

Re: RFC v3: Another proposed hash function transition plan

2017-09-29 Thread Johannes Schindelin
Hi Joan, On Fri, 29 Sep 2017, Joan Daemen wrote: > if ever there was a SHA-2 competition, it must have been held inside NSA:-) Oops. My bad, I indeed got confused about that, as you suggest below (I actually thought of the AES competition, but that was obviously not about SHA-2). Sorry. > But m

Re: RFC v3: Another proposed hash function transition plan

2017-09-29 Thread Joan Daemen
Dear Johannes, if ever there was a SHA-2 competition, it must have been held inside NSA:-) But maybe you are confusing with the SHA-3 competition. In any case, when considering SHA-2 vs SHA-3 for usage in git, you may have a look at arguments we give in the following blogpost: https://keccak

Re: RFC v3: Another proposed hash function transition plan

2017-09-29 Thread Johannes Schindelin
Hi Gilles, On Tue, 19 Sep 2017, Gilles Van Assche wrote: > On 19/09/17 00:16, Johannes Schindelin wrote: > >>> SHA-256 got much more cryptanalysis than SHA3-256 […]. > >> > >> I do not think this is true. > > > > Please read what I said again: SHA-256 got much more cryptanalysis > > than SHA3-256

Re: RFC v3: Another proposed hash function transition plan

2017-09-26 Thread Jonathan Nieder
Hi, Johannes Schindelin wrote: > Sorry, you are asking cryptography experts to spend their time on the Git > mailing list. I tried to get them to speak out on the Git mailing list. > They respectfully declined. > > I can't fault them, they have real jobs to do, and none of their managers > would

Re: RFC v3: Another proposed hash function transition plan

2017-09-26 Thread Johannes Schindelin
Hi Jason, On Tue, 26 Sep 2017, Jason Cooper wrote: > On Thu, Sep 14, 2017 at 08:45:35PM +0200, Johannes Schindelin wrote: > > On Wed, 13 Sep 2017, Linus Torvalds wrote: > > > On Wed, Sep 13, 2017 at 6:43 AM, demerphq wrote: > > > > SHA3 however uses a completely different design where it mixes a

Re: RFC v3: Another proposed hash function transition plan

2017-09-26 Thread Jason Cooper
Hi all, Sorry for late commentary... On Thu, Sep 14, 2017 at 08:45:35PM +0200, Johannes Schindelin wrote: > On Wed, 13 Sep 2017, Linus Torvalds wrote: > > On Wed, Sep 13, 2017 at 6:43 AM, demerphq wrote: > > > SHA3 however uses a completely different design where it mixes a 1088 > > > bit block

Re: RFC v3: Another proposed hash function transition plan

2017-09-19 Thread Gilles Van Assche
Hi Johannes, Thanks for your feedback. On 19/09/17 00:16, Johannes Schindelin wrote: >>> SHA-256 got much more cryptanalysis than SHA3-256 […]. >> >> I do not think this is true. > > Please read what I said again: SHA-256 got much more cryptanalysis > than SHA3-256. Indeed. What I meant is tha

Re: RFC v3: Another proposed hash function transition plan

2017-09-18 Thread Jonathan Nieder
Hi, Gilles Van Assche wrote: > Hi Johannes, >> SHA-256 got much more cryptanalysis than SHA3-256 […]. > > I do not think this is true. Keccak/SHA-3 actually got (and is still > getting) a lot of cryptanalysis, with papers published at renowned > crypto conferences [1]. > > Keccak/SHA-3 is recogni

Re: RFC v3: Another proposed hash function transition plan

2017-09-18 Thread Johannes Schindelin
Hi Gilles, On Mon, 18 Sep 2017, Gilles Van Assche wrote: > > SHA-256 got much more cryptanalysis than SHA3-256 […]. > > I do not think this is true. Please read what I said again: SHA-256 got much more cryptanalysis than SHA3-256. I never said that SHA3-256 got little cryptanalysis. Personally

Re: RFC v3: Another proposed hash function transition plan

2017-09-18 Thread Gilles Van Assche
Hi Johannes, > SHA-256 got much more cryptanalysis than SHA3-256 […]. I do not think this is true. Keccak/SHA-3 actually got (and is still getting) a lot of cryptanalysis, with papers published at renowned crypto conferences [1]. Keccak/SHA-3 is recognized to have a significant safety margin. E.

Re: RFC v3: Another proposed hash function transition plan

2017-09-15 Thread Philip Oakley
Hi Jonathan, "Jonathan Nieder" wrote; Johannes Schindelin wrote: On Wed, 13 Sep 2017, Jonathan Nieder wrote: As a side note, I am probably misreading, but I found this set of paragraphs a bit condescending. It sounds to me like you are saying "You are making the wrong choice of hash functi

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Johannes Schindelin
Hi Jonathan, On Thu, 14 Sep 2017, Jonathan Nieder wrote: > Johannes Schindelin wrote: > > On Wed, 13 Sep 2017, Jonathan Nieder wrote: > > >> [3] https://www.imperialviolet.org/2017/05/31/skipsha3.html, > > > > I had read this short after it was published, and had missed the updates. > > One link

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Johannes Schindelin
Hi, On Thu, 14 Sep 2017, demerphq wrote: > On 14 September 2017 at 17:23, Johannes Schindelin > wrote: > > > > SHA-256 has been hammered on a lot more than SHA3-256. > > Last year that was even more true of SHA1 than it is true of SHA-256 > today. I hope you are not deliberately trying to anno

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Jonathan Nieder
Johannes Schindelin wrote: > On Wed, 13 Sep 2017, Jonathan Nieder wrote: >> As a side note, I am probably misreading, but I found this set of >> paragraphs a bit condescending. It sounds to me like you are saying >> "You are making the wrong choice of hash function and everything else >> you are

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Johannes Schindelin
Hi Linus, On Wed, 13 Sep 2017, Linus Torvalds wrote: > On Wed, Sep 13, 2017 at 6:43 AM, demerphq wrote: > > > > SHA3 however uses a completely different design where it mixes a 1088 > > bit block into a 1600 bit state, for a leverage of 2:3, and the excess > > is *preserved between each block*.

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Jonathan Nieder
Hi, Johannes Schindelin wrote: > On Wed, 13 Sep 2017, Jonathan Nieder wrote: >> [3] https://www.imperialviolet.org/2017/05/31/skipsha3.html, > > I had read this short after it was published, and had missed the updates. > One link in particular caught my eye: > > https://eprint.iacr.org/2012

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Johannes Schindelin
Hi Jonathan, On Wed, 13 Sep 2017, Jonathan Nieder wrote: > [3] https://www.imperialviolet.org/2017/05/31/skipsha3.html, I had read this short after it was published, and had missed the updates. One link in particular caught my eye: https://eprint.iacr.org/2012/476 Essentially, the auth

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Brandon Williams
On 09/14, Johannes Schindelin wrote: > Hi Jonathan, > > On Wed, 13 Sep 2017, Jonathan Nieder wrote: > > > As a side note, I am probably misreading, but I found this set of > > paragraphs a bit condescending. It sounds to me like you are saying > > "You are making the wrong choice of hash functio

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread demerphq
On 14 September 2017 at 17:23, Johannes Schindelin wrote: > Hi Junio, > > On Thu, 14 Sep 2017, Junio C Hamano wrote: > >> Jonathan Nieder writes: >> >> > In other words, a long lifetime for the hash absolutely is a design >> > goal. Coping well with an unexpectedly short lifetime for the hash is

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Johannes Schindelin
Hi Junio, On Thu, 14 Sep 2017, Junio C Hamano wrote: > Jonathan Nieder writes: > > > In other words, a long lifetime for the hash absolutely is a design > > goal. Coping well with an unexpectedly short lifetime for the hash is > > also a design goal. > > > > If the hash function lasts 10 years

Re: RFC v3: Another proposed hash function transition plan

2017-09-14 Thread Johannes Schindelin
Hi Jonathan, On Wed, 13 Sep 2017, Jonathan Nieder wrote: > As a side note, I am probably misreading, but I found this set of > paragraphs a bit condescending. It sounds to me like you are saying > "You are making the wrong choice of hash function and everything else > you are describing is irrel

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Junio C Hamano
Jonathan Nieder writes: > In other words, a long lifetime for the hash absolutely is a design > goal. Coping well with an unexpectedly short lifetime for the hash is > also a design goal. > > If the hash function lasts 10 years then I am happy. Absolutely. When two functions have similar expec

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Junio C Hamano
Jonathan Nieder writes: > The NewHash-based signature is included in the SHA-1 content as well, > for the sake of round-tripping. It is not stripped out. Ah, OK, that allays my worries. We rely on the fact that unknown object headers from the future are ignored. We use something other than "g

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Linus Torvalds
On Wed, Sep 13, 2017 at 6:43 AM, demerphq wrote: > > SHA3 however uses a completely different design where it mixes a 1088 > bit block into a 1600 bit state, for a leverage of 2:3, and the excess > is *preserved between each block*. Yes. And considering that the SHA1 attack was actually predicate

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Jonathan Nieder
Hi, Yves wrote: > On 13 September 2017 at 14:05, Johannes Schindelin >> For example, I am still in favor of SHA-256 over SHA3-256, after learning >> some background details from in-house cryptographers: it provides >> essentially the same level of security, according to my sources, while >> hardw

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Jonathan Nieder
Junio C Hamano wrote: > In the proposed transition plan, the treatment of various signatures > (deliberately) makes the conversion not quite roundtrip. That's not precisely true. Details below. > When existing SHA-1 history in individual clones are converted to > NewHash, we obviously cannot re

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Jonathan Nieder
Hi, Stefan Beller wrote: > On Wed, Sep 13, 2017 at 2:52 PM, Junio C Hamano wrote: >> This is a tangent, but it may be fine for a shallow clone to treat >> the cut-off points in the history as if they are root commits and >> compute generation numbers locally, just like everybody else does. [...]

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Junio C Hamano
Junio C Hamano writes: > Jonathan Nieder writes: > >> Treating generation numbers as derived data (as in Jeff King's >> preferred design, if I have understood his replies correctly) would >> also be possible but it does not interact well with shallow clone or >> narrow clone. > > Just like we ha

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Stefan Beller
On Wed, Sep 13, 2017 at 2:52 PM, Junio C Hamano wrote: > Jonathan Nieder writes: > >> Treating generation numbers as derived data (as in Jeff King's >> preferred design, if I have understood his replies correctly) would >> also be possible but it does not interact well with shallow clone or >> na

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Junio C Hamano
Jonathan Nieder writes: > Treating generation numbers as derived data (as in Jeff King's > preferred design, if I have understood his replies correctly) would > also be possible but it does not interact well with shallow clone or > narrow clone. Just like we have skewed committer timestamps, the

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Jonathan Nieder
Hi Dscho, Johannes Schindelin wrote: > So even if the code to generate a bidirectional old <-> new hash mapping > might be with us forever, it *definitely* should be optional ("optional" > at least as in "config setting"), allowing developers who only work with > new-hash repositories to save the

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread demerphq
On 13 September 2017 at 14:05, Johannes Schindelin wrote: > For example, I am still in favor of SHA-256 over SHA3-256, after learning > some background details from in-house cryptographers: it provides > essentially the same level of security, according to my sources, while > hardware support seem

Re: RFC v3: Another proposed hash function transition plan

2017-09-13 Thread Johannes Schindelin
Hi Brandon, On Mon, 11 Sep 2017, Brandon Williams wrote: > On 09/08, Junio C Hamano wrote: > > Junio C Hamano writes: > > > > > One thing I still do not know how I feel about after re-reading the > > > thread, and I didn't find the above doc, is Linus's suggestion to > > > use the objects thems

Re: RFC v3: Another proposed hash function transition plan

2017-09-11 Thread Brandon Williams
On 09/08, Junio C Hamano wrote: > Junio C Hamano writes: > > > One thing I still do not know how I feel about after re-reading the > > thread, and I didn't find the above doc, is Linus's suggestion to > > use the objects themselves as NewHash-to-SHA-1 mapper [*1*]. > > ... > > [Reference] > > >

Re: RFC v3: Another proposed hash function transition plan

2017-09-07 Thread Jeff King
On Fri, Sep 08, 2017 at 11:40:21AM +0900, Junio C Hamano wrote: > Our "git fsck" already treats certain brokenness (like a tree whose > entry has mode that is 0-padded to the left) as broken but still > tolerate them. I am not sure if it is sufficient to diagnose and > declare broken and invalid

Re: RFC v3: Another proposed hash function transition plan

2017-09-07 Thread Junio C Hamano
Junio C Hamano writes: > One thing I still do not know how I feel about after re-reading the > thread, and I didn't find the above doc, is Linus's suggestion to > use the objects themselves as NewHash-to-SHA-1 mapper [*1*]. > ... > [Reference] > > *1* I think this falls into the same category

Re: RFC v3: Another proposed hash function transition plan

2017-09-05 Thread Junio C Hamano
Jonathan Nieder writes: > Linus Torvalds wrote: >> On Fri, Mar 3, 2017 at 5:12 PM, Jonathan Nieder wrote: > >>> This document is still in flux but I thought it best to send it out >>> early to start getting feedback. >> >> This actually looks very reasonable if you can implement it cleanly >> en

Re: RFC v3: Another proposed hash function transition plan

2017-03-10 Thread Jonathan Nieder
Jeff King wrote: > On Thu, Mar 09, 2017 at 12:24:08PM -0800, Jonathan Nieder wrote: >>> SHA-1 to SHA-3: lookup SHA-1 in .msha1, reverse .idx, find offset to >>> read the SHA-3. >>> SHA-3 to SHA-1: lookup SHA-3 in .idx, and reverse the .msha1 file to >>> translate offset to SHA-1. >> >> Thanks for

Re: RFC v3: Another proposed hash function transition plan

2017-03-10 Thread Jeff King
On Thu, Mar 09, 2017 at 12:24:08PM -0800, Jonathan Nieder wrote: > > SHA-1 to SHA-3: lookup SHA-1 in .msha1, reverse .idx, find offset to > > read the SHA-3. > > SHA-3 to SHA-1: lookup SHA-3 in .idx, and reverse the .msha1 file to > > translate offset to SHA-1. > > Thanks for this suggestion. I

Re: RFC v3: Another proposed hash function transition plan

2017-03-09 Thread Jonathan Nieder
Hi, Shawn Pearce wrote: > On Mon, Mar 6, 2017 at 4:17 PM, Jonathan Nieder wrote: >> Alongside the packfile, a sha3 repository stores a bidirectional >> mapping between sha3 and sha1 object names. The mapping is generated >> locally and can be verified using "git fsck". Object lookups use this >>

Re: RFC v3: Another proposed hash function transition plan

2017-03-09 Thread Shawn Pearce
On Mon, Mar 6, 2017 at 4:17 PM, Jonathan Nieder wrote: > Linus Torvalds wrote: >> On Fri, Mar 3, 2017 at 5:12 PM, Jonathan Nieder wrote: > >>> This document is still in flux but I thought it best to send it out >>> early to start getting feedback. >> >> This actually looks very reasonable if you

RFC v3: Another proposed hash function transition plan

2017-03-06 Thread Jonathan Nieder
Linus Torvalds wrote: > On Fri, Mar 3, 2017 at 5:12 PM, Jonathan Nieder wrote: >> This document is still in flux but I thought it best to send it out >> early to start getting feedback. > > This actually looks very reasonable if you can implement it cleanly > enough. Thanks for the kind words on