Re: Question about tcp hash function tcp_hashfn()

2006-06-02 Thread Brian F. G. Bidulock
Florian, On Fri, 02 Jun 2006, Florian Weimer wrote: > > I see them now. Hmm. Is there a theoretical explanation for them? Jenkins is an ad hoc function that is far from ideal. As you know, the ideal hash changes 1/2 the bits in the output value for each one bit change in the input value(s).

Re: Question about tcp hash function tcp_hashfn()

2006-06-02 Thread Florian Weimer
* Evgeniy Polyakov: > :) thats true, but to be 100% honest I used different code to test for > hash artifacts... Ah, okay. > But it still does not fix artifacts with for example const IP and random > ports or const IP and linear port selection. I see them now. Hmm. Is there a theoretical expl

Re: Question about tcp hash function tcp_hashfn()

2006-06-02 Thread Brian F. G. Bidulock
Evgeniy, I agree, even with constant source IP, the hash still should have performed better (but didn't). Constant source IP and varying port is a realistic data set for a port proxy. - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED]

Re: Question about tcp hash function tcp_hashfn()

2006-06-02 Thread Evgeniy Polyakov
On Fri, Jun 02, 2006 at 07:40:38AM +0200, Florian Weimer ([EMAIL PROTECTED]) wrote: > * Evgeniy Polyakov: > > > That is wrong. And I have a code and picture to show that, > > and you dont - prove me wrong :) > > Here we go: > > static inline num2ip(__u8 a1, __u8 a2, __u8 a3, __u8 a4) > { >

Re: Question about tcp hash function tcp_hashfn()

2006-06-02 Thread Florian Weimer
* Evgeniy Polyakov: > That is wrong. And I have a code and picture to show that, > and you dont - prove me wrong :) Here we go: static inline num2ip(__u8 a1, __u8 a2, __u8 a3, __u8 a4) { __u32 a = 0; a |= a1; a << 8; a |= a2; a << 8; a |= a3;

Re: Question about tcp hash function tcp_hashfn()

2006-06-02 Thread Evgeniy Polyakov
On Thu, Jun 01, 2006 at 12:40:10PM -0600, Brian F. G. Bidulock ([EMAIL PROTECTED]) wrote: > So what are your thoughts about my sequence number approach (for > connected sockets)? Depending on how you are going to use it. Generic socket code does not have TCP sequence numbers since it must work wi

Re: Question about tcp hash function tcp_hashfn()

2006-06-01 Thread David Miller
From: "Brian F. G. Bidulock" <[EMAIL PROTECTED]> Date: Thu, 1 Jun 2006 12:40:10 -0600 > I think the sun shines more in Moscow than in Edmonton, so it is not > so random. ;) Go Oilers :) - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTE

Re: Question about tcp hash function tcp_hashfn()

2006-06-01 Thread Brian F. G. Bidulock
Evgeniy, On Thu, 01 Jun 2006, Evgeniy Polyakov wrote: I think the sun shines more in Moscow than in Edmonton, so it is not so random. ;) > > Specially for you :) Thank you for being so gracious and patient with me. > It does not have artifacts, but it's dispersion is wider than XOR one. > _Mu

Re: Question about tcp hash function tcp_hashfn()

2006-06-01 Thread Evgeniy Polyakov
On Thu, Jun 01, 2006 at 04:24:57AM -0600, Brian F. G. Bidulock ([EMAIL PROTECTED]) wrote: > For purely random numbers you could amplify thermal noise off an > open transitor junction (the audiofile's white noise generator) > and feed it into an analog to digital converter. It is also possible to

Re: Question about tcp hash function tcp_hashfn()

2006-06-01 Thread Brian F. G. Bidulock
Evgeniy, On Thu, 01 Jun 2006, Evgeniy Polyakov wrote: For purely random numbers you could amplify thermal noise off an open transitor junction (the audiofile's white noise generator) and feed it into an analog to digital converter. > > I've run it with following source ip/port selection algo: >

Re: Question about tcp hash function tcp_hashfn()

2006-06-01 Thread Evgeniy Polyakov
On Thu, Jun 01, 2006 at 01:11:25AM -0600, Brian F. G. Bidulock ([EMAIL PROTECTED]) wrote: > Evgeniy, > > On Thu, 01 Jun 2006, Evgeniy Polyakov wrote: > > > On Thu, Jun 01, 2006 at 12:46:08AM -0600, Brian F. G. Bidulock ([EMAIL > > PROTECTED]) wrote: > > > > Since pseudo-randomness affects both

Re: Question about tcp hash function tcp_hashfn()

2006-06-01 Thread Brian F. G. Bidulock
Evgeniy, On Thu, 01 Jun 2006, Evgeniy Polyakov wrote: > On Thu, Jun 01, 2006 at 12:46:08AM -0600, Brian F. G. Bidulock ([EMAIL > PROTECTED]) wrote: > > > Since pseudo-randomness affects both folded and not folded hash > > > distribution, it can not end up in different results. > > > > Yes it wo

Re: Question about tcp hash function tcp_hashfn()

2006-06-01 Thread Evgeniy Polyakov
On Thu, Jun 01, 2006 at 12:46:08AM -0600, Brian F. G. Bidulock ([EMAIL PROTECTED]) wrote: > > Since pseudo-randomness affects both folded and not folded hash > > distribution, it can not end up in different results. > > Yes it would, so to rule out pseudo-random effects the pseudo- > random numbe

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
Evgeniy, On Thu, 01 Jun 2006, Evgeniy Polyakov wrote: > > Since pseudo-randomness affects both folded and not folded hash > distribution, it can not end up in different results. Yes it would, so to rule out pseudo-random effects the pseudo- random number generator must be removed. > > You are

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Evgeniy Polyakov
On Thu, Jun 01, 2006 at 12:18:25AM -0600, Brian F. G. Bidulock ([EMAIL PROTECTED]) wrote: > Evgeniy, > > On Thu, 01 Jun 2006, Evgeniy Polyakov wrote: > > > > for (i=0; i > saddr = num2ip(get_random_byte(), get_random_byte(), > > get_random_byte(), get_random_byte()); > >

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread David Miller
From: "Brian F. G. Bidulock" <[EMAIL PROTECTED]> Date: Thu, 1 Jun 2006 00:22:21 -0600 > I thought you said you were considering jenkins_3word(), not > jenkins_2word()? We could xor some of the inputs in order to use jenkins_2word(). - To unsubscribe from this list: send the line "unsubscribe netd

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
David, On Wed, 31 May 2006, David Miller wrote: > > Ok I believe you now :) > I'll believe it if he interates through a subset and gets the same results instead of using a pseudo-random number generator. I thought you said you were considering jenkins_3word(), not jenkins_2word()? - To unsubs

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
Evgeniy, On Thu, 01 Jun 2006, Evgeniy Polyakov wrote: > > for (i=0; i saddr = num2ip(get_random_byte(), get_random_byte(), > get_random_byte(), get_random_byte()); > sport = get_random_word(); You still have a problem: you cannot use a pseudo-random numbe

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread David Miller
From: Evgeniy Polyakov <[EMAIL PROTECTED]> Date: Thu, 1 Jun 2006 10:12:36 +0400 > I've present the new simple code and test results which show > that folded and not folded Jenkins hashes _do_ produce _exactly_ the > same distribution. Ok I believe you now :) > I think I've already said that fair

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Evgeniy Polyakov
On Wed, May 31, 2006 at 12:29:55PM -0600, Brian F. G. Bidulock ([EMAIL PROTECTED]) wrote: > Evgeniy, > > On Wed, 31 May 2006, Evgeniy Polyakov wrote: > > > > > Worse: he folded the jenkins algorith result with > > > > > > > > > >h ^= h >> 16; > > > > >h ^= h >> 8; > > > > > > > > > >

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Evgeniy Polyakov
On Wed, May 31, 2006 at 11:41:27AM -0700, David Miller ([EMAIL PROTECTED]) wrote: > > > Worse: he folded the jenkins algorith result with > > > > > >h ^= h >> 16; > > >h ^= h >> 8; > > > > > > Destroying the coverage of the function. > > > > It was done to simulate socket code which

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
Evgeniy, On Wed, 31 May 2006, Evgeniy Polyakov wrote: > > > > Worse: he folded the jenkins algorith result with > > > > > > > >h ^= h >> 16; > > > >h ^= h >> 8; > > > > > > > > Destroying the coverage of the function. > > > > > > It was done to simulate socket code which uses the sa

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread David Miller
From: Evgeniy Polyakov <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 14:58:18 +0400 > On Wed, May 31, 2006 at 03:51:24AM -0600, Brian F. G. Bidulock ([EMAIL > PROTECTED]) wrote: > > Worse: he folded the jenkins algorith result with > > > >h ^= h >> 16; > >h ^= h >> 8; > > > > Destroyin

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Evgeniy Polyakov
On Wed, May 31, 2006 at 03:04:59PM +0400, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: > On Wed, May 31, 2006 at 02:58:18PM +0400, Evgeniy Polyakov ([EMAIL > PROTECTED]) wrote: > > On Wed, May 31, 2006 at 03:51:24AM -0600, Brian F. G. Bidulock ([EMAIL > > PROTECTED]) wrote: > > > Evgeniy, > > >

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Evgeniy Polyakov
On Wed, May 31, 2006 at 02:58:18PM +0400, Evgeniy Polyakov ([EMAIL PROTECTED]) wrote: > On Wed, May 31, 2006 at 03:51:24AM -0600, Brian F. G. Bidulock ([EMAIL > PROTECTED]) wrote: > > Evgeniy, > > > > On Wed, 31 May 2006, Evgeniy Polyakov wrote: > > > 2. Compared Jenkins hash with XOR hash used

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Evgeniy Polyakov
On Wed, May 31, 2006 at 03:51:24AM -0600, Brian F. G. Bidulock ([EMAIL PROTECTED]) wrote: > Evgeniy, > > On Wed, 31 May 2006, Evgeniy Polyakov wrote: > > 2. Compared Jenkins hash with XOR hash used in TCP socket selection code. > > http://tservice.net.ru/~s0mbre/blog/2006/05/14#2006_05_14 > > Tw

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
Evgeniy, On Wed, 31 May 2006, Evgeniy Polyakov wrote: > > 1. Netchannel. > http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel This one refers to the erroneous result below. > > 2. Compared Jenkins hash with XOR hash used in TCP socket selection > code. > http://tservice.net.r

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
Evgeniy, On Wed, 31 May 2006, Evgeniy Polyakov wrote: > 2. Compared Jenkins hash with XOR hash used in TCP socket selection code. > http://tservice.net.ru/~s0mbre/blog/2006/05/14#2006_05_14 Two problems with the comparison: Port numbers can be collected into a 32 bit register in network byte

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Evgeniy Polyakov
On Wed, May 31, 2006 at 02:12:43AM -0700, David Miller ([EMAIL PROTECTED]) wrote: > I think it will need to be changed nevertheless. Even though this > hash works on established sockets, it is attackable just like the > routing hash used to be. If an attacker has sufficient resources, he > can m

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
David, On Wed, 31 May 2006, David Miller wrote: > > I don't know how practical this is. The 4GB sequence space > wraps very fast on 10 gigabit, so we'd be rehashing a bit > and 100 gigabit would make things worse whenever that shows > up. It works better for SCTP, because the vtags are constant

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread David Miller
From: Evgeniy Polyakov <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 13:03:02 +0400 > Current simple XOR hash used in socket selection code is just bloody > good! Jenkins hash unfortunately has _significant_ artefacts which > were found in netchannel [1] hash selection analysis [2]. And > Jenkins h

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Evgeniy Polyakov
On Wed, May 31, 2006 at 02:00:09AM -0600, Brian F. G. Bidulock ([EMAIL PROTECTED]) wrote: > > For sure and there are plans afoot to move over to > > dynamic table sizing and the Jenkins hash function. > > Yes, that could be far more efficient. In theory practice and theory are the same, but in p

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread David Miller
From: "Brian F. G. Bidulock" <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 02:49:54 -0600 > This would be faster than a Jenkins hash approach because it > would not be necessary to calculate the hash function at all for > in-window segments. Per packet overheads would decrease and > better small pac

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
David, On Wed, 31 May 2006, David Miller wrote: > > For sure and there are plans afoot to move over to > dynamic table sizing and the Jenkins hash function. Just a suggestion, but I have an approach that stands to be faster than Jenkins deriving from the verification tag approach that I took for

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
David, On Wed, 31 May 2006, David Miller wrote: > > For sure and there are plans afoot to move over to > dynamic table sizing and the Jenkins hash function. Yes, that could be far more efficient. - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EM

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread David Miller
From: "Brian F. G. Bidulock" <[EMAIL PROTECTED]> Date: Wed, 31 May 2006 01:45:40 -0600 > It might sound like I'm complaining, but I'm not. The > function works for me. But from a purist point of view, > the hash function is not as efficient as it could be and > there is room for improvement. Fo

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread Brian F. G. Bidulock
David, On Wed, 31 May 2006, David Miller wrote: > From: "Brian F. G. Bidulock" <[EMAIL PROTECTED]> > Date: Tue, 30 May 2006 23:55:26 -0600 > > > For example, it goes to great pains to permute upper order bits in > > the local address, which for most connections will be a constant > > value. > >

Re: Question about tcp hash function tcp_hashfn()

2006-05-31 Thread David Miller
From: "Brian F. G. Bidulock" <[EMAIL PROTECTED]> Date: Tue, 30 May 2006 23:55:26 -0600 > For example, it goes to great pains to permute upper order bits in > the local address, which for most connections will be a constant > value. Consider an apache server hosting thousands of virtual hosts. Th