Ilan and Shanker,
I did as you asked.
Setup: Three nodes all connected to the same Ethernet hub.
Node 1: fd01:1b10:1000::1 is running 3.18.36
Node 2: fd01:1b10:1000::2 is running 2.6.23.12
Node 3: fd01:1b10:1000::3 is running 3.18.36 and generates the traffic with the
command "ping6 -I lan1 ff0e::1"
All three nodes are configured with the following script (IP6ADDR adjusted for
every node):
==================Begin==========================
#!/bin/bash
IP6ANYADDR=::/0
IP6BCAST=ff0e::1
KEY="0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3"
#IFNAME=radio
IFNAME=lan1
IP6ADDR=fd01:1b10:1000::X
PREFIX6=64
SPI=0x1
ip link set dev ${IFNAME} up
ip addr add ${IP6ADDR}/${PREFIX6} dev ${IFNAME}
echo "flush; spdflush;" | setkey -c
echo "add ${IP6ADDR} ${IP6BCAST} esp ${SPI} -m tunnel -E aes-cbc ${KEY};" |
setkey -c
echo "spdadd ${IP6ANYADDR} ${IP6BCAST} any -P in ipsec
esp/tunnel/${IP6ADDR}-${IP6BCAST}/require;" | setkey -c
echo "spdadd ${IP6ANYADDR} ${IP6BCAST} any -P out ipsec
esp/tunnel/${IP6ADDR}-${IP6BCAST}/require;" | setkey -c
===================End===========================
To capture the debug information I used the following script:
==================Begin==========================
uname -a
echo "tcpdump -lni lan1 -c 5"
tcpdump -lni lan1 -c 5
echo "cat /proc/net/xfrm_stat"
cat /proc/net/xfrm_stat
echo "ip -s xfrm state"
ip -s xfrm state
echo "ip -s xfrm policy"
ip -s xfrm policy
===================End===========================
Debug log from Node 1, Receiver running 3.6.18:
==================Begin==========================
Linux node-0-001 3.18.36-node-5875 #1 Tue Jul 12 14:00:52 CEST 2016 i686 i686
i686 GNU/Linux
tcpdump -lni lan1 -c 5
12:03:09.580943 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x69),
length 136
12:03:10.581006 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x6a),
length 136
12:03:11.581068 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x6b),
length 136
12:03:12.581131 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x6c),
length 136
12:03:13.581193 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x6d),
length 136
cat /proc/net/xfrm_stat
XfrmInError 0
XfrmInBufferError 0
XfrmInHdrError 0
XfrmInNoStates 0
XfrmInStateProtoError 0
XfrmInStateModeError 0
XfrmInStateSeqError 0
XfrmInStateExpired 0
XfrmInStateMismatch 0
XfrmInStateInvalid 0
XfrmInTmplMismatch 0
XfrmInNoPols 0
XfrmInPolBlock 0
XfrmInPolError 0
XfrmOutError 0
XfrmOutBundleGenError 0
XfrmOutBundleCheckError 0
XfrmOutNoStates 0
XfrmOutStateProtoError 0
XfrmOutStateModeError 0
XfrmOutStateSeqError 0
XfrmOutStateExpired 0
XfrmOutPolBlock 0
XfrmOutPolDead 0
XfrmOutPolError 0
XfrmFwdHdrError 0
XfrmOutStateInvalid 0
XfrmAcquireError 0
ip -s xfrm state
src fd01:1b10:1000::1 dst ff0e::1
proto esp spi 0x00000001(1) reqid 0(0x00000000) mode tunnel
replay-window 0 seq 0x00000000 flag (0x00000000)
enc cbc(aes) 0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3 (192
bits)
sel src ::/0 dst ::/0 uid 0
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 12:01:44 use -
stats:
replay-window 0 replay 0 failed 0
ip -s xfrm policy
src ::/0 dst ff0e::1/128 uid 0
dir out action allow index 41 priority 2147483648 share any flag
(0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 12:01:44 use -
tmpl src fd01:1b10:1000::1 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
src ::/0 dst ff0e::1/128 uid 0
dir fwd action allow index 34 priority 2147483648 share any flag
(0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 12:01:44 use -
tmpl src fd01:1b10:1000::1 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
src ::/0 dst ff0e::1/128 uid 0
dir in action allow index 24 priority 2147483648 share any flag
(0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 12:01:44 use -
tmpl src fd01:1b10:1000::1 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
===================End===========================
Debug log from Node 2, Receiver running 2.6.23.12 (Note: 2.6.23 did not have
/proc/net/xfrm_stat):
==================Begin==========================
Linux node-0-001 2.6.23.12-node-4908 #3 Mon Dec 15 17:28:03 CET 2014 i686
GNU/Linux
tcpdump -lni lan1 -c 5
11:54:06.506723 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x75),
length 136
11:54:06.506723 IP6 fd01:1b10:1000::3 > ff0e::1: ICMP6, echo request, seq 117,
length 64
11:54:07.506696 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x76),
length 136
11:54:07.506696 IP6 fd01:1b10:1000::3 > ff0e::1: ICMP6, echo request, seq 118,
length 64
11:54:08.506731 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x77),
length 136
cat /proc/net/xfrm_stat
ip -s xfrm state
src fd01:1b10:1000::2 dst ff0e::1
proto esp spi 0x00000001(1) reqid 0(0x00000000) mode tunnel
replay-window 0 seq 0x00000000 flag (0x00000000)
enc cbc(aes) 0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3 (192
bits)
sel src ::/0 dst ::/0 uid 0
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
8320(bytes), 80(packets)
add 2016-07-21 11:52:48 use 2016-07-21 11:52:49
stats:
replay-window 0 replay 0 failed 0
ip -s xfrm policy
src ::/0 dst ff0e::1/128 uid 0
dir in action allow index 24 priority 2147483648 share any flag
0x00000000
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 11:52:48 use -
tmpl src fd01:1b10:1000::2 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask ffffffff auth-mask ffffffff comp-mask ffffffff
src ::/0 dst ff0e::1/128 uid 0
dir out action allow index 41 priority 2147483648 share any flag
0x00000000
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 11:52:48 use -
tmpl src fd01:1b10:1000::2 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask ffffffff auth-mask ffffffff comp-mask ffffffff
src ::/0 dst ff0e::1/128 uid 0
dir fwd action allow index 34 priority 2147483648 share any flag
0x00000000
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 11:52:48 use -
tmpl src fd01:1b10:1000::2 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask ffffffff auth-mask ffffffff comp-mask ffffffff
===================End===========================
Debug log from Node 3, Sender running 3.6.18:
==================Begin==========================
Linux node-0-005 3.18.36-node-5875 #1 Tue Jul 12 14:00:52 CEST 2016 i686 i686
i686 GNU/Linux
tcpdump -lni lan1 -c 5
10:27:07.369198 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5b),
length 136
10:27:08.369261 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5c),
length 136
10:27:09.369323 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5d),
length 136
10:27:10.369386 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5e),
length 136
10:27:11.369448 IP6 fd01:1b10:1000::3 > ff0e::1: ESP(spi=0x00000001,seq=0x5f),
length 136
cat /proc/net/xfrm_stat
XfrmInError 0
XfrmInBufferError 0
XfrmInHdrError 0
XfrmInNoStates 0
XfrmInStateProtoError 0
XfrmInStateModeError 0
XfrmInStateSeqError 0
XfrmInStateExpired 0
XfrmInStateMismatch 0
XfrmInStateInvalid 0
XfrmInTmplMismatch 0
XfrmInNoPols 0
XfrmInPolBlock 0
XfrmInPolError 0
XfrmOutError 0
XfrmOutBundleGenError 0
XfrmOutBundleCheckError 0
XfrmOutNoStates 0
XfrmOutStateProtoError 0
XfrmOutStateModeError 0
XfrmOutStateSeqError 0
XfrmOutStateExpired 0
XfrmOutPolBlock 0
XfrmOutPolDead 0
XfrmOutPolError 0
XfrmFwdHdrError 0
XfrmOutStateInvalid 0
XfrmAcquireError 0
ip -s xfrm state
src fd01:1b10:1000::3 dst ff0e::1
proto esp spi 0x00000001(1) reqid 0(0x00000000) mode tunnel
replay-window 0 seq 0x00000000 flag (0x00000000)
enc cbc(aes) 0x7bef6ecaf06d29ef55b24aca6e19964b332e02e75be676a3 (192
bits)
sel src ::/0 dst ::/0 uid 0
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
9880(bytes), 95(packets)
add 2016-07-21 10:25:32 use 2016-07-21 10:25:37
stats:
replay-window 0 replay 0 failed 0
ip -s xfrm policy
src ::/0 dst ff0e::1/128 uid 0
dir out action allow index 41 priority 2147483648 share any flag
(0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 10:25:32 use 2016-07-21 10:27:11
tmpl src fd01:1b10:1000::3 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
src ::/0 dst ff0e::1/128 uid 0
dir fwd action allow index 34 priority 2147483648 share any flag
(0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 10:25:32 use -
tmpl src fd01:1b10:1000::3 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
src ::/0 dst ff0e::1/128 uid 0
dir in action allow index 24 priority 2147483648 share any flag
(0x00000000)
lifetime config:
limit: soft (INF)(bytes), hard (INF)(bytes)
limit: soft (INF)(packets), hard (INF)(packets)
expire add: soft 0(sec), hard 0(sec)
expire use: soft 0(sec), hard 0(sec)
lifetime current:
0(bytes), 0(packets)
add 2016-07-21 10:25:32 use -
tmpl src fd01:1b10:1000::3 dst ff0e::1
proto esp spi 0x00000000(0) reqid 0(0x00000000) mode tunnel
level required share any
enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
===================End===========================
A difference between 2.6.23 and 3.6.18 is the value of the Masks in the
policies:
2.6.23: enc-mask ffffffff auth-mask ffffffff comp-mask ffffffff
3.6.18: enc-mask 00000000 auth-mask 00000000 comp-mask 00000000
I have no idea, what these masks mean. Is this important?
Regards
Joerg
> -----Ursprüngliche Nachricht-----
> Von: Ilan Tayari [mailto:il...@mellanox.com]
> Gesendet: Dienstag, 19. Juli 2016 17:04
> An: Pommnitz Jörg
> Cc: netdev@vger.kernel.org
> Betreff: RE: IPv6 IPSec incompatibilities between 2.6.23 and 3.6.18 (and
> probably later)
>
> > On the receiving side (e.g. fd01:1b10:1000::1) I see the decrypted
> packets with
> > the 2.6.23 kernel:
> > but NOT with the newer kernel:
>
> Hi Joerg,
>
> First steps to debug this would be:
> cat /proc/net/xfrm_stat
> ip -s xfrm state
> ip -s xfrm policy
>
> First command will show some error accounting, which can point to the
> culprit code.
> Second and third command will show existing objects, and some statistics
> like when the last packet was used with them.
>
> Last thing - for your safety you should keep those session keys private.
>
> Ilan.
________________________________
Industrieanlagen-Betriebsgesellschaft mbH
Sitz der Gesellschaft: Ottobrunn, Registergericht: Amtsgericht München, HRB 5499
Geschäftsführung: Prof. Dr.-Ing. Rudolf F. Schwarz
Vorsitzender des Aufsichtsrats: RA Engelbert Kupka MdL a.D.
