Hi,
I found a bug while working on a route server implementation based on
OpenBGPD. I have a IPv6 session from OpenBGPD 4.4 (on OpenBSD 4.4,
routeertnix) to Quagga 0.99.5 (laborantix).
I have multiple IPv4 peers, and multiple IPv6 peers in the setup. When I
start the BGP daemon, everything starts up nicely. All sessions come up.
When I clear a IPv6 peering session, the connection shifts to the
Idle state. When I look in the log, I can see it connect and establish a
connection, but break as soon as a mistery update gets send out.
While looking in to the problem, we found out that OpenBGPD sends a
empty UPDATE, on which quagga responds by terminating the process.
The /var/log/daemon log shows the following:
Jan 29 16:07:39 routeertnix bgpd[31121]: neighbor
2001:db8:1::a506:5502:1 (laborantix ipv6): state change Idle -> Connect,
reason: Start
Jan 29 16:07:39 routeertnix bgpd[31121]: neighbor
2001:db8:1::a506:5502:1 (laborantix ipv6): state change Connect ->
OpenSent, reason: Connection opened
Jan 29 16:07:39 routeertnix bgpd[31121]: neighbor
2001:db8:1::a506:5502:1 (laborantix ipv6): state change OpenSent ->
OpenConfirm, reason: OPEN message received
Jan 29 16:07:39 routeertnix bgpd[31121]: neighbor
2001:db8:1::a506:5502:1 (laborantix ipv6): state change OpenConfirm ->
Established, reason: KEEPALIVE message received
Jan 29 16:07:40 routeertnix bgpd[16710]: neighbor
2001:db8:1::a506:5502:1 (laborantix ipv6) AS65502: update
2001:db8:97::/64 via 2001:db8:1::a506:5502:1
Jan 29 16:07:40 routeertnix bgpd[25774]: nexthop 2001:db8:1::a506:5502:1
now valid: directly connected
Jan 29 16:07:40 routeertnix bgpd[31121]: neighbor
2001:db8:1::a506:5502:1 (laborantix ipv6): received notification: error
in UPDATE message, network unacceptable
Jan 29 16:07:40 routeertnix bgpd[31121]: neighbor
2001:db8:1::a506:5502:1 (laborantix ipv6): state change Established ->
Idle, reason: NOTIFICATION received
While doing a tcpdump we found the following packets leading to a
NOTIFICATION. As you can see, frame 19 is an empty UPDATE packet.
Frame 18 (167 bytes on wire, 167 bytes captured)
Arrival Time: Jan 29, 2009 15:54:28.184019000
[Time delta from previous packet: 0.807505000 seconds]
[Time since reference or first frame: 1.009967000 seconds]
Frame Number: 18
Packet Length: 167 bytes
Capture Length: 167 bytes
[Frame is marked: False]
[Protocols in frame: eth:ipv6:tcp:bgp]
Ethernet II, Src: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec), Dst:
00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2)
Destination: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2)
Address: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address
(factory default)
Source: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec)
Address: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address
(factory default)
Type: IPv6 (0x86dd)
Internet Protocol Version 6
Version: 6
Traffic class: 0x00
Flowlabel: 0x00000
Payload length: 113
Next header: TCP (0x06)
Hop limit: 1
Source address: 2001:db8:1::a506:5502:1 (2001:db8:1::a506:5502:1)
Destination address: 2001:db8:1::a500:6777:1 (2001:db8:1::a500:6777:1)
Transmission Control Protocol, Src Port: 179 (179), Dst Port: 10379
(10379), Seq: 84, Ack: 229, Len: 81
Source port: 179 (179)
Destination port: 10379 (10379)
Sequence number: 84 (relative sequence number)
[Next sequence number: 165 (relative sequence number)]
Acknowledgement number: 229 (relative ack number)
Header length: 32 bytes
Flags: 0x18 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 5712
Checksum: 0x626e [incorrect, should be 0xc328 (maybe caused by
checksum offloading?)]
Options: (12 bytes)
NOP
NOP
Timestamps: TSval 2877490800, TSecr 257498766
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 17]
[The RTT to ACK the segment was: 0.807505000 seconds]
Border Gateway Protocol
UPDATE Message
Marker: 16 bytes
Length: 81 bytes
Type: UPDATE Message (2)
Unfeasible routes length: 0 bytes
Total path attribute length: 58 bytes
Path attributes
ORIGIN: IGP (4 bytes)
Flags: 0x40 (Well-known, Transitive, Complete)
0... .... = Well-known
.1.. .... = Transitive
..0. .... = Complete
...0 .... = Regular length
Type code: ORIGIN (1)
Length: 1 byte
Origin: IGP (0)
AS_PATH: 65502 (7 bytes)
Flags: 0x40 (Well-known, Transitive, Complete)
0... .... = Well-known
.1.. .... = Transitive
..0. .... = Complete
...0 .... = Regular length
Type code: AS_PATH (2)
Length: 4 bytes
AS path: 65502
AS path segment: 65502
Path segment type: AS_SEQUENCE (2)
Path segment length: 1 AS
Path segment value: 65502
MULTI_EXIT_DISC: 0 (7 bytes)
Flags: 0x80 (Optional, Non-transitive, Complete)
1... .... = Optional
.0.. .... = Non-transitive
..0. .... = Complete
...0 .... = Regular length
Type code: MULTI_EXIT_DISC (4)
Length: 4 bytes
Multiple exit discriminator: 0
COMMUNITIES: 6777:6777 (7 bytes)
Flags: 0xc0 (Optional, Transitive, Complete)
1... .... = Optional
.1.. .... = Transitive
..0. .... = Complete
...0 .... = Regular length
Type code: COMMUNITIES (8)
Length: 4 bytes
Communities: 6777:6777
Community: 6777:6777
Community AS: 6777
Community value: 6777
MP_REACH_NLRI (33 bytes)
Flags: 0x80 (Optional, Non-transitive, Complete)
1... .... = Optional
.0.. .... = Non-transitive
..0. .... = Complete
...0 .... = Regular length
Type code: MP_REACH_NLRI (14)
Length: 30 bytes
Address family: IPv6 (2)
Subsequent address family identifier: Unicast (1)
Next hop network address (16 bytes)
Next hop: 2001:db8:1::a506:5502:1 (16)
Subnetwork points of attachment: 0
Network layer reachability information (9 bytes)
2001:db8:97::/64
MP Reach NLRI prefix length: 64
MP Reach NLRI prefix: 2001:db8:97::
Frame 19 (124 bytes on wire, 124 bytes captured)
Arrival Time: Jan 29, 2009 15:54:28.185078000
[Time delta from previous packet: 0.001059000 seconds]
[Time since reference or first frame: 1.011026000 seconds]
Frame Number: 19
Packet Length: 124 bytes
Capture Length: 124 bytes
[Frame is marked: False]
[Protocols in frame: eth:ipv6:tcp:bgp]
Ethernet II, Src: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2), Dst:
00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec)
Destination: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec)
Address: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address
(factory default)
Source: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2)
Address: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address
(factory default)
Type: IPv6 (0x86dd)
Internet Protocol Version 6
Version: 6
Traffic class: 0x00
Flowlabel: 0x809f0
Payload length: 70
Next header: TCP (0x06)
Hop limit: 1
Source address: 2001:db8:1::a500:6777:1 (2001:db8:1::a500:6777:1)
Destination address: 2001:db8:1::a506:5502:1 (2001:db8:1::a506:5502:1)
Transmission Control Protocol, Src Port: 10379 (10379), Dst Port: 179
(179), Seq: 229, Ack: 165, Len: 38
Source port: 10379 (10379)
Destination port: 179 (179)
Sequence number: 229 (relative sequence number)
[Next sequence number: 267 (relative sequence number)]
Acknowledgement number: 165 (relative ack number)
Header length: 32 bytes
Flags: 0x18 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 16384
Checksum: 0xe291 [correct]
Options: (12 bytes)
NOP
NOP
Timestamps: TSval 257498768, TSecr 2877490800
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 18]
[The RTT to ACK the segment was: 0.001059000 seconds]
Border Gateway Protocol
UPDATE Message
Marker: 16 bytes
Length: 38 bytes
Type: UPDATE Message (2)
Unfeasible routes length: 0 bytes
Total path attribute length: 0 bytes
Frame 20 (107 bytes on wire, 107 bytes captured)
Arrival Time: Jan 29, 2009 15:54:28.185337000
[Time delta from previous packet: 0.000259000 seconds]
[Time since reference or first frame: 1.011285000 seconds]
Frame Number: 20
Packet Length: 107 bytes
Capture Length: 107 bytes
[Frame is marked: False]
[Protocols in frame: eth:ipv6:tcp:bgp]
Ethernet II, Src: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec), Dst:
00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2)
Destination: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2)
Address: 00:06:5b:8d:1a:c2 (00:06:5b:8d:1a:c2)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address
(factory default)
Source: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec)
Address: 00:06:5b:8d:1a:ec (00:06:5b:8d:1a:ec)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address
(factory default)
Type: IPv6 (0x86dd)
Internet Protocol Version 6
Version: 6
Traffic class: 0x00
Flowlabel: 0x00000
Payload length: 53
Next header: TCP (0x06)
Hop limit: 1
Source address: 2001:db8:1::a506:5502:1 (2001:db8:1::a506:5502:1)
Destination address: 2001:db8:1::a500:6777:1 (2001:db8:1::a500:6777:1)
Transmission Control Protocol, Src Port: 179 (179), Dst Port: 10379
(10379), Seq: 165, Ack: 267, Len: 21
Source port: 179 (179)
Destination port: 10379 (10379)
Sequence number: 165 (relative sequence number)
[Next sequence number: 186 (relative sequence number)]
Acknowledgement number: 267 (relative ack number)
Header length: 32 bytes
Flags: 0x18 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 5712
Checksum: 0x6232 [incorrect, should be 0xbdaa (maybe caused by
checksum offloading?)]
Options: (12 bytes)
NOP
NOP
Timestamps: TSval 2877490800, TSecr 257498768
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 19]
[The RTT to ACK the segment was: 0.000259000 seconds]
Border Gateway Protocol
NOTIFICATION Message
Marker: 16 bytes
Length: 21 bytes
Type: NOTIFICATION Message (3)
Error code: UPDATE Message Error (3)
Error subcode: Invalid Network Field (10)
When the NOTIFICATION is received, the peer is set back to the state
Idle, where the process starts again. The only way to break the cicle is
to restart the entire OpenBGPD daemon.
Kind regards,
Arnoud Vermeer
