Hi,
I have two firewalls in a carp failover setup, but the failover does not work
as expected...
The problem happens when I reboot the backup firewall (while in backup state).
Just after the reboot, I have these entries in dmesg :
carp0: state transition: BACKUP -> MASTER
carp1: state transition: BACKUP -> MASTER
carp0: state transition: MASTER -> BACKUP
carp1: state transition: MASTER -> BACKUP
Why would there be no mention of carp2?
And no corresponding entries on the master?
States are consistent (all backup on backup, and all master on master), but
forwarded connections hang, until I force back the master with this :
sudo ifconfig -g carp carpdemote 128
sudo ifconfig -g carp -carpdemote 128
Between these two commands, on the backup firewall, I see traffic coming from
WAN and DMZ, but almost nothing from LAN, so it may be related to the LAN
switch. I cannot see what the problem is though...
Here is the setup :
On both firewalls :
- em0 is connected to WAN
- em1 is connected to LAN
- em2 is connected to DMZ
- em3 is interconnected with a crossover cable, used for pfsync and rdist
WAN and DMZ connections are on the same switch, but on different untagged VLANs
(Procurve 2524)
LAN is on a separate layer 3 switch (Procurve 5300xl)
Another strange behavior :
With tcpdump, on the backup, I can see this traffic :
- on em1 and em2, I see only carp advertisements to the configured unicast IP
address and physical MAC address
- on em3, I see only pfsync packets
- but on em0, I see carp advertisements, but also a lot of traffic from the
ISP router's MAC, to the virtual MAC (00:00:5e:00:01:01)
Which situation is normal? (em0 with lots of packets, or em1/em2 with only carp
advertisements)
The only difference I see :
- on em0, both firewalls and the ISP router are connected to the switch
- on em1, both firewalls are connected to the L3 switch, which is also the
router
- on em2, there is no router, the firewalls communicate directly with hosts
connected on the switch
Common configuration (public addresses anonymized, but the network sizes are
correct) :
/etc/mygate
192.0.2.1
/etc/sysctl.conf
net.inet.carp.preempt=1
net.inet.ip.forwarding=1
/etc/pf.conf (excerpt only)
ext_if = "em0"
ext_if_carp = "carp0"
int_if = "em1"
int_if_carp = "carp1"
dmz_if = "em2"
dmz_if_carp = "carp2"
sync_if = "em3"
set skip on lo
set skip on $sync_if
pass quick on { $int_if, $ext_if, $dmz_if } inet proto carp keep state (no-sync)
Firewall A (expected to be always master) :
OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
/etc/hostname.em0
inet 192.168.3.9/30
/etc/hostname.em1
inet 192.168.3.1/29
!route add 192.168.0.0/16 192.168.3.5
!route add 172.16.0.0/12 192.168.3.5
/etc/hostname.em2
inet 192.168.3.13/30
/etc/hostname.em3
inet 192.168.3.17 255.255.255.252
/etc/hostname.carp0
advskew 0 carpdev em0 carppeer 192.168.3.10 pass secret1 state master vhid 1
inet 192.0.2.2/28
alias 192.0.2.3/32
alias 192.0.2.4/32
alias 192.0.2.5/32
/etc/hostname.carp1
advskew 0 carpdev em1 carppeer 192.168.3.4 pass secret2 state master vhid 2
inet 192.168.3.6/32
/etc/hostname.carp2
advskew 0 carpdev em2 carppeer 192.168.3.14 pass secret3 state master vhid 3
inet 192.0.2.17/28
alias 192.0.2.29/32
/etc/hostname.pfsync0
up
syncdev em3
syncpeer 192.168.3.18
Firewall B (expected to be always backup) :
OpenBSD 5.6 (GENERIC.MP) #5: Thu Dec 11 09:51:08 CET 2014
[email protected]:/binpatchng/work-binpatch56-amd64/src/sys/arch/amd64/compile/GENERIC.MP
/etc/hostname.em0
inet 192.168.3.10/30
/etc/hostname.em1
inet 192.168.3.4/29
!route add 192.168.0.0/16 192.168.3.5
!route add 172.16.0.0/12 192.168.3.5
/etc/hostname.em2
inet 192.168.3.14/30
/etc/hostname.em3
inet 192.168.3.18/30
/etc/hostname.carp0
advskew 200 carpdev em0 carppeer 192.168.3.9 pass secret1 state backup vhid 1
inet 192.0.2.2/28
alias 192.0.2.3/32
alias 192.0.2.4/32
alias 192.0.2.5/32
/etc/hostname.carp1
advskew 200 carpdev em1 carppeer 192.168.3.1 pass secret2 state backup vhid 2
inet 192.168.3.6/32
/etc/hostname.carp2
advskew 200 carpdev em2 carppeer 192.168.3.13 pass secret3 state backup vhid 3
inet 192.0.2.17/28
alias 192.0.2.29/32
/etc/hostname.pfsync0
up
syncdev em3
syncpeer 192.168.3.17
This message is already long, but if any other information would be helpful, I
would be glad to provide it.
Any help or suggestion is appreciated.
Thank you!
Sebastien
