Thank you, This (having unique VHID) was the solution.
I had considered originally that since each carp device is on its own VLAN, that would represent a unique broadcast domain and it wouldn't be violating anything - but without your suggestion I'm not sure I would have gone back to review that decision. I'm still a bit curious how it came to that. I did snoop if carp announcements were leaking from a tagged vlan onto the default network, but didn't see any sign of that. So maybe it was because the VLANs were riding on top of the same physical interface... but a lot less important now. Regards, Andrew On Tue, Aug 23, 2016 at 8:34 PM, John Jasen <[email protected]> wrote: > All your carp devices have the same VHID. As two share the same network, > that could cause problems. > > > > > On 08/23/2016 01:40 PM, Andrew Seguin wrote: > > Hi, > > > > I'm building up an OpenBSD router/firewall (migrating away from FreeBSD) > > but have been blocked by a behavior of carp in combination with VLANs > that > > I didn't expect or experience before. I'm hoping somebody could enlighten > > me a little bit about why carp floating IPs stop working when the carp > > status is master for the physical interface. > > > > > > Originally, there was a pair of FreeBSD systems (FW1 and FW2) where I had > > no issues with carp managed IPs. > > > > At the moment, one system is reinstalled with OpenBSD 5.9 (FW1), the > other > > remains with FreeBSD (FW2). > > > > The network is setup in such a way that the default vlan (1) is untagged, > > and this network is for all the network management. All other traffic > goes > > over tagged networks. The network switches we have simply work in this > way > > and so I can't make vlan 1 also a tagged interface to test the impact of > > such a configuration. > > > > As long as the OpenBSD system is not the master for the default / > untagged > > network associated to the physical network interface, the system will > > accept packets for its CARP IPs. > > > > When OpenBSD becomes master for the untagged network, it won't forward or > > respond (ping) to packets addressed to its floating IP. > > > > Configuration files for the physical interface (sk0) and a couple VLANs > (I > > run a dozen, but trimmed back to two for the purpose of this mail). > > > > # cat /etc/sysctl.conf > > net.inet.carp.allow=1 > > net.inet.carp.preempt=1 > > net.inet.ip.forwarding=1 > > > > # cat /etc/hostname.sk0 > > inet > > 10.1.0.2 255.255.255.0 NONE description "main link" > > inet 10.0.0.2 255.255.255.0 > > > > # cat /etc/hostname.carp1 > > vhid 1 pass password carpdev sk0 advskew 150 > > inet 10.1.0.1 255.255.255.0 > > inet alias 10.0.0.1 255.255.255.0 > > > > # cat /etc/hostname.vlan10 > > inet 10.10.0.2 255.255.255.0 NONE vlan 10 vlandev sk0 description > "Printer > > network" > > > > # cat /etc/hostname.carp10 > > vhid 1 > pass > > password carpdev vlan10 advskew 150 > > inet 10.10.0.1 255.255.255.0 > > > > # cat /etc/hostname.vlan50 > > inet 10.50.0.2 255.255.255.0 NONE vlan 50 vlandev sk0 description > "Wireless > > backbone" > > > > # cat /etc/hostname.carp50 > > vhid 1 pass password carpdev vlan50 advskew 150 > > inet 10.50.0.1 255.255.255.0 > > > > > > The other system has a similar configuration with the exception that IPs > > ending in .2 are .3 on FW2 and FW2 has advskew 100. > > > > > > If I make FW1 (OpenBSD) the master for vlan10 and vlan50 (ifconfig carp10 > > advskew 1; ifconfig carp50 advskew) but not for sk0, then it will forward > > packets between those two networks without problem and ping 10.10.0.1 > works > > fine. > > > > The moment I make it the master for sk0 (ifconfig carp1 advskew 1), it no > > longer forwards packets (between vlan10 and vlan50, vlan10 and the > untagged > > vlan) and it no longer responds to ping for any of the IPs associated to > > the carp interfaces from external systems (ping 10.10.0.2 works, ping > > 10.10.0.1 doesn't work) although from the local box it works (ping > > 10.10.0.1 from FW1 works). Output from ifconfig shows FW1 is the master > for > > all interfaces. > > > > Throughout, I am able to keep working with the box remotely as long as I > > logged in via the local subnet IP (ie: from a workstation with IP > > 10.10.0.50, I can ssh to 10.10.0.2). > > > > For testing ... while the FW1 (OpenBSD) is master for all interfaces, I > > used tcpdump and could see the packets arriving at the system only if I > > took the dump on sk0 or carp1. No packets show up on vlan10 or carp10 for > > the box. On vlan10 - I can see all traffic addressed to 10.10.0.2 without > > problem. On carp10 - I only see the "CARPv2-advertise" and arp > > request/response packets. > > > > To rule things out, I've kept the PF configuration as simple as possible > > for testing (simply 1 line: "pass"). > > > > I always made sure that the corresponding CARP interfaces were in a > backup > > state on FW2 (freebsd) and via tcpdump that packets weren't ending up > there > > by some accident of the switches. > > > > I've tried setting the subnet masks for the floating (carp) IP addresses > to > > be 255.255.255.255 - didn't change the behavior. > > > > I set net.inet.carp.log=7 - nothing is noted in /var/log/messages beyond > > the transitions (carp1: state transition: BACKUP -> MASTER; MASTER -> > > BACKUP). > > > > Since then, I'm out of ideas what to try and am turning to the mailing > list > > for help. > > > > I'm rather new to OpenBSD, but I reviewed the FAQ and searched on google, > > read man pages for carp, ifconfig, hostname.if, etc but didn't get any > new > > ideas. > > > > Any ideas or suggestions what else I might look at? > > > > Is this expected behavior or have I overlooked some configuration option? > > > > Thanks in advance, > > Andrew
