On 2022-05-15, Tom Smyth <[email protected]> wrote: > Hi Stuart, > I have huge regard for you and all you contribute to OpenBSD and the community > Im going to clarify what I meant and what my experience with PMTU and > constrained MTUs behind > NAT, > My humble experience is that if we have a constrained MTU behind a NAT > Path MTU discovery from the server to the client fails because > > [website]--- public IP MTU 1500 bytes ----------[firewall/Nat] > private network MTU 1492 bytes-client > > so while MTU discovery may work outbound...(from client to the website) > the public website to the public IP has no way to discover the > constrained PMTU behind the nat...
There's no reason for this to fail? 1500 byte packet with DF set hits the firewall/nat box, route lookup, exit MTU is 1492, too big -> surely it just sends back frag needed? Even if you have a nat device with 1500 exit mtu and it then hits 1492 mtu on another device, similar case but the original frag-needed is sourced from a private address so it gets natted on the way out. There could be some specific cases where things aren't setup to allow this to work but there's nothing in general to cause it to fail. The problem case is when you have router hops on private addresses where there is *no* nat in the path in which case icmp is generated from the private address but there's nithing ti translate it, so that case you do often lose the message due to "no martian" packet filtering. > This corner case was discovered when I setup My ISP initially and I > had not many IP addresses many moons ago > It would be rare for a client behind a NAT to have a smaller MTU than > their public IP internet connection. > > Is my reasoning and analysis here correct ? > > > Regarding my comment >> PMTU cannot properly account for underlay restrictions Inside a VPN > > what I meant was, that if you set an MTU of 1500 on a VPN Tunnel interface > but in sending 1500 Bytes in an IP packet across the tunnel it > requires a the VPN encapsulated Packet + a Fragment Packet to be sent > also, (on the underlay interface) > the Router on the VPN wont sent a Fragment needed IP message to the > client because the MTU of the Tunnel was not exceeded > (but the MTU on the underlay was exceeded) This depends on the MTU stored in the route table entry used to send the packet over the vpn. With a separate tunnel interface the mtu on that interface and thus the route table can be set low enough that frag needed is sent. With standard flow-based IPsec the route used is normally the default route with either a standard ethernet MTU or a pppoe MTU. But if there's another route (route-based IPsec on OS which have this, or a dummyinterface such as is sometines used in combo with flow-based IPsec, for example a vether interface with a netmask that covers the "other side" of the IPsec tunnel as defined in the flow) with a lower MTU set on that interface, when the packet is attempted to be transmitted it will again see the lower MTU via the route table and be able to send frag-needed. It's easy to hit the blackhole case with IPsec tunnels *but* also often not so hard to avoid it. My preference is to try and set things up as much as possible so that you don't get PMTU blackholes or have to fragment the tunnel packet, but also clamp mss so that even if you do hit a blackhole there's no problem. There are some downsides to clamping MSS but they're relatively small and it's something done by almost every off-the-shelf home CPE so it's very very xommon on the internet. > I hope the clarifications helps and that im right or at least that I > learn something new :) > Thanks > Tom Smyth > > > > > > > > > On Sun, 15 May 2022 at 19:37, Stuart Henderson ><[email protected]> wrote: >> >> On 2022-05-15, Tom Smyth <[email protected]> wrote: >> > IP fragments on internet are avoided generally through PMTU discovery (mtu >> > path >> > discovery) but >> > PMTU does not work beyond a Nat (if a smaller MTU interface exists >> > behind a NAT then the smaller >> > MTU will not be discovered. >> >> That's not right, NAT doesn't break PMTU detection. >> >> > PMTU cannot properly account for underlay restrictions Inside a VPN >> >> Depends on the VPN type. For VPNs using a tunnel device (openvpn, >> WireGuard, gif/gre/l2tp etc, maybe route-based IPsec) then PMTU works >> like it would on another network type. Not nornally for flow-based IPsec >> though as the MTU is taken from the route (but it can be made to work >> with a dummy interface covering the VPN range with a lower MTU set in >> it). >> >> > > -- Please keep replies on the mailing list.
