On Thu, 2024-11-07 07:41:26 -0800, Avram Dorfman wrote: > Package: bootp > Version: 2.4.3-19.1 > Severity: grave > Justification: renders package unusable > X-Debbugs-Cc: dorf...@est.org > > Dear Maintainer, > > *** Reporter, please consider answering these questions, where appropriate *** > > * What led up to the situation? > We tried to set up a bootpd/bootpc testbed to implement a solution > for a customer who requires bootp support. We installed bootp on one > Debian 12 host, and bootpc on another. > > the bootpc process never gets responses from the bootp server, > despite the fact that the responses are sent and received by the > client's NIC. > > * What exactly did you do (or not do) that was effective (or > ineffective)? > > Packet inspection on server & client. Also ran dropwatch on client. > We can see the kernel dropping the replies. With tcpdump, we can > see that the replies use a DST IP ADDR value of the IP address that > is not yet configured on the client. It is normal behavior for the > kernel to drop packets for non-local IP destinations. > > We tried removing the :ip: tag from the client's bootptab entry. > This resulted in DST IP ADDR changing to 0.0.0.0, which is valid, > and bootpc process received and processed the reply. However this > is not an adequate solution, since it does not give the client an > IP address. > > We also tried using the --serverbcast option on the client. This > results in bootpd sending replies to the subnet broadcast address. > This fails for the same reason: The client does not have an IP > configuration yet, so the kernel does not consider the subnet > broadcast address local. > > We also tried using iptables to NAT addres--map the replies to > 255.255.255.255. This works, the kernel delivers the reply to the > bootpc process, and it processes them. However it is a hack that > should not be necessary. > > * What was the outcome of this action? > > There is no possible configuration of bootpd and the corresponding > bootpc packages on Linux that successfully communicate an IP > configuration from server to client. > > * What outcome did you expect instead? > > - In unicast (default) mode, bootpd should still use a global broadcast IP > address as the IP destination. This mode should only be a unicast > at the MAC layer > - In broadcast mode, bootpd should use a global rather than subnet IP > broadcast address as the IP destination. > - If there is a valid use case for these destination IP choices, > there should be configuration or command line options to instruct > the server to use global broadcast destination addresses > > *** End of the template - remove these template lines *** > > > -- System Information: > Debian Release: 12.2 > APT prefers stable-updates > APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, > 'stable') > Architecture: arm64 (aarch64) > > Kernel: Linux 6.1.0-13-arm64 (SMP w/1 CPU thread) > Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not > set > Shell: /bin/sh linked to /usr/bin/dash > Init: systemd (via /run/systemd/system) > LSM: AppArmor: enabled > > Versions of packages bootp depends on: > ii libc6 2.36-9+deb12u3 > ii netbase 6.4 > ii update-inetd 4.53 > > bootp recommends no packages. > > bootp suggests no packages. > > -- Configuration Files: > /etc/bootptab changed: > .est-default:\ > :sm=255.255.255.0:\ > :gw=10.0.1.1:\ > :ds=10.0.1.1:\ > :hn: > > gw-est:ht=1:ha=3c5cf167d2ad:ip=10.0.1.1:tc=.est-default > client-ad02-est:\ > :ht=1:\ > :ha=0xea4a1fad0002:\ > :ip=10.0.1.198:\ > :tc=.est-default: > > > -- no debconf information
Avram, thanks for asking on stackexchange about this bug. [1] https://unix.stackexchange.com/questions/786133/get-bootpd-bootpc-to-talk-to-each-other-successfully At [1], it reads: Get bootpd & bootpc to talk to each other successfully The current Debian bootpc and bootpd (from bootp) do not appear to actually work together in today's Linux kernel environment. There seems to be a chicken-egg problem; bootpd sends the replies as unicast UDP packets to the as-yet-unconfigured IP address. The client's kernel then drops them w/out delivering to the bootpc client's socket, b/c that IP address is not (yet) a valid local address on the host. How did this ever work? Is there a kernel parameter or other modification that can get the kernel to give these packets to the bootpc process? Is there a configuration for bootpd that will get it to use all 1's or all 0's destination IP addresses instead of the as-yet-unconfigured client IP unicast address? We have a prospective customer with a large bootp non-DHCP infrastructure. Old-school bootp support is one of their requirements. Issue Details bootpd sends its reply packets as unicast packets to the bootpc client's MAC address, using a destination IP address value of the address that it is also in the payload telling the client to configure itself with. The kernel then drops these packets, rather than deliver them to the bootpc process that requested them. bootpc has correctly opened a listening socket for 0.0.0.0:68 0.0.0.0:* (see netstat output below) I've verified this analysis in several ways: I've run tcpdump and I can see the replies getting to the NIC I've run dropwatch and see the packets getting dropped with a reason of IPINADDRERROR, which basically means "invalid IP address" I can trick bootpd into using 0.0.0.0 as the destination IP address by omitting and actual IP assignment; when I do this, bootpc gets the responses and processes them. However this doesn't help b/c then the client doesn't get an IP address I've tried adding the IP address to the interface while bootpc is making requests. Once I add it, the next reply gets to the process. I've also tried the bootpc --serverbcast option. This fails for a similar reason: bootpd sends the replies to the subnet broadcast address (e.g. 10.0.43.255) Since the IP address & subnet mask aren't configured on an interface yet, the kernel has no reason to consider this a valid address for itself. Here is our current bootptab configuration: .vs-default:\ :sm=255.255.254.0:\ :gw=10.0.42.1:\ :ds=10.0.42.1:\ :hn: client-ad02-vs:\ ht=1:\ ha=0xea4a1fad0002:\ ip=10.0.42.31:\ tc=.vs-default: Netstat output showing bootpc's listening socket: $ sudo netstat -unlp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name udp 0 0 0.0.0.0:68 0.0.0.0:* 2295/bootpc Hacky Workaround I've come up with a work-around that feels hacky. In particular, I think it leverages an unintended behavior, maybe we can't count on it to work in the future: the iptables conntrack module has a match ctstate with a value of DNAT. This can be used to accept packets that have an unrecognized destination IP address I don't think it is intended to match arbitrary DST IP's: It can only be used with packets that were NAT'd by the same host's kernel I think its purpose is to match packets that have had their address rewritten (which we are not doing), but it is a "cheap" implementation that ignores the NAT mapping table instead of verifying that there is an entry that matches this specific packet. I'm concerned this will either be removed, or re-written to be less promiscuous in the future To get this to work, I have to port-map these packets to their existing DST port, just to get NAT connection state data onto the packet, otherwise --ctstate DNAT doesn't apply to the packet. Background/Due Diligence Our testbed is using is Debian 12 on Linux 6.1.0. Our testbed hosts are from UTM Debian 12 off-the-shelf VM images bootpd has very few cli options, none of which involve replies, ports or addresses. The man page doesn't mention "broadcast" or "unicast", and references to "address" or "destination" are few & not relevant. Our actual product is Debian 12 on a custom Linux 5.19.9 kernel (and behaves the same way) IP connectivity is fully working between the testbed client and server bootpc refuses to send requests unless the following are true: There is no routable IP address already on the interface (169.254/16 is allowed, although I've tried it without too) There is a 0/0 default route pointing to the interface where the bootp server is expected Otherwise it says network unreachable Note: specifying --dev <iface> on the bootpc command line doesn't help with this I'm still working on evaluating bootptab configuration options to see if there are any that affect this. Haven't found a good reference for them, the man bootptab man page is very terse about them. Alternate Workaround I've been able to get this to work with a less convoluted iptables solution: I added a NAT rule that matches UDP port 68 (bootpc), and maps the destination IP address to 255.255.255.255 (and keeps the same UDP port). This works, but I consider this hacky. Clearly this isn't how the protocol is meant to work, so I'd prefer a "real" solution if possible. networkingip 1 Answer > There seems to be a chicken-egg problem; bootpd sends the replies as > unicast UDP packets to the as-yet-unconfigured IP address. The client's > kernel then drops them w/out delivering to the bootpc client's socket, b/c > that IP address is not (yet) a valid local address on the host. > > How did this ever work? DHCP behaves the same way; most DHCP clients, and I expect also BOOTP clients, use raw sockets to receive whole IP packets before the usual kernel processing (as well as to send IP packets while not yet having any IP address to send them from). The client software itself builds the full IP and UDP header when sending the discover or request, and then uses BPF¹ to filter inbound packets instead of the usual bind()-to-local-port. (Among other things, this also bypasses iptables inbound filtering and UDP checksum fixups.) > Is there a kernel parameter or other modification that can get the kernel > to give these packets to the bootpc process? The client should be rewritten to use raw sockets (AF_PACKET, PF_LINK, or similar). For example, systemd-networkd uses socket(AF_PACKET, SOCK_DGRAM, ...) like in this source. The dhcpcd client uses AF_LINK on some systems, AF_PACKET on others, and likewise filters packets using BPF¹ instead of having the kernel do it. To make things simpler, some parts could be handled via libpcap. ¹ (This is "classic" BPF – Berkeley Packet Filter – rather than modern eBPF, so it is not Linux-specific and doesn't require any unusual features.) Is there a configuration for bootpd that will get it to use all 1's or all 0's destination IP addresses instead of the as-yet-unconfigured client IP unicast address? The server should honor a "wants broadcast" flag in the BOOTP or DHCP request. Debian's bootpc client sets it when you pass --serverbcast (because it doesn't use raw sockets apparently): /* Server needs to broadcast for me to see it */ if (broadcast || givenhwaddr) bootp_xmit->bp_flags |= htons(BPFLAG_BROADCAST); But the bootp server only honors this flag when compiled with the DHCP option that Debian patches into it: #ifdef DHCP /* * This code is placed here, because otherwise the siaddr * will not be found... */ if (ntohs(bp->bp_flags) & 0x8000 && bp->bp_giaddr.s_addr==0) dst.s_addr = INADDR_BROADCAST; #endif If you are trying to integrate your new Bootp server into a network existing Bootp clients, then ISC dhcpd might work better for you as it seems to have a Bootp compatibility mode. (Yes, the one that's now EOL, but at least it's less EOL than Debian's bootp server – which seems to have been last touched in 1995). [grawity] Thank you - so the basic answer is something like "No, this never worked." I haven't found a DHCP client with true bootp capability yet, but we did find that udhcpc "works." It has a static default lease time built in, which prevents it from chocking when there is no lease time offered, and ultimately it does receive the reply and configure the interface. – [Avrama]
