Bug#866094: linux-image-3.16.0: select() on network device loses packets at low data rates

[Jim Hague]

Package: linux-image-3.16.0
Version: linux-image-3.16.0-4-amd64
Severity: normal

Using the default Jessie kernel, we found that when capturing network packets
with libpcap using a loop

    pcap_dispatch(handle, -1, dispatcher, NULL);
    select(fd + 1, & fd_selected, 0, 0, &timeval);

where timeval = 2s, packets are missed if they arrive at low rates.

I have a small test capture program (attached) which shows pings
not being captured on an otherwise idle network connection.

The problem does not occur on later (e.g. 4.7) kernels.

We believe the following is the cause:

https://github.com/torvalds/linux/commit/da413eec729dae5dcb150e2eb34c5e7e5e4e1b49

Author:     Dan Collins <d...@dcollins.co.nz>
AuthorDate: Fri Dec 19 16:49:25 2014 +1300
Commit:     David S. Miller <da...@davemloft.net>
CommitDate: Mon Dec 22 15:41:15 2014 -0500

    packet: Fixed TPACKET V3 to signal poll when block is closed rather than
every packet

    Make TPACKET_V3 signal poll when block is closed rather than for every
    packet. Side effect is that poll will be signaled when block retire
    timer expires which didn't previously happen. Issue was visible when
    sending packets at a very low frequency such that all blocks are retired
    before packets are received by TPACKET_V3. This caused avoidable packet
    loss. The fix ensures that the signal is sent when blocks are closed
    which covers the normal path where the block is filled as well as the
    path where the timer expires. The case where a block is filled without
    moving to the next block (ie. all blocks are full) will still cause poll
    to be signaled.

    Signed-off-by: Dan Collins <d...@dcollins.co.nz>
    Signed-off-by: David S. Miller <da...@davemloft.net>



-- System Information:
Debian Release: 8.8
  APT prefers oldstable-updates
  APT policy: (500, 'oldstable-updates'), (500, 'oldstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.16.0-4-amd64 (SMP w/3 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)

#include <cstdio>
#include <cstdlib>
#include <iostream>

#include <pcap/pcap.h>
#include <sys/select.h>

void error(const char* msg)
{
    std::perror(msg);
    std::exit(1);
}

void usage()
{
    std::cerr << "Usage: test_pcap [-n|--non-blocking] next|dispatch|loop <interface>\n";
    std::exit(1);
}

void dispatcher(u_char *user, const struct pcap_pkthdr* h, const u_char* bytes)
{
    static int no_read = 0;
    std::cout << "Read packet " << ++no_read << std::endl;
}

int main(int ac, char *av[])
{
    bool non_blocking = false;

    if ( ac < 3 || ac > 4 )
        usage();

    ac--;
    av++;

    std::string arg(*av);

    if ( arg == "-n" || arg == "--non-blocking" )
    {
        non_blocking = true;
        ac--;
        av++;

        if ( ac < 2 )
            usage();
        arg = *av;
    }

    char errbuf[PCAP_ERRBUF_SIZE];
    pcap_t* handle = pcap_create(av[1], errbuf);
    if ( !handle )
        error(av[1]);

    if ( pcap_set_snaplen(handle, 65535) != 0 )
        error("snaplen");

    if ( pcap_activate(handle) < 0 )
        error("activate");

    if ( non_blocking && pcap_setnonblock(handle, 1, errbuf) < 0 )
        error("nonblock");

    int fd = pcap_get_selectable_fd(handle);
    if ( fd < 0 )
        error("selectable");

    fd_set fdset;
    FD_ZERO(&fdset);
    FD_SET(fd, &fdset);

    for(;;)
    {
        if ( arg == "next" )
        {
            struct pcap_pkthdr* hdr;
            const u_char* data;
            bool read_one;

            do
            {
                read_one = false;

                std::cout << "Reading..." << std::endl;
                switch (pcap_next_ex(handle, &hdr, &data))
                {
                case 1:
                    read_one = true;
                    dispatcher(nullptr, nullptr, nullptr);
                    continue;

                case 0:
                    std::cout << "Nothing read" << std::endl;
                    break;

                default:
                    error("packet read");
                    break;
                }
            } while ( read_one );
        }
        else if ( arg == "dispatch" )
        {
            std::cout << "Dispatching..." << std::endl;
            pcap_dispatch(handle, -1, dispatcher, nullptr);
        }
        else if ( arg == "loop" )
        {
            std::cout << "Looping..." << std::endl;
            pcap_loop(handle, -1, dispatcher, nullptr);
        }
        else
            usage();

        fd_set fd_selected = fdset;
        struct timeval tv;
        tv.tv_sec = 2;
        tv.tv_usec = 0;
        switch (select(fd + 1, &fd_selected, nullptr, nullptr, &tv))
        {
        case -1:
            switch(errno)
            {
            case EAGAIN:
            case EINTR:
                break;

            default:
                error("select");
                break;
            }

        default:
            break;
        }
    }
}