On Sat, Nov 11, 2006 at 03:21:17PM -0800, Jonathan Day ([EMAIL PROTECTED]) wrote: > > --- Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > You can use netchannels, which were designed for > > exactly that kind of > > load. > > Actually, netchannels are a mechanism I've been > looking at intensely, as a way to simplify this and > keep it sane, without loosing performance. > > > You need to process some headers anyway - to select > > appropriate socket > > or netchannel or whatever. > > I was planning on cheating there. I've revised the > method slightly, based on previous comments, so that > I'm now doing the following: > > The master table has a set of pointers to tables. One > group, one table - for groups in use. Unused groups > have no associated table and the pointer is null. > Pointers are always at (group number * pointer size) > from the start of the table. There's a max of 4096 > groups in this system, or the hardware guys start > shooting. > > Each group has a table that contains a set of pointers > to buffers. Offset is always (node number * pointer > size). Nodes that are active members have pointers to > pinned, allocated and cleaned buffers. Nodes that > aren't members get null pointers. There are 32 nodes > in the current system. > > The packet header contains the group address at a > fixed offset and the source node number at another > fixed offset. The hardware can then grab these and do > the necessary calculations. The hardware has a DMA > controller which can get the group pointer from the > master table then get the buffer pointer from the > group table. > > (Essentially, I'm mimicking a primitive virtual memory > system here.)
Do you have any benchmarks of that system, it looks quite complex to avoid some problems... > The hardware then stores enough packet payloads to > fill one page of memory (unless it gets an > end-of-transmit signal before then) and then DMAs the > page of data into the physical page in one shot. > (Since I did the original design, I worked out that > there's too much overhead in setting up a transmit to > just do a single packet at a time.) The hardware is > apparently smart enough to re-order packers if they > come in out of sequence and to NACK anything that's > missing. I.e. it supports TCP sequence number checks? In that case you store a set of flows in hardware, what happens when it overflows? > Once a page has been filled, the driver simply pushes > the pointer onto a queue and then grabs a pointer to a > fresh page. > > > Based on what I'm hearing, I'll probably finish this > driver simply to see what it does and post it up - > it'll have educational value, whatever happens. I'll > also work on a netchannels version which is less > hardware-specific and more "normal". That would be great. > > > I can try getting beer. Oregon has some acceptable > > > microbreweries, but I miss having a pint of > > Hatters in > > > England. Mead is easier. I brew mead. Strong, dry, > > > rocket-fuel mead. > > > > Then you definitely just need to send your driver to > > [EMAIL PROTECTED] > > Sounds good to me. Liquid driver review aids will be prepared. That's good - send your dirver and some kind of benchmarks for throughput and latency tests. -- Evgeniy Polyakov - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html