On Tue, Feb 7, 2012 at 03:47, Mark Thomas <ma...@apache.org> wrote: > On 07/02/2012 02:23, Jonathan Drake wrote: > > I'm one the three CS grad students working on WebSocket (along with Petr > > Praus). > > > > Just wanted to give an update on our progress, to let you know what we're > > working on: > > > > Adding support for fragmented payloads: > > Excellent. That and handling control frames are the biggest gaps right > now in my view. > > > Right now, after receiving a frame, StreamInbound unmasks the payload in > a > > WsInputStream and passes it up to the servlet via > > onBinaryData()/onTextData(). > > To support fragmented frames, we add an intermediate step: after > unmasking > > the payload, write it to a PipedOutputStream connected to a > > PipedInputStream that we pass upward via onBinaryData()/onTextData(). > When > > the next fragment arrives, keep streaming data through the pipe. > > Piped[Input|Output]Stream are intended to be used with a separate thread > at each end. There is currently only a single thread processing the > incoming data. How do you propose to provide the additional thread? >
Ah, then it's different. I thought the client handler is running in different thread. Then we'll probably need something different. > > This has the advantage of also allowing us to stream huge payloads (RFC > > 6455 allows for a 64-bit extended length field---way too much data to > > buffer in memory all at once). > > That is certainly a requirement. However, there is more than one way to > meet that requirement. > Our initial idea was some form of a streaming API where the reader code receiving the message fills a stream that's somehow connected to a client code that would be consuming it. Hence the Piped[Input|Output]Stream stuff. I also think the individual frames of a fragmented messages should not be exposed to the client code. If I remember correctly, RFC specifies that its up to the implementation to decide. What do you think? And, what are your other ideas for supporting huge payloads? > > > It has the minor disadvantage of breaking the ByteBuffer wrappers from > > MessageInbound (we can still use them for small payloads if we buffer > > fragments in memory) > > Could you clarify what is broken in what circumstances please. > MessageInbound is expected to work providing that the message > (regardless of how many fragments it is spread across) is smaller than > the available buffer size. > > My expectation is that the current echo example would continue to work > regardless of whether or not the messages were in a single fragment or > multiple fragments. > > I'm working on a patch that implements this...maybe a day or two. > > > > I'd appreciate any early criticism you may have---otherwise I mainly just > > want to prevent duplicate work by explaining what we're up to. > > Thanks for the heads up. > > The approach you describe isn't the one I had in mind, but that is a > good thing. It provides an opportunity to compare and to take the best > from both. > > It would be nice to have a test case or an example client for this. > Unless there is an easy way to force a browser to fragment packets, I > suspect a test case will be required. > I'm not sure about other browsers but Chrome 16 fragments payloads so that WS frames fit into MSS of the underlying protocol (TCP). So if I sent string larger than roughly 1450 bytes from the client javascript, I got fragments packets. Beware that "lo" device on Linux has by default MTU 16436 so on localhost, the above value of 1450 bytes is much larger. > Given that the implementation currently uses blocking IO Just to make sure, processor.read() calls are always blocking, right? I'm somewhat confused by Nio/Bio/Apr in combination with this. As I understand it, any of the three can be used at the user's will and so the processor can be of type UpgradeNioProcessor. In that case the read on the underlying socket is not blocking. So how come read on the UpgradeProcessor will always be blocking? Can you shed some light on this? Thanks > , my approach was going to be something along the lines of: > a) read the headers > b) call onBinaryData() / onTextData() > c) make the payload available via WsInputStream until the end of the > fragment > d) read the headers for the next fragment > e) make the payload available via WsInputStream until the end of the > fragment > f) repeat d) & e) until the final fragment is processed > g) return EOF on the next read > This would mean moving most of the code currently in > StreamInBound.onData() to WsInputStream > > I haven't tried coding this up or anything so there is no guarantee it > would actually work. > This seems like a sensible approach, I'll look into it. Petr
