Django Channel asgi specs - question on how to handle a websocket connection

[Oskar Hahn]

Hi,

this is a question on how to interpret the asgi specs of django channels:

https://channels.readthedocs.io/en/latest/asgi.html

If this is the wrong place to discuss it, then please point me to the
right place.

I used some free time to learn the language "go" and tried to write a
protocol servers in go. This works quite well, thanks for the asgi specs
and daphne as an example. Most parts already work. But I do not
understand how to handle incoming websocket connections.

https://channels.readthedocs.io/en/latest/asgi.html#connection

If I understand the specs right, then the protocol server should not
respond to an incoming websocket connection right away but keep the
http-connection open with no response at all. But it should send a
connection message to the channel layer,  wait for the responding
Send/Close/Accept message and handle the websocket-request accordingly.

But there are some cases which I do not understand:

What should happen if the python application does not send a
Send/Close/Accept at all? Should the websocket-handshake stay in an
incomplete state?

It seems, that the reconnecting-websocket javascript library[1], that is
used by the django channels examples[2], closes and reconnects an
websocket handshake, if there is no response after one second.

[1] https://github.com/joewalnes/reconnecting-websocket
[2] https://github.com/andrewgodwin/channels-examples



The specs says, that the Send/Close/Accept message has a "close" and a
"accept" flag. What should the protocol server do while the connection
is waiting for acceptance and

* accept AND close are both set to true?
* neither accept nor close  (nor byte nor text) is set to true?

When I compare the "stable" specs with the "latest" specs I see, that
the "accept" flag has not been there in the "stable" version. I do not
understand way it was introduced. If I understand the Send/Close/Accept
message correctly it should be possible to remove the "accept" flag by
merge it with the "close" flag into one "close/reject" flag.

* If this flag is set while the connection is waiting for acceptance,
then the websocket connection is rejected.
* If the flag is set to true while the connection is established, then
the connection should be closed.
* If it is set to false while the connection is waiting for acceptance,
then the connection is accepted
* If it is set to false while the connection is established, then it is
kept open.



The specs say, that close can be a positive integer specifying the
response code. What should the protocol server do while the connection
is waiting for acceptance when it receives an Send/Close/Accept message
like this one:

{"close": 1006}

At this moment the websocket handshake was not finished, so the usual
way to "close" it would be not to open it at all. The protocol server
can just send an http 403 response. So should the "close" value be
ignored? Is there a way for the python application to specify the http
status code? I think this is not possible because the specs allow the
protocol server to open the websocket connection before the
Send/Close/Accept message was received and buffer all incoming messages.
In this case it is not possible for the protocol server to send an http
status code to the client. Maybe the specs should be more strict and say
that it is not allowed for the protocol server to finish the websocket
handshake before the first Send/Close/Accept message was received. In
this case the close-value of the first Send/Close/Accept message could
be interpreted as an http response code.

Best regards,
Oskar

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/26bf7c7e-bc74-618f-818a-bd7b6538caf2%40oshahn.de.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Django Channel asgi specs - question on how to handle a websocket connection

[Oskar Hahn]

Thanks. I updated my code accordingly. But I open the websocket
connection after one second if the python application does not send an
Send/Close/Accept message. In other case the software would not be
compatible with the examples in

https://github.com/andrewgodwin/channels-examples

I uploaded the code to:

https://github.com/ostcar/goasgiserver

I probably have no time to work on it further. But maybe some of the
code can be reused.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/83a29d7b-9451-8284-affc-ac5e5ed8e6a0%40oshahn.de.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: How Channels and Daphne handle huge HTTP requests?

[Oskar Hahn]

Hi Alexander,

if you need an asgi-server, that currently supports body_channel, then
you can use https://github.com/ostcar/geiss

Oskar


Am 19.03.2017 um 19:15 schrieb Andrew Godwin:
> Hi Alexander,
> 
> I believe Daphne does not yet implement body_channel support for large
> HTTP requests, just the response version right now. It's on the roadmap,
> I just need to get round to adding it in. Until then, I would advise not
> using it for large uploads.
> 
> Andrew
> 
> On Sun, Mar 19, 2017 at 1:17 AM, Alexander Prokhorov  > wrote:
> 
> Dear Colleagues,
> 
> I tried to understand how Channels and Daphne work with
> single-reader channels in case when there is huge HTTP request. To
> my surprise I have not found any code in Daphne which works with
> `body_channel`. The only occurrence of `body_channel` I found is in
> the test `assert_valid_http_request_message`, but the relevant
> branch of execution is never triggered during testing.
> 
> Could someone clarify where to search?
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to django-developers+unsubscr...@googlegroups.com
> .
> To post to this group, send email to
> django-developers@googlegroups.com
> .
> Visit this group at
> https://groups.google.com/group/django-developers
> .
> To view this discussion on the web visit
> 
> https://groups.google.com/d/msgid/django-developers/3254159d-2d04-4b2e-98e0-0386eb59c282%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout
> .
> 
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-developers@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFwN1uo%2B2hG37UQu27%3Dy7d0peZ%2BRKuiufNBTDhJ1pJJ1Sm3-eA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4c90cc90-f036-6ad6-2082-ed413cb35d43%40oshahn.de.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Changing how Channels/ASGI receives messages

[Oskar Hahn]

Hi,

I am not sure, if I understand your proposal correctly. Do you mean, the
asgi server has to listen only to one channel or do you mean it has to
listen to one channel of any channel-type?

So with redis, do I have to call

"BLPOP LOCALID TIMEOUT"

or do I have to call

"BLPOP http.response.PROCESSID! TIMEOUT" and
"BLPOP websocket.send.PROCESSID! TIMEOUT"

I think you mean the second one, but then your goal, that daphne has
only to listen "on a single channel at any time" seams not be fulfilled.

I would like a solution, were the asgi server really listens only to one
channel. I don't like the idea, that there is a fixed set of channel
names. This seems not flexible.


Another thing:

Currently the usual case looks like this:

Time | ASGI Application | ASGI Server|
--
1|  | receive("foo!123") |
2| send("foo!123")  ||

First, the asgi server listens to a specific channel, for example
http.response.RANDOMPART!CLIENTID. Then the Django app sends something
to this channel and the asgi server gets it.

But what should happen when the django app sends to a channel and the
asgi server starts to listen on this channel at a later time:

Time | ASGI Application | ASGI Server|
--
1| send("foo!123")  ||
2|  | receive("foo!123") |

With the current implementation the messages remains in the asgi-backend
until it expires. So the ASGI Server can fetch it later.

With your proposal this could change. The ASGI-Server will get the
message from the asgi-backend, as soon, as it was send by the
application. So if nobody has called receive() for this channel, it
either has to be dropped or saved.

Do you have a recommendation what to do with a message, that can not be
dispatched to a client socket? This should not happen so often.

Oskar


Am 28.03.2017 um 03:50 schrieb Andrew Godwin:
> Hi all,
> 
> I wanted to ask your feedback on a proposed change I want to make to the
> ASGI spec for channel layers.
> 
> In particular, my goal is to make it so that Daphne only has to
> receive() on a single channel at any time, massively simplifying the
> work it takes to poll for new messages and the overhead that switching
> channel sets all the time caused.
> 
> The way I want to do this is to redefine process-local channels - those
> with a ! in them - so that they are written like this:
> 
> http.response.PROCESSID!LOCALID
> 
> Then, change the semantics of receive() so that you only call
> 
> channel_layer.receive(["http.response.PROCESSID!"])
> 
> And this then receives any message sent to any channel with that prefix.
> The channel name returned from receive() is still the full name as sent
> by the client; it's basically acting as a very limited wildcard.
> 
> This will allow Daphne (and any other server) to listen on just a single
> process-local channel, with a random part made by the process at boot
> (e.g. "http.response.A34GF3!"), and then internally dispatch messages as
> they come in to the appropriate client socket. It also means that the
> client API - the one seen by end-developers and the Channels library
> itself - does not change at all, as the reply_channel can still be set
> to something like "http.response.A34GF3!K89C3", and everything will flow
> correctly.
> 
> It's better than full wildcard support as that may be very inefficient
> to implement depending on the server; the limitation means that
> implementations can just deliver into queues based on the non-local part
> and stuff the "full" channel name into the message storage area, and
> unpack it again on receive.
> 
> The potential benefits to efficiency are hard to understate; the current
> definition of process-local channels was meant to allow channel layers
> to do something like this internally but it still needed server code to
> swap out the list of things it was listening on every receive() call. By
> removing this abstraction, servers can be written with a single tight
> loop around a fixed set of channel names, and we can block way more
> efficiently on servers like Redis.
> 
> I want to know if anyone has any feedback on this, especially from the
> "potential problems" department; if not, I'm going to try and get this
> out before DjangoCon Europe.
> 
> Thanks!
> 
> Andrew
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com
> .
> To post to this group, send email to django-developers@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://g