Thank you for your comments and I have some brief replies. On Thursday, May 5, 2016 at 4:20:06 PM UTC-4, Andrew Godwin wrote: > > > > On Thu, May 5, 2016 at 12:34 PM, Mark Lavin <markd...@gmail.com > <javascript:>> wrote: > > The main gains are (in my opinion): > - The same server process can serve both HTTP and WebSockets without path > prefixing (auto-negotiation based on the Upgrade header); without this you > need an extra web layer in front to route requests to the right backend > server > - HTTP long-polling is supported via the same mechanism (like WebSockets, > it does not fit inside the WSGI paradigm in a performant way) > - You get to run less processes overall >
As noted I don't see serving them both as an advantage. Also given that daphne is single-thread/eventloop based, it's likely that frontend proxy already would be needed to handle balance multiple processes or SSL termination. I don't think this reduces the number of processes. As stated I think it's the same. > > Firstly, nothing in channels uses pub/sub - channels deliver to a single > reader of a queue, and thus cannot be built on a broadcast solution like > pub/sub. > > If I'm understanding it correctly, groups are an emulated broadcast. I'm saying it would be an advantage for it to use pub/sub but it does not. > > I've always tried to be clear that it is not a Celery replacement but > instead a way to offload some non-critical task if required. > I don't agree that this has been clear. That is my primary criticism here. I don't think this should be encouraged. Ryan's reply continues with this confusion. > >> So Channels is at best on par with the existing available approaches and >> at worst adds a bunch of latency, potentially dropped messages, and new >> points of failure while taking up more resources and locks everyone into >> using Redis. It does provide a clear message framework but in my opinion >> it’s too naive to be useful. Given the complexity in the space I don’t >> trust anything built from the ground up without having a meaningful >> production deployment to prove it out. It has taken Kombu many years to >> mature and I don’t think it can be rewritten easily. >> > > a) ASGI does not lock everyone into using Redis; it just so happens that > is the first backend I have written. It is designed to run against other > suitable datastores or socket protocols and we have the money to fund such > an endeavour. > > b) Kombu solves a different problem - that of abstracting task queues - > and it would still be my first choice for that; I have used it for many > years and it would continue to be my choice for task queuing. > Yes the lock-in is an exaggeration, however, given the poor support/upkeep for third-party DB backends, I doubt the community will have better luck with Channel backends not officially supported by the Django core team. I'd be happy to be wrong here. Kombu is not to be confused with Celery. Kombu is a general purpose AMQP/messaging abstraction library. I don't think we agree on its potential role here. Perhaps it's better stated that I think Channel's minimalist API is too minimalist. I would prefer if additional AMQP-like abstractions existed such as topic routing and QoS. > > ASGI is essentially meant to be an implementation of the CSP/Go style of > message-passing interprocess communication, but cross-network rather than > merely cross-thread or cross-process as I believe that network transparency > makes for a much better deployment story and the ability to build a more > resilient infrastructure. > Again I don't agree with this argument and I don't see anything in Channels which backs up this claim. I believe this is where we likely have a fundamental disagreement. I see this network transparency as additional latency. I see the addition of the backend/broker as another moving part to break. > >> It’s hard for me to separate this work from the process by which it was >> created. Russ touched on my previous experience with the DEP process and I >> will admit that has jaded many of my interactions with the core team. >> Building consensus is hard and I’m posting this to help work towards the >> goal of community consensus. Thanks for taking the time to read this all >> the way through and I welcome any feedback. >> > > I will put my hand up and say that this sidestepped the DEP process, and > that's entirely my fault. It was not my intention; I've been working on > this for over two years, and only last year did I go public with my > semi-final design and start asking for feedback; I should probably have > taken it into a DEP then, but failed to. > > The problem is likely that I kept discussing channels with various members > of the core team and other people I know in the Django community, and > always received implicit approval, which is a terrible way to go about > being transparent. > > That said, I hope that my efforts over the last year to publicise and talk > about this in every available avenue have gone somewhat towards alleviating > the lack of a DEP; I have never tried to smuggle this in or be quiet about > it, in fact very much the contrary. I've had the ASGI spec (which I > potentially would like to push as a PEP) up for a while now, too, and have > been trying to actively get feedback on it from both the Django and the > wider Python community. > > I hope we can resolve our differences on this and both walk away happy; > you have some very valid points about deployment, reliability, and the > newness of all this code, but I also believe that the path from here to > having this deployed widely will be a good one. > > I have been working on this problem for a long time, and between > experiments both by myself and internally at Eventbrite where our engineers > tried a large number of different messaging backends for message transport > (in our case, for a SOA layer, though it performs a similar function and > requires similarly low latency), Redis seemed like the best choice for a > first and canonical transport implementation. (AMQP, Kafka, enterprise > message buses all have different problems). > > I don't expect people to adopt Channels overnight and switch to running > Daphne in front of all their traffic; if anything, I expect a lot of people > will run it just for WebSockets (I likely would at the moment if faced with > a very large deployment). That said, I believe it is certainly at the point > where it can be included in Django, if nothing else because the very design > of channels and ASGI means that the interface servers and transport layer > are both improveable and swappable out of the context of Django core. > > The patch to Django core is mostly routing and consumer design - an API > I've tried hard to refine to make accessible for beginners while having > flexibility for more advanced cases - and that's the only part that will be > directly locked in stone for the future. The other components - interface > servers and transport layers - exist outside the Django release cycle and > have the potential for large improvement or complete replacement as the > community starts using Channels and we start getting the feedback and > communal knowledge that only the large deployment of this kind of thing can > get. > > Sorry about circumventing the DEP process and pulling this off in a very > strange way; it feels particularly guilty now it's been highlighted to me > and I know that you are yourself working on a DEP, and it probably seems > like I've abused my position on the core team to pull this off; please > understand that was not my intention, and I've always wanted to have an > open, frank discussion about channels in Django. In many ways, I'm glad > someone has finally brought up all the things I thought would be valid > counter-arguments but haven't really been advanced yet. > > Andrew > What's done is done and I don't want to start another process discussion at this point. Maybe another day. I'm doing my best to focus on the technical aspects of the proposal. That isn't to say that I'm without bias and I'm trying to own that. The fact is I have looked into Channels, the docs and the code, and I remain unconvinced this should be the blessed solution for websockets and I've tried to make it clear why. I'd much prefer to continue to run Tornado/aiohttp for the websocket process. That's not a personal attack. I just don't see Channels as a meaningful improvement over that direction. - Mark -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/56c453e0-d6b8-4b1a-978d-5b07c907b3ea%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
