Hi everyone, It's been almost a year since the Channels funding project started, and a little while since it became an official Django project, and it's been hard to know what the state of it is at times.
I'm going to start doing monthly updates on how Channels is progressing and what's going on, and in this first update additionally reflect on the last twelve months of work and what's happened since we applied to Mozilla for funding. Funding Retrospective --------------------- There was some delays in getting the funds through from Mozilla thanks to a series of issues on their end and then our end, but work started almost straight away anyway and invoices came later once everything was sorted out. The initial few months were mostly getting the API and channel layers built out, and getting Channels to a stable state where it could serve WebSocket and HTTP requests. By about March this was there, and around this time I proposed merging it into 1.10 but I retracted that proposal after some valid feedback - more on that and the future of channels below. The months since have been a series of polishing and small improvements as we get feedback from both internal sources, such as the load testing we funded using some of the MOSS money, as well as people running Channels in development or production and filing bugs and feeding back when things went well. I've not been as good as I could have been at getting others involved under the proposal system; around ten people have approached me interested in working on a proposal but only five of those have got as far as sending one in, leaving more of the work on me as I would like. Things are fortunately looking up for getting a second paid maintainer on board, however, and I hope to share more about this soon. We've spent $60,000 of the $150,000 that we got from the fund; we're a bit behind where I'd like to be at this point due to some of the early delays but we have around another $20,000 in proposals/invoices coming up soon so it feels like about the right rate of spend. The plan is to continue to use the fund to keep Channels polishing going (especially making sure it works well at larger scale), and to start adding on more protocols and features (see below for more about that). Current State ------------- Let's start with where the project is at the moment. I'm currently prepping for a 1.0 release and the first officially stable API, with a few exceptions (data binding will be kept as an "alpha" API for now). The 1.0 release will have a couple of major changes (these will be more fully explained in the release notes): * WebSockets will have an accept/reject phase rather than always accepting. This removes the need for strict ordering, but some code will need a change as it's partially backwards-incompatible, and it's not currently possible to provide a shim to fix this. * Sending from consumers is now delayed until after the consumer exits so they are "atomic" (and discouraging long-running consumers). This is backwards-compatible as long as consumers were being used as described, and not being held open on some external trigger inside. The third breaking change I am considering is changing how message sending is provided inside Channels so that it's possible to modify via decorators and superclasses - current ideas include providing a send() callable as a second argument to consumers, or attaching it on the message object. This might also enable a backwards-compatible shim for the websocket acceptance change if it's implemented, which is another reason to consider it. Inclusion in Django ------------------- The effort to "include" Channels in Django started with a proposed merge into 1.10, which was rejected with some sensible arguments and ultimately failed. I did not think merging into 1.11 made sense, partially as it would be a while before it was stable enough that it would be accepted and particularly as 1.11 is an LTS. (Some of the objections raised in the inclusion discussion have resulted in changes in Channels' design or optimisations in the relevant code; I think it was right to not put it in at that point, in retrospect.) As a result, I worked on the "official projects" DEP to carve out a place to have projects that are Django-the-project not Django-the-core-codebase, and this seems to have gone down well (and opened the door to having more of these in future as we consider how to move Django forwards) Channels is now happily an official project, and there's no hurry to include it so far; neither I nor other core team members I've talked to know for sure if we'd want to include it in future or not, especially as it's been a long-standing goal of ours to make Django more modular (but Channels might end up being low level enough that it makes sense to merge). It will partially be down to performance concerns, too; if Channels were to be part of base Django it would need to not cause a performance impact in a default installation. In short, it's definitely not being included in 1.11, and we'll look at 2.0 and beyond as we get there and see how things are. Upcoming Features ----------------- There are a couple of in-flight features that are worth mentioning: * Multiplexing is being overhauled to be simpler and use a standard Consumer pattern (and not re-route messages via the channel layer again) * Work is hopefully going to commence shortly on a RabbitMQ-backed channel layer, which if things go well will become our second production-quality layer. There's also some more things that are in the works but too far out to confirm yet. Potential Changes ------------------ There are also some projects I'm planning to work on in the near future - if you're interested in helping out, these could be good things to help with (feel free to email me to ask any questions about helping): * Investigating a channel layer that uses an on-disk SQLite database and locking rather than the IPC posix primitives. The asgi_ipc layer has some performance concerns and compatibility issues, and I'm starting to wonder if POSIX IPC was the correct choice there. * Implementing support for socket.io, sock.js or a similar websocket-and-polling combination protocol so that people can use these more compatible methods of pushing data rather than raw WebSockets. * Implementing better support in Channels directly for doing long-polling or other consumer patterns with a delay or event wait in the middle so that it's easy to carry local context over, potentially laying the groundwork for asynchronous consumers and how one might use HTTP 2 rather than WebSockets. GitHub Issues Issues -------------------- Channels uses GitHub issues on the various repositories to track issues rather than the Django Trac, mostly to try and keep that Trac for core issues only. This has generally worked well, and labels serve to categorise the tickets by type and difficulty level well, but GitHub does not allow anyone not contributing to the repository to be added as an assignee, which means I would have to manually add everyone to the repo as a contributor before they could assign themselves (or I or another admin could assign them) I am investigating some sort of bot that aids in this process, having asked around and not found another good option, but views on this are welcome. Mailing Lists ------------- It's not been terribly clear until now where general discussion of Channels things should happen, and it's ended up being a combination of talking in issues, IRC, direct email to myself and the Django mailing lists. We discussed this at the core meeting this week and going forward I'll be pushing all development discussion (bar helping people with proposals for funding) to the django-developers list, and all questions about usage to the django-users list. Ongoing Updates --------------- I'll be trying to do updates like this (or having another maintainer do it) on roughly a monthly basis going forward, though there'll be less content than this first initial knowledge dump. I'd like to make sure people can keep abreast of Channels work and plans without scouring through the mailing lists and issues discussions. Any feedback is welcome; feel free to reply to this thread and I'll happily answer any questions. Andrew -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1up7w1FPy3bm2AkU1779d4RmnnudLAxEp0YNeLwnu5b8_g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
