Re: Channels integration plan, first draft

2015-12-18 Thread Andrew Godwin
On Fri, Dec 18, 2015 at 5:34 PM, Marc Tamlyn wrote: > [Note: I have not read all the channels docs, sorry if some of these > points are covered there.] > > On a packaging note, is there a way to use django[channels] type syntax > like flask does? I'm not familiar with the restrictions of this but

Re: Channels integration plan, first draft

2015-12-18 Thread Florian Apolloner
On Friday, December 18, 2015 at 6:34:41 PM UTC+1, Marc Tamlyn wrote: > > I've said this in person to you, but I think a REDIS_SERVERS setting like > DATABASES would be a hugely useful feature for django independently of > channels, especially if it supported tests well. I'm yet to find a third >

Re: Channels integration plan, first draft

2015-12-18 Thread Donald Stufft
That syntax allows you to add extra, opt in lists of dependencies to install. It does not pass through to runtime. Sent from my iPhone > On Dec 18, 2015, at 12:34 PM, Marc Tamlyn wrote: > > On a packaging note, is there a way to use django[channels] type syntax like > flask does? I'm not fam

Re: Channels integration plan, first draft

2015-12-18 Thread Andrew Godwin
On Fri, Dec 18, 2015 at 5:28 PM, Anssi Kääriäinen wrote: > On Friday, December 18, 2015, Andrew Godwin wrote: > >> >> >> On Fri, Dec 18, 2015 at 3:44 PM, Mark Lavin wrote: >> >>> > You seem to be assuming I'm here to foist a brand new middle layer on >>> everyone; I'm not. I'm here to make one

Re: Channels integration plan, first draft

2015-12-18 Thread Marc Tamlyn
[Note: I have not read all the channels docs, sorry if some of these points are covered there.] On a packaging note, is there a way to use django[channels] type syntax like flask does? I'm not familiar with the restrictions of this but it may remove the need for try/except imports. I'm also curio

Re: Channels integration plan, first draft

2015-12-18 Thread Anssi Kääriäinen
On Friday, December 18, 2015, Andrew Godwin wrote: > > > On Fri, Dec 18, 2015 at 3:44 PM, Mark Lavin > wrote: > >> > You seem to be assuming I'm here to foist a brand new middle layer on >> everyone; I'm not. I'm here to make one that fits neatly into Django, that >> I think most people will wan

Re: Channels integration plan, first draft

2015-12-18 Thread Andrew Godwin
On Fri, Dec 18, 2015 at 3:44 PM, Mark Lavin wrote: > > You seem to be assuming I'm here to foist a brand new middle layer on > everyone; I'm not. I'm here to make one that fits neatly into Django, that > I think most people will want to turn on, and that provides a lot of value > in exchange for

Re: Channels integration plan, first draft

2015-12-18 Thread Mark Lavin
> You seem to be assuming I'm here to foist a brand new middle layer on everyone; I'm not. I'm here to make one that fits neatly into Django, that I think most people will want to turn on, and that provides a lot of value in exchange for a slight round-trip performance hit - my goal is sub-5ms, and

Re: Channels integration plan, first draft

2015-12-18 Thread Andrew Godwin
On Fri, Dec 18, 2015 at 2:00 PM, Mark Lavin wrote: > Anssi criticisms are fair and I feel that some of these responses are > glossing over the details. > I'm sorry if it comes across like that - a lot of these are things I've been considering for a while and so I can forget to provide context.

Re: Channels integration plan, first draft

2015-12-18 Thread Mark Lavin
Anssi criticisms are fair and I feel that some of these responses are glossing over the details. You've claimed this is the same or equivalent to a forked worker model but it isn't because there is no process management/link between the interface and worker and because you've chosen to make this ne

Re: Channels integration plan, first draft

2015-12-18 Thread Andrew Godwin
On Fri, Dec 18, 2015 at 7:34 AM, Anssi Kääriäinen wrote: > I have a gut feeling this isn't going to work that well. The reasons > include: > - Backwards compatibility: how is a large site going to upgrade from 1.9 > to 1.10? > None of the core view API will change. A 1.9 codebase will boot and

Re: Channels integration plan, first draft

2015-12-17 Thread Anssi Kääriäinen
I have a gut feeling this isn't going to work that well. The reasons include: - Backwards compatibility: how is a large site going to upgrade from 1.9 to 1.10? - Complexity of setup. - Error conditions: for example, what happens when an interface server sends a request to worker, and then die

Re: Channels integration plan, first draft

2015-12-17 Thread Andrew Godwin
The problem is that even with a specially named channel you can't guarantee the messages will all go to the same worker so you can re-assemble them on local disk, which kind of makes it still a hard problem. It works for responses because there's only one interface server listening to any response

Re: Channels integration plan, first draft

2015-12-17 Thread Sam Willis
Hi, To support file uploads or a large message body the http.request message could have an file_channel or body_channel (much like its reply_channel)? These would be something like http.request.file.Dj3Hd9J and would stream chunked file or body content in the same way as the http.response messa

Re: Channels integration plan, first draft

2015-12-17 Thread Andrew Godwin
Yes, that is the idea. While it obviously adds overhead (a millisecond or two in my first tests), it also adds natural load balancing between workers and then lets us have the same architecture for websockets and normal HTTP. (The interface server does do all the HTTP parsing, so what gets sent ov

Re: Channels integration plan, first draft

2015-12-17 Thread Anssi Kääriäinen
Is the idea a large site using classic request-response architecture would get the requests at interface servers, these would then push the HTTP requests through channels to worker processes, which process the message and push the response through the channel backend back to the interface server an

Re: Channels integration plan, first draft

2015-12-17 Thread Raphael Michel
Hi, Am Thu, 17 Dec 2015 12:27:07 -0700 > I'll admit to not being an expert on this use case at all, since I > don't generally do it, but AFAIK SCRIPT_NAME remains pretty key for > transparently deploying a Django site at non-root URL paths. I used this before and can confirm that it is currently

Re: Channels integration plan, first draft

2015-12-17 Thread Andrew Godwin
Yeah, I definitely see the need for it, and things like making the webserver transparently change the paths isn't going to work since it can't rewrite output. At least we can rename it and call it request['root_path'] or something. On Thu, Dec 17, 2015 at 7:27 PM, Carl Meyer wrote: > On 12/17/20

Re: Channels integration plan, first draft

2015-12-17 Thread Carl Meyer
On 12/17/2015 11:50 AM, Andrew Godwin wrote: > - I thought SCRIPT_NAME was basically unused by anyone these days, but > hey, happy to be proved wrong. Do we have any usage numbers on it to > know if we'd need it for a new standalone server to implement? It's > really not hard to add it into the req

Re: Channels integration plan, first draft

2015-12-17 Thread Andrew Godwin
To address the points so far: - I'm not yet sure whether "traditional" WSGI mode would actually run through the in memory backend or just be plugged in directly to the existing code path; it really depends on how much code would need to be moved around in either case. I'm pretty keen on keeping a

Re: Channels integration plan, first draft

2015-12-17 Thread Anssi Kääriäinen
On Thursday, December 17, 2015, Carl Meyer wrote: > Hi Andrew, > - I share Mark's concern about the performance (latency, specifically) > implications for projects that want to keep deploying old-style, given > all the new serialization that would now be in the request path. I think > some furt

Re: Channels integration plan, first draft

2015-12-17 Thread Florian Apolloner
On Thursday, December 17, 2015 at 5:50:39 PM UTC+1, Carl Meyer wrote: > > - I share Mark's concern about the performance (latency, specifically) > implications for projects that want to keep deploying old-style, given > all the new serialization that would now be in the request path. > It would

Re: Channels integration plan, first draft

2015-12-17 Thread Florian Apolloner
It would be interesting to add a few sentences about file uploads and how they are going to work with the new system. On Thursday, December 17, 2015 at 12:35:18 PM UTC+1, Andrew Godwin wrote: > > Hi everyone, > > One of the first steps I want to get done for Channels is get a rough plan > in pla

Re: Channels integration plan, first draft

2015-12-17 Thread Carl Meyer
Hi Andrew, On 12/17/2015 04:34 AM, Andrew Godwin wrote: > Hi everyone, > > One of the first steps I want to get done for Channels is get a rough > plan in place for how things are going to work in terms of where code > goes, supported versions, etc. I've written up my thoughts on this into > a fi

Re: Channels integration plan, first draft

2015-12-17 Thread Mark Lavin
I have concerns about "built-in feature" means for existing applications (and future applications really). Under "Preserving Simplicty" you note that you should be able to run Django as simply as you do now. Is there a consideration to run "classic" WSGI applications without channels? This in-m

Channels integration plan, first draft

2015-12-17 Thread Andrew Godwin
Hi everyone, One of the first steps I want to get done for Channels is get a rough plan in place for how things are going to work in terms of where code goes, supported versions, etc. I've written up my thoughts on this into a first draft of what I'm calling the "integration plan": http://channel