Re: injecting settings

2019-05-09 Thread J. Pic
In the example Config.ready() calls for a dict update() which will
probably work for a while, before changing the update() call in the example
with more elaborated code.
System checks shoud catch cases where configuration is invalid at all.

Not sure how much you can pull from entry points, if you want to keep
INSTALLED_APPS, but have a hook for your app to autoconfigure itself when
added maybe something like this:

[django.middleware]
DebugToolbarMiddleware = debug_toolbar.middleware.DebugToolbarMiddleware

Otherwise a freeform callbacks should be more flexible to provision common
stuff apps contribute to such as urls, middlewares, template engines ...

It's true that it doesn't look like you're going to be able to meta program
settings really profoundly, ie. different packages wanting different
defaults, but at least seems it can automate most of the work when
installing an app from the ecosystem into a new django project, that could
at least hook sane defaults, for example django-debug-toolbar's AppConfig
could provision middleware and urlconf and then you would only need to add
debug_toolbar to INSTALLED_APPS.

Does INSTALLED_APPS allow to plugin in a given AppConfig with arguments ?
Ie. in cakephp 3.7 that was just released they don't have a list such as
INSTALLED_APPS, but instead you add apps as such:

# that's all they need to setup the debug toolbar in cakephp
Plugin::load('DebugKit')

# but you could load a plugin without having urls injected as such:
Plugin::load('Cors', ['routes' => false]);

Which would be equivalent to something like:

INSTALLED_APPS.load('debug_toolbar')
INSTALLED_APPS.load('otherapp', urls=False)

It looks like app maintainers really ought to give django-gdaps a shot.
Perhaps Christian, another way would be to contribute gdap support to the
apps you like to use in the ecosystem. Then their maintainers could benefit
from it if they install gdaps and so it can propagate in the ecosystem, and
you can consolidate your patterns.

Best

-- 
∞

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op1_dMFwwxPwECqc8hk%3DT5HFkcMJjaE3VJ%3Dgo8W_%2B72a5Zg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: injecting settings

2019-05-09 Thread J. Pic
ERRATA in the code above, a mistake I make really often, instead of:

DebugToolbarMiddleware = debug_toolbar.middleware.DebugToolbarMiddleware

Should be:

DebugToolbarMiddleware = debug_toolbar.middleware:DebugToolbarMiddleware

In one python module I rely on this (cli2), I ended just making so that
both forms work ^^

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op196pwJs0%2B_uuq4OtLg1E%2B%3D-dPjOUhp7Exk1p-1T5BS7Ww%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: injecting settings

2019-05-09 Thread Christian González

>
> It looks like app maintainers really ought to give django-gdaps a
> shot. Perhaps Christian, another way would be to contribute gdap
> support to the apps you like to use in the ecosystem. Then their
> maintainers could benefit from it if they install gdaps and so it can
> propagate in the ecosystem, and you can consolidate your patterns.
>
That's what I wanted when I created GDAPS. It should be an extension,
and not replace Djangos app loading mechanism. Not all applications are
designed to be pluggable, so a deterministic order in INSTALLED_APPS is
fine. How I designed it, was meant to *enable* Django apps to be
pluggable just by adding gdaps to INSTALLED_APPS.

But yes, this "sane defaults" what I need there would be handy in lots
of other Django apps, like you mentioned django-debug-toolbar.

And it's not about settings, but also e.g. urlpatterns - apps should be
able to inject them too. I found a way that works in GDAPS, using my
(adapted from pyutilib) plugin system.

The main settings/urls.py file could/should always be the master - this
is different in every application. Think of ordering urls.

Christian

-- 
Dr. Christian González
https://nerdocs.at

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8b8bf367-3026-fecd-4bf1-3c13327f12a2%40nerdocs.at.
For more options, visit https://groups.google.com/d/optout.


pEpkey.asc
Description: application/pgp-keys


Re: injecting settings

2019-05-09 Thread J. Pic
Maybe you would want to install an app without having their urls injected
then you're going to need to do things like

INSTALLED_APPS = [
 someapp.AppConfig(urls=False)
]

Of course this is going to make complicate the settings system, but why not
hook a callback in AppConfig that is executed in-between the moment
settings are parsed and the moment they are loaded into Django ?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op18yUasg74LVeW9FVQCA%3Dx%2BEnP3KhuDtRh_t8L5UtGkDkg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Django Async DEP

2019-05-09 Thread Andrew Godwin
Hello everyone,

While the ASGI patch (https://github.com/django/django/pull/11209) is maybe
the first small step in a long road to async, it's the only real one we
could do without a DEP as it purely pulls on existing specs and bugs.

To that end, I have drafted a DEP (provisionally #0009), "Async-capable
Django". It is a summary - and I use that word very lightly - of the rest
of the work to be done to make a version of Django that one could call
"async capable".

You can view it here:
https://github.com/andrewgodwin/deps/blob/async/draft/0009-async.rst

It is approximately 7,000 words long - while it could probably do with some
editing, the subject matter means it has to be quite in-depth. Even then,
it does not propose an exact plan to follow; instead, it proposes the
overall strategy and the high-level ideas about what needs to be done.

If you are short on time, please read the Foreword and the High-Level
Summary, and then jump around using the table of contents to the sections
that interest you.

Feedback on this is very much appreciated; you can either reply here or, if
you have comments that would benefit from precise location on the diff, use
the pull request here: https://github.com/django/deps/pull/56

This is a very complex topic, and I appreciate feedback might take a while;
at least a couple of weeks to get everyone's thoughts. If you want a more
private channel to discuss concerns or design questions you're not happy to
talk about publicly, feel free to email me directly.

This proposal does not quite fit the DEP template - we never really planned
for something of this scale - but I have done my best to make it work; I
think we still benefit from discussing and voting on it in the DEP format,
but suggestions on how to make it work better are welcome, provided they
fit within how Django operates as a community now.

Finally, if there is positive feedback on this, we have the separate
question of funding it that I will discuss separately, if and when we get
that far. The DEP covers some of the topics, but it's notable that the
implementation speed of this varies considerably; if we could get, say, me
and someone else on it full-time (which is not a cheap or likely prospect),
we can probably be done inside of a single release. If we rely just on
volunteer time, it may take years. There is hopefully somewhere between
those two that works.

Thanks for taking the time to read through!

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uqav3JXQA3m%2Bajf0Bd15QQX3JTZTfmH3Hc3ECWyRR3CVg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Patryk Zawadzki
Hey Andrew,

Great work on the DEP, the task at hand is humongous.

Do you think it's worth it to try and make the ORM async? It contains tons 
of magic that is inherently incompatible with explicit I/O that is required 
for async/await to work, things like silently fetching relations on first 
access, silently fetching deferred fields, queryset laziness etc. Those 
need to be explicitly awaited now and having to await regular field access 
for the sake of making deferred fields work is (I believe) not worth it.

Conveniently, all of these magic properties are things that silently break 
performance of applications leading to the n+1 queries problem so I'm more 
than happy to get rid of them ;)

Seeing that we can't get rid of anything in the old synchronous ORM, one 
idea would be to just depend on a separate, asyncio-based ORM and configure 
it to use the same tables and relations that the old ORM uses. It could 
work like model forms or formset factories, give it a "classic" model 
class, a list of overrides (we can't hope to automatically match old field 
types to anything understood by another engine) and receive an 
asyncio-capable table. There's an obvious candidate that Tom Christie is 
working on.

We could even improve the querying syntax to use the SQL Alchemy's 
filtering operators with the hope of eventually reaching full static type 
safety[2] (one can only dream).

The new ORM would only be usable in new async views and other features 
could be implemented gradually. You don't need to async forms (but with Tom 
Christie's typesystem[3] you don't really need async forms) and async 
template loading to achieve great things.

You can already use things like requests-async[4] to access external 
resources, you can use libraries like Ariadne[5] to implement a GraphQL API 
with WebSocket-based subscriptions without having any of those (disclaimer: 
I work on Ariadne). It's okay if some things can't be ported initially or 
if certain features are only available in their sync flavor. The community 
will pave the way and provide many missing pieces before we get to 
implementing them. It's okay if new async counterparts have different APIs 
if these APIs are either better or are a closer match to how async/await 
works.

[1] https://github.com/encode/orm
[2] https://docs.sqlalchemy.org/en/13/core/tutorial.html#operators
[3] https://github.com/encode/typesystem
[4] https://github.com/encode/requests-async
[5] https://github.com/mirumee/ariadne


PS: If Django provided abstractions for common event loop operations such 
as "await this collections of futures" and "ensure this task is executed 
even if not explicitly awaited" then I think it may be possible to avoid 
explicitly depending on asyncio.

All the best

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/da0d4c65-562b-44cc-abe0-e5c31e28d91d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread J. Pic
Hi Patryk,

I'm not sure but for me the "What is Django" section answers the question.
For me Django is full of philosophy that seeds a great ecosystem of apps of
all sorts with a growing user base nonetheless, and a bunch of brilliant
hackers to look up to and inspire for more. Of course if you're into fixing
a particular technical issue such as "I want to make an efficient GraphQL
server", then even Django works even if it's not necessarily the most fit
for X reason. But for "I want to get a web page going with some forms and a
database and see where that goes" then Django is definitely enjoyable, and
at this point in time where tons of more sophisticated frameworks are born
it's clear that Django is still in the game and will be for the next 10
years. So, why not also get more features out of it while we've not been
abandoned by all the talented contributors ? It's not because we like
Django that we cant haz nice things ;)

Great move Andrew and very exciting, keep it up and best luck to you !

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op19DRYB%2BtUJX-9fG_9EOeo0oqGZayn5SHHBdec9qweq8ng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread J. Pic
Oops too fast, if it's possible to split the DEP and delay the ORM as
advised by Patrick it could make it a lot easier to distribute the work, I
don't know really sry. Have a great day !

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op1-tDvM-raHfOLGCQYFFk7Aoz2CAa%3DY_bXcaj30Y887wzw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Patryk Zawadzki

>
> I'm not sure but for me the "What is Django" section answers the question. 
> For me Django is full of philosophy that seeds a great ecosystem of apps of 
> all sorts with a growing user base nonetheless, and a bunch of brilliant 
> hackers to look up to and inspire for more. Of course if you're into fixing 
> a particular technical issue such as "I want to make an efficient GraphQL 
> server", then even Django works even if it's not necessarily the most fit 
> for X reason. But for "I want to get a web page going with some forms and a 
> database and see where that goes" then Django is definitely enjoyable, and 
> at this point in time where tons of more sophisticated frameworks are born 
> it's clear that Django is still in the game and will be for the next 10 
> years. So, why not also get more features out of it while we've not been 
> abandoned by all the talented contributors ? It's not because we like 
> Django that we cant haz nice things ;)
>

I'm arguing the opposite, that by limiting the scope of the MVP to 
asynchronous views (even if middleware stays synchronous for now) and using 
an existing ORM we can make Django a viable solution for many new 
applications. And some of them don't need the full stack.

I'd go as far as arguing that fewer and fewer applications benefit from 
Django providing all of the pieces (as many new applications only implement 
an API and render in the client) so treating them all as blockers here may 
not be beneficial.

Best regards

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/314d7b71-1bf1-441b-ae49-e4fbeb5e1891%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Andrew Godwin
I would agree with both of you - I think the most important thing is to get
the view layer async-capable, as that then lets sites use any manner of
asynchronous libraries that already exist to get experiments and unique
things going. Tom Christie, for example, has already started work on an
asynchronous ORM. Some of Django's biggest sites don't use the ORM - for
example, Instagram.

That said, I also think it's important to allow the ORM to support both
modes in the long term. I truly believe the best way to be able to write
async code is to _have the choice to write it_, rather than being made to
all the time; if we make people use a separate, async ORM, then we force
them to write every view asynchronously, with all the extra danger and
thinking that requires. It's much better for Django to do the hard work,
and say "hey, if you want to write asynchronously or synchronously, that's
fine - it takes literally zero extra effort to go either way".

This is why I propose in the DEP that we do the view layer first, and then
move onto the ORM as a second wave.

Andrew

On Thu, May 9, 2019 at 12:29 PM Patryk Zawadzki  wrote:

> I'm not sure but for me the "What is Django" section answers the question.
>> For me Django is full of philosophy that seeds a great ecosystem of apps of
>> all sorts with a growing user base nonetheless, and a bunch of brilliant
>> hackers to look up to and inspire for more. Of course if you're into fixing
>> a particular technical issue such as "I want to make an efficient GraphQL
>> server", then even Django works even if it's not necessarily the most fit
>> for X reason. But for "I want to get a web page going with some forms and a
>> database and see where that goes" then Django is definitely enjoyable, and
>> at this point in time where tons of more sophisticated frameworks are born
>> it's clear that Django is still in the game and will be for the next 10
>> years. So, why not also get more features out of it while we've not been
>> abandoned by all the talented contributors ? It's not because we like
>> Django that we cant haz nice things ;)
>>
>
> I'm arguing the opposite, that by limiting the scope of the MVP to
> asynchronous views (even if middleware stays synchronous for now) and using
> an existing ORM we can make Django a viable solution for many new
> applications. And some of them don't need the full stack.
>
> I'd go as far as arguing that fewer and fewer applications benefit from
> Django providing all of the pieces (as many new applications only implement
> an API and render in the client) so treating them all as blockers here may
> not be beneficial.
>
> Best regards
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/314d7b71-1bf1-441b-ae49-e4fbeb5e1891%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1urQVr%3DZ55Xqn7E%2B5-AeZU-egJAZ_%2BooAp_3nSSYnbNX7w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread J. Pic
In one project I really enjoyed using channels just as a background worker
instead of celery, and the DEP does not talk much in the Email section: use
the async variant, can be tackled separately, low priority.

For now the ORM works, but if sending an email fails (ie. SMTP down) it
will raise an exception that can end up in a 500 error, it would be great
if Django could retry so that emails leave when its SMTP is up again, would
that maybe weight in priority ?

Also, you would probably like to view the queue in the admin, maybe make an
unmanaged Model for tasks. So, as simple as Email would look like, I think
efforts in it could benefit in a much wider way too.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op18rkxnz04hVciJs_pHEvmfG2ir_bMafPaYQ7d9HGTJhoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Patryk Zawadzki

>
> That said, I also think it's important to allow the ORM to support both 
> modes in the long term. I truly believe the best way to be able to write 
> async code is to _have the choice to write it_, rather than being made to 
> all the time; if we make people use a separate, async ORM, then we force 
> them to write every view asynchronously, with all the extra danger and 
> thinking that requires. It's much better for Django to do the hard work, 
> and say "hey, if you want to write asynchronously or synchronously, that's 
> fine - it takes literally zero extra effort to go either way".
>

Slightly off-topic but once we have an async ORM, making it synchronous is 
not impossible (I believe either Channels or Daphne already have shims that 
use a worker thread to spin the event loop until a future is fulfilled).

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/402dbe77-44fc-485b-b058-e619a1576ddb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Aymeric Augustin
Hello Andrew,

Thanks for your work putting together this plan. Within our constraints, it's a 
good plan.

Regarding templating, I would say it isn't a priority because a developer who 
knows how to parallelize I/O bound operations will prefer (or at least accept) 
to perform these operations in the view, not in the template.

I'm on the fence about the convention for async APIs. I'm not super excited by 
spraying async code with _async prefixes. The namespacing approach would allow 
for cleaner async code. Most Python modules should be either sync or async, not 
a mix of both styles. But I might be underestimating the importance of 
explicitness...

I don't have much else to say on the DEP. It makes a lot of sense. I'm happy to 
see this happening!

Cheers,

-- 
Aymeric.



> On 9 May 2019, at 18:10, Andrew Godwin  wrote:
> 
> Hello everyone,
> 
> While the ASGI patch (https://github.com/django/django/pull/11209 
> ) is maybe the first small step 
> in a long road to async, it's the only real one we could do without a DEP as 
> it purely pulls on existing specs and bugs.
> 
> To that end, I have drafted a DEP (provisionally #0009), "Async-capable 
> Django". It is a summary - and I use that word very lightly - of the rest of 
> the work to be done to make a version of Django that one could call "async 
> capable".
> 
> You can view it here: 
> https://github.com/andrewgodwin/deps/blob/async/draft/0009-async.rst 
> 
> 
> It is approximately 7,000 words long - while it could probably do with some 
> editing, the subject matter means it has to be quite in-depth. Even then, it 
> does not propose an exact plan to follow; instead, it proposes the overall 
> strategy and the high-level ideas about what needs to be done.
> 
> If you are short on time, please read the Foreword and the High-Level 
> Summary, and then jump around using the table of contents to the sections 
> that interest you.
> 
> Feedback on this is very much appreciated; you can either reply here or, if 
> you have comments that would benefit from precise location on the diff, use 
> the pull request here: https://github.com/django/deps/pull/56 
> 
> 
> This is a very complex topic, and I appreciate feedback might take a while; 
> at least a couple of weeks to get everyone's thoughts. If you want a more 
> private channel to discuss concerns or design questions you're not happy to 
> talk about publicly, feel free to email me directly.
> 
> This proposal does not quite fit the DEP template - we never really planned 
> for something of this scale - but I have done my best to make it work; I 
> think we still benefit from discussing and voting on it in the DEP format, 
> but suggestions on how to make it work better are welcome, provided they fit 
> within how Django operates as a community now.
> 
> Finally, if there is positive feedback on this, we have the separate question 
> of funding it that I will discuss separately, if and when we get that far. 
> The DEP covers some of the topics, but it's notable that the implementation 
> speed of this varies considerably; if we could get, say, me and someone else 
> on it full-time (which is not a cheap or likely prospect), we can probably be 
> done inside of a single release. If we rely just on volunteer time, it may 
> take years. There is hopefully somewhere between those two that works.
> 
> Thanks for taking the time to read through!
> 
> Andrew
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAFwN1uqav3JXQA3m%2Bajf0Bd15QQX3JTZTfmH3Hc3ECWyRR3CVg%40mail.gmail.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/

Re: Django Async DEP

2019-05-09 Thread Andrew Godwin
> Slightly off-topic but once we have an async ORM, making it synchronous
is not impossible (I believe either Channels or Daphne already have shims
that use a worker thread to spin the event loop until a future is
fulfilled).

This is in fact the proposed way to make the ORM work with both sync and
async worlds once the interior is fully-async.

> In one project I really enjoyed using channels just as a background
worker instead of celery, and the DEP does not talk much in the Email
section: use the async variant, can be tackled separately, low priority.
>
> For now the ORM works, but if sending an email fails (ie. SMTP down) it
will raise an exception that can end up in a 500 error, it would be great
if Django could retry so that emails leave when its SMTP is up again, would
that maybe weight in priority ?

If you want guaranteed email delivery, that's a task for something like
Celery or a third-party API; any method of sending emails in the background
in the same process, be it threads or async coroutines, is going to be
unreliable as the server may die any time. I don't think it's sensible for
Django to try and solve this problem internally, at least not as part of
this async push.

Andrew

On Thu, May 9, 2019 at 12:56 PM Patryk Zawadzki  wrote:

> That said, I also think it's important to allow the ORM to support both
>> modes in the long term. I truly believe the best way to be able to write
>> async code is to _have the choice to write it_, rather than being made to
>> all the time; if we make people use a separate, async ORM, then we force
>> them to write every view asynchronously, with all the extra danger and
>> thinking that requires. It's much better for Django to do the hard work,
>> and say "hey, if you want to write asynchronously or synchronously, that's
>> fine - it takes literally zero extra effort to go either way".
>>
>
> Slightly off-topic but once we have an async ORM, making it synchronous is
> not impossible (I believe either Channels or Daphne already have shims that
> use a worker thread to spin the event loop until a future is fulfilled).
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/402dbe77-44fc-485b-b058-e619a1576ddb%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1urOnhEJDb6Sm45U59iF5hCZmN%3Dw-J%2B0er4UCnqh3qsmCw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Andrew Godwin
On Thu, May 9, 2019 at 1:04 PM Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> Hello Andrew,
>
> Thanks for your work putting together this plan. Within our constraints,
> it's a good plan.
>
> Regarding templating, I would say it isn't a priority because a developer
> who knows how to parallelize I/O bound operations will prefer (or at least
> accept) to perform these operations in the view, not in the template.
>
> I'm on the fence about the convention for async APIs. I'm not super
> excited by spraying async code with _async prefixes. The namespacing
> approach would allow for cleaner async code. Most Python modules should be
> either sync or async, not a mix of both styles. But I might be
> underestimating the importance of explicitness...
>
>
I am also not a fan of the approach, but I did err towards being explicit.
Jannis suggested what I think might be a nicer approach on Twitter, which
is to add an async "proxy object" to access methods with, e.g.:

cache.get("foo")
cache.async.get("foo")

This is still explicit but looks less ugly, and gives us a way in future to
make some modules transition to async-by-default (by e.g. supplying
cache.sync.get). What do you think?

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uowvL3FH9BR80hpOoBdRw-qUwpu5bA%3DBDLXmhY0WzBM%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Patryk Zawadzki

>
> I am also not a fan of the approach, but I did err towards being explicit. 
> Jannis suggested what I think might be a nicer approach on Twitter, which 
> is to add an async "proxy object" to access methods with, e.g.:
>
> cache.get("foo")
> cache.async.get("foo")
>
> This is still explicit but looks less ugly, and gives us a way in future 
> to make some modules transition to async-by-default (by e.g. supplying 
> cache.sync.get). What do you think?
>

I think I'd personally prefer to have to do `from django.asyncore.cache 
import cache` and if the synchronous cache raised an exception when 
attempting to use it with an event loop active. Like Aymeric, I don't see 
myself mixing async and sync code within one module.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d2c879cb-fad7-4e56-a1d9-1a31d5134e96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread J. Pic
Hi Andrew,

On Thu, May 9, 2019 at 10:04 PM Andrew Godwin  wrote:

>
> If you want guaranteed email delivery, that's a task for something like
> Celery or a third-party API; any method of sending emails in the background
> in the same process, be it threads or async coroutines, is going to be
> unreliable as the server may die any time. I don't think it's sensible for
> Django to try and solve this problem internally, at least not as part of
> this async push.
>

I'm a bit confused here, what benefit are you getting from async emails if
you're already retrying emails in the background in production ?

Thanks for your kind reply,

Have a great day

-- 
∞

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op18AA3Q7wguoZn%3D%2BfqHTAquXU0cESxF2i0GYzp%2BjB_ft%3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Aymeric Augustin
> On 9 May 2019, at 22:06, Andrew Godwin  wrote:
> 
> Jannis suggested what I think might be a nicer approach on Twitter, which is 
> to add an async "proxy object" to access methods with, e.g.:
> 
> cache.get("foo")
> cache.async.get("foo")
> 
> This is still explicit but looks less ugly, and gives us a way in future to 
> make some modules transition to async-by-default (by e.g. supplying 
> cache.sync.get). What do you think?


I don't think it makes a significant difference from a readability perspective 
in this example. It does have some advantages, though:

- It could be easier to implement a sync version based on the async one, or 
vice-versa, if each one has its own class. It will probably be more DRY.
- It could also be a bit more usable by developers, especially in IDEs

Also it could provide an alternative solution for the async attribute access 
problem: `model_instance.async.related_field` could be a Future that, when 
awaited, resolves to the related field. So you could `await 
model_instance.async.related_field` instead of having to `select_related`. To 
be honest I didn't investigate all the consequences of this idea. It seemed 
worth mentioning, though, even if it turns out to be impractical :-)

We've always considered that implicit queries on attribute access were an 
intractable problem. I said it on stage an DjangoCon US 2013. I'm now wondering 
if I was wrong all along! In an async ORM context, every time we traverse a 
relation, we could create a Future that would execute the query and resolve to 
its result. This would require one await per SQL query so we'd still get the 
benefit of making queries explicit, although to a lesser extent than with an 
explicit select/prefetch_related.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/258CBCB2-2456-439F-82DF-E8760AE5EEF0%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread J. Pic
Nevermind my question you will get a lot more out of the workers, that
Django 3.0 is going to be really blazing fast like channels that calls for
a celebration xD

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op1-1apTUFZq9z5%3DEanpL%2B%2BA9nx9%3D3zS6_C-ZGNtJyBacog%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Patryk Zawadzki
W dniu czwartek, 9 maja 2019 22:47:48 UTC+2 użytkownik J. Pic napisał:
>
> I'm a bit confused here, what benefit are you getting from async emails if 
> you're already retrying emails in the background in production ?
>

Anything that uses I/O should be async to unblock the worker to process 
other things while for example waiting for an SMTP server to return. 
Network I/O like that often requires tens to hundreds of milliseconds, in 
this time you could process another request from another browser.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2ce12106-b423-4dc1-a95a-56f112e63a0f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Patryk Zawadzki

>
> I don't think it makes a significant difference from a readability 
> perspective in this example. It does have some advantages, though: 
>
> - It could be easier to implement a sync version based on the async one, 
> or vice-versa, if each one has its own class. It will probably be more DRY. 
> - It could also be a bit more usable by developers, especially in IDEs 
>

Speaking about convenience, it would be great if we could make the sync 
versions raise exceptions when used with an event loop attached and the 
async ones fail when there isn't a loop running. I assume synchronous views 
would be executed within a worker thread with the event loop explicitly set 
to None. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5b6222c3-9f76-4eb3-8619-e3f44a17d6a4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread KimSia Sim
Hi Andrew, 

I joined this group and chat because I saw the twitter post you made about 
this DEP. I  find that interesting.


On Friday, May 10, 2019 at 3:49:30 AM UTC+8, Andrew Godwin wrote:
>
>  Tom Christie, for example, has already started work on an asynchronous 
> ORM. Some of Django's biggest sites don't use the ORM - for example, 
> Instagram.
>
>>
>>
I have a question that strays slightly away from the main topic. I have 
looked at Tom's repos. Is it encode/databases 
 https://github.com/encode/databases that you're referring to?

Or do you mean Tom's working on an async ORM that works within Django?

Thank you. I think Django will be even more awesome with async features

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/521a0e83-a23d-4d0b-a40f-45ecd30c4ef6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Andrew Godwin
>
> I don't think it makes a significant difference from a readability
> perspective in this example. It does have some advantages, though:
>
> - It could be easier to implement a sync version based on the async one,
> or vice-versa, if each one has its own class. It will probably be more DRY.
> - It could also be a bit more usable by developers, especially in IDEs
>

Right. It could also be a sub-class on the class in question to make the
implementation easier - e.g. "core.cache.cache.sync" is a literal
passthrough, and "core.cache.cache.async" is the async wrapper
implementation.


>
> Also it could provide an alternative solution for the async attribute
> access problem: `model_instance.async.related_field` could be a Future
> that, when awaited, resolves to the related field. So you could `await
> model_instance.async.related_field` instead of having to `select_related`.
> To be honest I didn't investigate all the consequences of this idea. It
> seemed worth mentioning, though, even if it turns out to be impractical :-)
>

I think this might be feasible? I'd want to probably push it to have its
own research and proposal, probably, but it might be a nice way out of the
initial thing of requiring select_related. I just don't know enough about
how that might cascade down the ORM internals to judge it at this point!

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1upEsCpnL-81guSw_vKLn%3DWmfM3pamuFY%2BSZWLMerNC0yA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Andrew Godwin
On Thu, May 9, 2019 at 2:15 PM Patryk Zawadzki  wrote:

> I don't think it makes a significant difference from a readability
>> perspective in this example. It does have some advantages, though:
>>
>> - It could be easier to implement a sync version based on the async one,
>> or vice-versa, if each one has its own class. It will probably be more DRY.
>> - It could also be a bit more usable by developers, especially in IDEs
>>
>
> Speaking about convenience, it would be great if we could make the sync
> versions raise exceptions when used with an event loop attached and the
> async ones fail when there isn't a loop running. I assume synchronous views
> would be executed within a worker thread with the event loop explicitly set
> to None.
>

My ASGI patch already does this with the @async_unsafe decorator all over
the ORM so people can't screw up by accident. It would be quite easy to
extend this to enforcement on both the sync and async versions - there's
maybe an edge case that you can call an async function from a thread you
have not started an event loop in _yet_, but I'd rather see if and when
that happens and provide a workaround, maybe.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uq4HX%3DHQa9Gmymckv-T_qXbGgA%3DtUrHn2pG8ejVSML3GA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django Async DEP

2019-05-09 Thread Andrew Godwin
On Thu, May 9, 2019 at 2:44 PM KimSia Sim  wrote:

> I have a question that strays slightly away from the main topic. I have
> looked at Tom's repos. Is it encode/databases
> https://github.com/encode/databases that you're referring to?
>
> Or do you mean Tom's working on an async ORM that works within Django?
>

It is indeed encode/databases!

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uoRf2Ug5b870zspfudncKPK%3DaWjff-z%2BGCFbW53pGsxxQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.