Hi Pete, A pluggable backend sounds like a good idea.
Regarding DJANGO_ENV, why not ship the template with two settings files and use the existing DJANGO_SETTINGS_MODULE, instead of adding a new environment variable? Maybe that would be controversial too, I'm not sure, but perhaps less so than introducing a new paradigm? In any event, perhaps a pluggable backend would allow everyone to be happy, or very nearly so... ๐ To be clear, I'm fully in support of this, no matter how it ends up. Providing a decent production-capable server and some sane settings/configuration defaults for people to work from -- rather than having to scour the documentation for every mention of a production environment and figure out how to make those changes by hand -- will be a big win IMO. As it stands, first time users might reasonably assume they should use Apache to deploy to production (maybe they should, I'm not sure...but I haven't seen it done in over a decade): "Weโve included this with Django so you can develop things rapidly, without having to deal with configuring a production server โ such as Apache โ until youโre ready for production." https://docs.djangoproject.com/en/4.1/intro/tutorial01/ Cheers, Tobias On Sat, Oct 29, 2022, 7:15 PM Peter Baumgartner <p...@lincolnloop.com> wrote: > > > On Sat, Oct 29, 2022 at 10:06 AM Peter Baumgartner <p...@lincolnloop.com> > wrote: > >> Thanks for the thorough review Florian! Some comments inline... >> >> On Wed, Oct 26, 2022 at 1:30 AM Florian Apolloner <f.apollo...@gmail.com> >> wrote: >> >>> Hi Pete, >>> >>> this does look interesting. And I agree that this is something that >>> would be nice to have in core in one form or another. That said I think we >>> have to do it iteratively (ie in smaller parts) and evaluate along the way. >>> >> >> Agreed. I think adding the additional settings with some sort of >> production switch is the least obtrusive place to start. >> >> >>> >>> Since you asked for feedback, I'll tell you what I'd do differently (and >>> this is also a sign that production environment vary widely). >>> >> >> Production environments do vary widely, but I don't think that should >> stop us from providing defaults that are one-size-fits-most. startproject >> already does this and users are welcome to adjust it how they see fit. >> >> >>> >>> * As soon as health checks check something aside from simply returning >>> a 200 status code I'd want to be able to limit access to them >>> >> >> Yep, that seems reasonable. Although depending on the level of failure, >> you may not be able to log in to see them. Starting with just a view that >> responds with a 200 is reasonable. >> >> >>> * I like my dev env to be as close as possible to production so I >>> usually also use whitenoise in dev. I had to many times where dev worked >>> and whitenoise in prod failed then :D >>> >> >> Also a good point... I'll add that. >> >> >>> * I do hate `DJANGO_ENV` but I guess there is no way around it. >>> Currently my projects often piggy-back on DEBUG but that can be rather hard >>> from time to time >>> >> >> I'm open to other suggestions here, but am kind of in the same boat. >> There may be times when you briefly want DEBUG=True in a non-public >> deployed environment or DEBUG=False in a development environment. >> Overloading that makes those scenarios a challenge. >> >> >>> * I prefer settings in settings files (ini/yaml/toml) as opposed to env >>> variables that leak down the whole process-tree. django-environ does >>> support this; that said there are other options like dynaconf which also >>> support hashicorp vault which is a massive plus imo. Just being able to >>> utilize vault for free from every django application out there would be >>> stellar. >>> >> >> I'm 100% in agreement here. I wrote goodconf [1] for this before I knew >> about dynaconf which does a lot of the same things. I would love to see >> Django adopt a split between "settings" (which I see as things that don't >> change across environments) and "configuration" (things that you set per >> environment). I did switch to django-environ's FileAwareEnv [2] which >> allows loading from files, but not in the yaml./toml sense. I didn't go all >> in here because I felt like it was a more controversial stance and at the >> end of the day, you probably still need to support environment variables >> because many (most?) PaaS providers don't have support for files like that. >> >> With the goal of getting something merged into Django eventually, I felt >> like environment variables were the lowest bar. We have precedence for >> loading from environment variables with DJANGO_SETTINGS_MODULE. Do you >> think a more full-featured library like dynaconf would make it harder to >> push some of this upstream? Users that want to use dynaconf or goodconf or >> whatever can still edit their settings and use those like they do today. >> >> [1] https://github.com/lincolnloop/goodconf >> [2] >> https://django-environ.readthedocs.io/en/latest/tips.html#docker-style-file-based-variables >> > > Had another thought here... what about providing a pluggable configuration > backend? Django could ship with some native backends like environment > variables and toml (when it's available). Third party packages could > provide Vault, secret files, etc. Same as how storages currently works and > how you can easily drop in something like django-storages. > > >> >> >>> >>> Another thing about settings, and this is something I really like in >>> spring-boot is their relaxed binding for configuration variables. You can >>> look at >>> https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0 >>> for an example. They describe there how hierarchical variables are handled >>> and translated between yaml and env variables. Of course this binding is >>> not exactly bidirectional always (I think) but it allows for a few nice >>> things. First off you can start with a config file in yaml (one might >>> dislike yaml but I find it easier to read than say a .env file with 100 >>> settings): >>> >>> ``` >>> db: >>> host: myhost >>> port: 1234 >>> >>> caches: >>> default: >>> type: memory >>> second: >>> type: redis >>> ``` >>> >>> which would translate to the following env variables: >>> ``` >>> DB_HOST: myhost >>> DB_PORT: 1234 >>> CACHES_DEFAULT_TYPE: memory >>> CACHES_SECOND_TYPE: redis >>> ``` >>> >>> This is imo really a powerful concept and allows for so much nicer and >>> easier definitions. >>> >>> Cheers, >>> Florian >>> >>> On Friday, October 21, 2022 at 9:56:50 PM UTC+2 pe...@lincolnloop.com >>> wrote: >>> >>>> I put together a first stab at this at >>>> https://github.com/lincolnloop/django-production. Feedback is >>>> appreciated! >>>> >>>> On Tue, Aug 23, 2022 at 1:34 PM pe...@lincolnloop.com < >>>> pe...@lincolnloop.com> wrote: >>>> >>>>> I hadn't seen django-simple-deploy. At first glance, it looks >>>>> interesting, but more platform-specific than what I was thinking. >>>>> >>>>> I'll play around with creating a third-party app that encapsulates >>>>> some of these thoughts and see where I end up. >>>>> >>>>> Thanks for your time and feedback! >>>>> >>>>> On Tuesday, August 23, 2022 at 1:17:12 PM UTC-6 carlton...@gmail.com >>>>> wrote: >>>>> >>>>>> Hey Pete. >>>>>> >>>>>> Yes, there can be a lot of steps... (and trouble is every one has a >>>>>> different opinion once you get to specifics :) >>>>>> >>>>>> django-webserver looks good โ in a similar ballpark to what I had in >>>>>> mind, yes, but I'd probably not want to bundle the options for each >>>>>> server >>>>>> myself, as that seems a lot to maintain over time... (Better to outsource >>>>>> to the individual projects no?) >>>>>> >>>>>> Have you seen what Eric Matthes is doing with django-simple-deploy? >>>>>> >>>>>> https://github.com/ehmatthes/django-simple-deploy >>>>>> >>>>>> I really like the idea: in an opinionated way, it applies the project >>>>>> changes you need to deploy (and it's plugable so folks could provide a >>>>>> *template* for their own flavour of deployment if they wanted, which is >>>>>> kind of important given the number of options in the landscape, even if >>>>>> you >>>>>> only to *ship* a beginner option.) >>>>>> >>>>>> Tim Allen also recently raised improving the Django project template, >>>>>> which I think is related here. (That's on my list too: async this QTR, >>>>>> then >>>>>> hoping to take on Adam's proposal to modernise the Request object for the >>>>>> end of year, then ๐ค swinging back here for "OK, what does it look like, >>>>>> what can we do?" after that: there's a bunch of related tickets around >>>>>> #21978 that it would be very sweet if we could clear up...) >>>>>> >>>>>> I think all these thoughts are really pursuable outside of core in >>>>>> the very short run, even if the goal is to merge them โ it's much easier >>>>>> to >>>>>> experiment that way, and then say, "This worked". >>>>>> >>>>>> Anyhow, other than a "I've been thinking along these lines", and a >>>>>> generally positive emote, I don't have too much more concrete at this >>>>>> stage. >>>>>> ๐ >>>>>> C. >>>>>> >>>>>> On Tue, 23 Aug 2022 at 19:00, Peter Baumgartner < >>>>>> pe...@lincolnloop.com> wrote: >>>>>> >>>>>>> Thanks for the additional background Carlton! I like the idea of >>>>>>> having some sort of protocol that folks can follow to make >>>>>>> ready-to-use Django server packages (django-webserver [1] is in the >>>>>>> same vein), but I'd strive for more. pip install + INSTALLED_APPS is >>>>>>> great, but getting to production with that approach can feel like >>>>>>> death by a thousand paper cuts. Do that with gunicorn, whitenoise, >>>>>>> dj-database-url, etc. and it ends up being a laundry list of things >>>>>>> that, in my experience, people rarely get right on the first try and >>>>>>> just result in frustration. >>>>>>> >>>>>>> It seems like part of the challenge here is reaching parity with the >>>>>>> current runserver command, but is that necessary? Running `runserver >>>>>>> --dev` locally and getting dev features like autoreloading and then >>>>>>> having `runserver` use a different server for a deployed app doesn't >>>>>>> seem unreasonable. >>>>>>> >>>>>>> [1] https://github.com/lincolnloop/django-webserver >>>>>>> >>>>>>> On Tue, Aug 23, 2022 at 10:14 AM Carlton Gibson >>>>>>> <carlton...@gmail.com> wrote: >>>>>>> > >>>>>>> > Hey Pete. >>>>>>> > >>>>>>> > Yes... this is a good one. >>>>>>> > >>>>>>> > It's difficult to see how we can even have an ASGI runserver in >>>>>>> core, simply because there's no SimpleServer equivalent as there is for >>>>>>> WSGI, and it's not clear one is on the card. (HTTP is getting **more** >>>>>>> complex, and I can't see Python bringing such into the standard >>>>>>> library.) >>>>>>> > So we have to use a third-party dependency for that. >>>>>>> > >>>>>>> > For the Channels v4 update (in progress now) I'm moving the >>>>>>> runserver command to the Daphne package. This was because folks wanted >>>>>>> to >>>>>>> use Channels without the Daphne/Twisted dependency. >>>>>>> > >>>>>>> > See: https://github.com/django/daphne/pull/429 >>>>>>> > >>>>>>> > tl;dr โ it came out pretty clean TBH. You pip install daphne, then >>>>>>> add "daphne" at the top of INSTALLED_APPS. (There's a system check to >>>>>>> make >>>>>>> sure you're not fighting staticfiles.) >>>>>>> > >>>>>>> > On the back of that I don't see why servers (any servers) can't >>>>>>> provide a runserver command (or a django-* mini-package to go with it) >>>>>>> and >>>>>>> offer what they offer in development as well as production. (Obviously >>>>>>> that >>>>>>> all needs writing but it's just a management command.) >>>>>>> > >>>>>>> > Related is strategising the reloader. >>>>>>> https://code.djangoproject.com/ticket/30213 >>>>>>> > >>>>>>> > We could then host *Protocols* โ "Your runserver should do this", >>>>>>> "Implement this for your reloader", and so on โ and (maybe) (slowly) >>>>>>> replace built-in options here with either optional dependencies (pip >>>>>>> install django["daphne"]) or recommendations to get going. (pip install >>>>>>> django["starter"] maybe). โ I don't know โ we seem a long way from >>>>>>> that. (I >>>>>>> don't suppose wsgiref is going anywhere too soon is it?) >>>>>>> > >>>>>>> > But to cut back to your point, I don't know that we need to >>>>>>> **include** a server โ just make it easy to get going. These days pip >>>>>>> install + a line in INSTALLED_APPS doesn't seem too much to ask (I >>>>>>> think.) >>>>>>> > >>>>>>> > Was it really 4 years ago Tom left that comment on #21978 ๐ณ (We'd >>>>>>> finally close this as wontfix: Django doesn't ship a webserver.) >>>>>>> > >>>>>>> > Does that fit in your view, or would that scenario not be good >>>>>>> enough? >>>>>>> > >>>>>>> > Kind Regards, >>>>>>> > >>>>>>> > Carlton >>>>>>> > >>>>>>> > On Tue, 23 Aug 2022 at 16:56, Peter Baumgartner < >>>>>>> pe...@lincolnloop.com> wrote: >>>>>>> >> >>>>>>> >> Hi all! I'd like to re-open the discussion around >>>>>>> >> https://code.djangoproject.com/ticket/21978 >>>>>>> >> >>>>>>> >> As a "batteries included" framework, Django kind of leaves people >>>>>>> to >>>>>>> >> fend for themselves when it comes to deployment. This makes it >>>>>>> harder >>>>>>> >> than necessary for a developer to go from works-on-my-laptop to >>>>>>> >> works-on-the-internet. The docs here are great, but also daunting >>>>>>> to a >>>>>>> >> newcomer https://docs.djangoproject.com/en/4.1/howto/deployment/ >>>>>>> >> >>>>>>> >> I'd love to see Django provide some sane defaults that people can >>>>>>> use >>>>>>> >> to deploy a site without having to make a bunch of decisions they >>>>>>> >> don't totally understand. >>>>>>> >> >>>>>>> >> I have two thoughts on how it could be handled: >>>>>>> >> >>>>>>> >> 1. Similar to the template backends. You get Django's template >>>>>>> system >>>>>>> >> by default, but you're welcome to swap in Jinja2. I don't believe >>>>>>> >> there is a one-size-fits-all webserver, but probably a >>>>>>> >> one-size-fits-most. >>>>>>> >> 2. Similar to cache/database backends. You define the backend you >>>>>>> want >>>>>>> >> to use and Django makes some sane choices for you and allows you >>>>>>> to >>>>>>> >> serve the site via some common `manage.py` command. >>>>>>> >> >>>>>>> >> For the first option, Gunicorn is a popular choice, but afaik it >>>>>>> does >>>>>>> >> not work on Windows. Waitress [1] is an interesting option for >>>>>>> WSGI >>>>>>> >> serving and daphne exists for ASGI. Whitenoise is a reasonable >>>>>>> choice >>>>>>> >> for serving static files. It doesn't yet support ASGI, but there >>>>>>> has >>>>>>> >> been some activity on that front [2]. >>>>>>> >> >>>>>>> >> Thanks! >>>>>>> >> >>>>>>> >> [1] https://pypi.org/project/waitress/ >>>>>>> >> [2] https://github.com/evansd/whitenoise/pull/359 >>>>>>> >> >>>>>>> >> -- >>>>>>> >> You received this message because you are subscribed to the >>>>>>> Google Groups "Django developers (Contributions to Django itself)" >>>>>>> group. >>>>>>> >> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to django-develop...@googlegroups.com. >>>>>>> >> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/django-developers/CAC6K9zk5PmcxYWXRdvco1pWXnO%2BHoYoYHf0pg5Mw%3DgmdefZArg%40mail.gmail.com >>>>>>> . >>>>>>> > >>>>>>> > -- >>>>>>> > You received this message because you are subscribed to the Google >>>>>>> Groups "Django developers (Contributions to Django itself)" group. >>>>>>> > To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to django-develop...@googlegroups.com. >>>>>>> > To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/django-developers/CAJwKpyTSGPJW%2BmdUADW8eBYaXakesHG9H9dkSzhLxWnCpBdH6A%40mail.gmail.com >>>>>>> . >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Django developers (Contributions to Django itself)" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to django-develop...@googlegroups.com. >>>>>>> >>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/django-developers/CAC6K9zmVDKcpM%3Dc9tyhrp2_K_dzLH5S9zMJcZSXduF9NkcoCKA%40mail.gmail.com >>>>>>> . >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Django developers (Contributions to Django itself)" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to django-develop...@googlegroups.com. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/django-developers/493f7f12-541c-4e12-ac25-0fd341ce2dc2n%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/django-developers/493f7f12-541c-4e12-ac25-0fd341ce2dc2n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django developers (Contributions to Django itself)" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to django-developers+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-developers/c79db112-a965-4700-90b2-2bcdcd9d65a8n%40googlegroups.com >>> <https://groups.google.com/d/msgid/django-developers/c79db112-a965-4700-90b2-2bcdcd9d65a8n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/CAC6K9zkoksAxEs1NPAVOw-e8JOn3Xn0w-NLbMOmm2gZOEHV-Hg%40mail.gmail.com > <https://groups.google.com/d/msgid/django-developers/CAC6K9zkoksAxEs1NPAVOw-e8JOn3Xn0w-NLbMOmm2gZOEHV-Hg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMGFDKRGRtfqKASW3Re2snr_HNGtmwYjTEOGFqC_U3X%3DToCq3Q%40mail.gmail.com.
