Re: test discovery

2013-05-13 Thread Daniel Lindsley
Russ & Carl,


I'm interested in this one, so I sketched up a potential avenue tonight 
(https://github.com/toastdriven/django/compare/check_setup). I actually 
implemented Russ' setup first, but ran into a couple issues, so I pared it 
down into an "opt-in" scheme. If there's a feeling like this is a 
worthwhile start, I'd be happy to flesh it out further, adding tests docs, 
refinements, etc. If not, no big deal.


Daniel



On Friday, May 10, 2013 8:32:38 PM UTC-7, Carl Meyer wrote:
>
> On Thursday, May 9, 2013 8:15:02 PM UTC-4, Russell Keith-Magee wrote:
>
>> On Thu, May 9, 2013 at 11:30 PM, Carl Meyer  wrote:
>>
>>> Hi Russ,
>>>
>>> On 05/09/2013 04:51 AM, Russell Keith-Magee wrote:
>>> > Great work Carl and Preston! (And everyone else who had a hand in the
>>> > patch - I know this has been kicking around for a while now)
>>>
>>> Thanks for the reminder; I should have mentioned that Mahdi Yusuf did
>>> the initial draft patch, based on Jannis' django-discover-runner, which
>>> was based on code I showed on a slide in my "Django and Testing" talk at
>>> PyCon 2012; it all comes full circle :-)
>>>
>>> > I agree with Jacob's tests aren't production sensitive, so making a 
>>> fast
>>> > transition to the new test discovery has less impact.
>>> >
>>> > However, we know from experience that it doesn't matter how much we 
>>> flag
>>> > this change in the release notes, they won't be read, and *someone* is
>>> > going to get stung. In an ideal world, it would be good to be able to
>>> > warn people who upgrade that their test suite may not be run as 
>>> expected.
>>> >
>>> > We've had a proposal kicking around for a while to add a management
>>> > command that does an "upgrade check". I'm wondering if this might be an
>>> > opportune time to dig into this some more.
>>> >
>>> > To be clear, this isn't something I consider to be a merge blocker. I'd
>>> > be happy to see 1.6 released with a fast transition to the new test
>>> > runner. It would just be nice icing on the cake if we can get an 
>>> upgrade
>>> > validator included in the same release.
>>>
>>> It wouldn't be too hard to write something that tried test discovery
>>> using the old runner and using the new runner and did a simple
>>> comparison of test counts to warn you if they differ.
>>>
>>> I'm not personally likely to work on this "upgrade checks" framework,
>>> but if it does happen I'd be happy to contribute this check to it.
>>
>>  
>> I thought about that approach, but my immediate reaction was that it will 
>> end up producing more false positives than helpful feedback. Even the 
>> simple case -- myapp.TestCase -> myapp.tests.TestCase -- is going to 
>> require some name munging to work out if the old test and the new test are 
>> actually the same one. Since this is for a short-lived transitional tool, 
>> the effort involved in getting it right exceeds the value gained by having 
>> it, IMHO.
>>
>
> I wasn't anticipating trying to actually match up individual tests by 
> name/path, just trying both runners and seeing whether the test counts 
> match. On second thought, though, I realize that contrib / third-party apps 
> will mean this check would alert "missing tests!" for pretty much everyone, 
> so it would need to be smarter to be useful, and you're right that's likely 
> not worth it.
>  
>
>> I'd prefer something much simpler:
>>
>>  * On syncdb/validate, check for a marker somewhere in user space. 
>>
>>  * If the marker isn't present, do a check for likely problems. In this 
>> case, look for the value of TEST_RUNNER; if it's set to the new default 
>> value, display a warning about the change in test discovery format. Add 
>> similar checks for other setting changes, or warnings about features that 
>> have been fully deprecated (e.g., the final transition of the {% url %} 
>> behavior)
>>
>>  * Provide a management command to set marker. 
>>
>>  * On the next syncdb/validate, the marker is present, so the install is 
>> verified as being "updated", and no warnings are displayed.
>>
>> This leaves the question of where to put the marker. Some initial ideas:
>>
>>  - a file on disk (e.g., an .updated file next to the settings file)
>>  - a new setting (VERIFIED_VERSION = "1.5")
>>  - a key-value table in a database used only for Django administrivia
>>
>> Essentially, this would give us a place to put the "NO REALLY, READ THIS" 
>> warnings that we need in release notes, and make those messages slightly 
>> targeted.
>>
>
> In theory this makes sense to me, but I think the question of where to put 
> the "I already ran the upgrade check" marker is thorny. Database is ugly, 
> since there's no requirement for a Django project to even use the ORM at 
> all; and requiring projects using the ORM to have a magical extra 
> not-related-to-a-voluntarily-installed-app table in their database sticks 
> in my craw. Similar with adding a file to the filesystem; we'd essentially 
> be requiring everyone to have this file in order t

Re: Python 3 and you

2011-09-14 Thread Daniel Lindsley
Jannis,


   "You have my sword." I want to see this happen & would love to be a
part of it. A couple questions:

* How should patches be provided? Trac? BitBucket?
* Where should feedback go? This mailing list? Somewhere else?
* This is further off, but once we have a ported Django, how do get
the community (specifically pluggable apps) onboard? I'm assuming the
docs are meant to do this but wondering if there's anything else we
can be doing (like perhaps a Django-specific 2to3 (extension?) to
cover common Django conventions).
* Do we have a target date? I know this is hard with a volunteer-only
effort, but if we setup some sort of timeline, we'd at least have a
metric & something to shoot/push for.

   Finally, a philosophical question on approach: Should we really be
doing 2to3 (leaving the Django codebase in Python 2.X for a long time)
or would it be better to port Django over to Python 3 & use 3to2 for
existing Python 2.X installs? I confess I don't know much about the
current state of 3to2 (nor how most other Python libraries are
handling the transition). But I do know Django will continue to grow
over time & I worry that, at some point in the future we'll be making
more even more work for someone else to do the 3-only work.

   Regardless, I'm excited & will see what I can do to help.


Daniel



On Sep 14, 8:03 am, Jannis Leidel  wrote:
> Hi all,
>
> After last week's sprint I wanted to get you up-to-speed about the
> current state of porting Django to Python 3.
>
> As some may be aware Martin von Löwis has been working on a port for
> a while [1] but only recently I've had the chance to meet with him and
> talk through the porting process.
>
> I'm not going to hide the fact that it'll be a long process, but I'm
> also convinced it's an important step for Django to make. I'm writing
> this in the hope to find volunteers to join the porting efforts.
>
> Goals
> -
>
> To allow Django to run on Python 3 there are several goals to achieve,
> some of which are our respsonsibility, some depend on 3rd party libraries
> we use internally and some left to the users that use Django to build
> their websites. It's my understanding that we can't solve everything
> at once, so take this with a grain of salt:
>
> - get Django to run on Python 3
> - provide helpers and docs for porting Django-based projects
> - help out 3rd party projects we rely only to make the jump (if needed)
>
> Porting strategies
> --
>
> As you can imagine there are still quite a few open questions at
> the moment about specific porting problems but taking from the
> experience in the Python community I think we have a good general
> strategy.
>
> There are a few assumptions we're applying either because it's
> unrealistic or impossible to maintain as long as Python 2.X is in
> use for the forseeable future; so these strategy *don't* work:
>
> - Create a Python 3 only port ("burning the bridges")
>
>   This is outright a no-go since it would leave all the Python 2.X
>   projects in dead water. Instead we need to provide a migration
>   path for them.
>
> - Maintaing a separate Python 3 branch ("dual releases")
>
>   While this would allow for new projects to use Python 3, I'm
>   convinced this has the potential to split the community. It'd
>   also be a major burden for the core team to maintain both
>   branches. Instead we need a combined effort.
>
> So as a result of that the only viable option is to support both major
> versions of Python at the same time, with the same code base.
>
> Fortunately the Python community gained lots of experience in the past
> years to make this happen (e.g. Lennart Regebro's book [4]). There are
> also tools to ease the transition of Django and the Django-based
> projects. Some of which are:
>
> - six [3] -- a compatibility library that includes many (if not all)
>   needed import proxies and utilities to prepare Django and Django-based
>   projects to be ported to Python 3.X. This only applies to API that
>   isn't syntactically changed, but only moved or enhanced in 3.X.
>
> - 2to3 [2] -- an extensible library which is able to translate the rest
>   of the Python 2 code to the Python 3 equivalent. For every Django
>   specific feature that isn't covered by the default 2to3 "fixers" we can
>   write our own if needed. It integrates with distutils (in Python 3.X)
>   and is able to convert Django at installation time. Installing Django
>   with Python 2 wouldn't trigger the translation process, of course.
>
> Code status
> ---
>
> During the sprint we've moved Martin's code from a Bitbucket clone to
> an own SVN branch:
>
>  https://code.djangoproject.com/browser/django/branches/features/py3k/
>
> Some notable changes:
>
> - a modified ``setup.py`` which automatically calls 2to3 during installation
>
> - a ``py3ktest`` helper bash script which -- for now -- installs Django in
>   a directory called "3k" in the same directory to trigger the translation
>   from Python 2 to Pyth

Re: Python 3 and you

2011-09-14 Thread Daniel Lindsley
Jannis,


   I wasn't trying to suggest we leave anyone behind, far from it. I
was suggesting move the code to Python 3 now, while there's less code
there (than some future date) but using 3to2[1] to help others on
Python 2.X. Since Django still supports 2.5, it's possible that this
isn't even an option, as I don't know if 3to2 can translate back that
far reliably. Simply getting the question out there for others to mull
over.


Daniel



On Sep 14, 10:36 am, Jannis Leidel  wrote:
> Daniel,
>
> >   "You have my sword." I want to see this happen & would love to be a
> > part of it.
>
> Huzzah!
>
> > A couple questions:
>
> > * How should patches be provided? Trac? BitBucket?
>
> For now via Trac, that's why we've moved the changes into a SVN branch.
> Unless anyone has a better idea I could create a Trac component "Python 3"
> so we can track the tickets easily.
>
> > * Where should feedback go? This mailing list? Somewhere else?
>
> Feedback should go here, on the developers mailing list, to get as many
> eyes on it as possible.  
>
> > * This is further off, but once we have a ported Django, how do get
> > the community (specifically pluggable apps) onboard? I'm assuming the
> > docs are meant to do this but wondering if there's anything else we
> > can be doing (like perhaps a Django-specific 2to3 (extension?) to
> > cover common Django conventions).
>
> Very good question, I'm uncertain as to how the "helpers" I mentioned
> will look like in the end. Whether they will be part of Django (e.g.
> a management command to run 2to3 on an app) or if we "only" provide the
> necessary compatibility library (e.g. "six") so that 3rd party app
> authors would still keep writing apps with Python 2 but would allow
> their apps to be translated to Python 3 automatically. Documenting ways
> of how to write a setup.py to do the conversion during install time
> is *in* the scope of what we need to provide, IMO. Whether we need
> Django-specific 2to3 fixers isn't clear at this time as the porting
> has only just begun.
>
> > * Do we have a target date? I know this is hard with a volunteer-only
> > effort, but if we setup some sort of timeline, we'd at least have a
> > metric & something to shoot/push for.
>
> One assumption of the strategy I outlined was the fact that Django is
> as close to 3.X as possible. Django 1.4 will require Python 2.5 or
> higher, but I'm not sure how quick we can do the jump to 2.6, which
> is recommended by the Python porting docs [1].
>
> >   Finally, a philosophical question on approach: Should we really be
> > doing 2to3 (leaving the Django codebase in Python 2.X for a long time)
> > or would it be better to port Django over to Python 3 & use 3to2 for
> > existing Python 2.X installs? I confess I don't know much about the
> > current state of 3to2 (nor how most other Python libraries are
> > handling the transition). But I do know Django will continue to grow
> > over time & I worry that, at some point in the future we'll be making
> > more even more work for someone else to do the 3-only work.
>
> I personally haven't ported a 2.X library completely to 3.X yet, so I
> can also only guess. But from what I've seen in the community I'm afraid
> of a "clean cut" port because it has a high risk of leaving many projects
> and apps behind. In that sense it seems more sensible to me to see the
> port to Python 3 just as another step of our Python version deprecation
> policy, which we at some point take with a complete conversion. Basically
> a "burn bridges as soon as everyone is safe" approach :)
>
> I don't dare to guess when that moment could be though, but it would probably
> happen after a potential Python 2.7 only release of Django -- whenever that 
> is.
>
> Jannis
>
> 1:http://docs.python.org/py3k/howto/pyporting.html#try-to-support-pytho...
>
>
>
>
>
>
>
>
>
> > On Sep 14, 8:03 am, Jannis Leidel  wrote:
> >> Hi all,
>
> >> After last week's sprint I wanted to get you up-to-speed about the
> >> current state of porting Django to Python 3.
>
> >> As some may be aware Martin von Löwis has been working on a port for
> >> a while [1] but only recently I've had the chance to meet with him and
> >> talk through the porting process.
>
> >> I'm not going to hide the fact that it'll be a long process, but I'm
> >> also convinced it's an important step for Django to make. I'm writing
> >> this in the hope to find volunteers to join the porting efforts.
>
> >> Goals
> >> -
>
> >> To allow Django to run on Python 3 there are several goals to achieve,
> >> some of which are our respsonsibility, some depend on 3rd party libraries
> >> we use internally and some left to the users that use Django to build
> >> their websites. It's my understanding that we can't solve everything
> >> at once, so take this with a grain of salt:
>
> >> - get Django to run on Python 3
> >> - provide helpers and docs for porting Django-based projects
> >> - help out 3rd party projects we rely only to make the jump (if neede

Re: Python 3 and you

2011-09-14 Thread Daniel Lindsley
Help to cite appropriately. [1] was http://pypi.python.org/pypi/3to2.


On Sep 14, 10:55 am, Daniel Lindsley  wrote:
> Jannis,
>
>    I wasn't trying to suggest we leave anyone behind, far from it. I
> was suggesting move the code to Python 3 now, while there's less code
> there (than some future date) but using 3to2[1] to help others on
> Python 2.X. Since Django still supports 2.5, it's possible that this
> isn't even an option, as I don't know if 3to2 can translate back that
> far reliably. Simply getting the question out there for others to mull
> over.
>
> Daniel
>
> On Sep 14, 10:36 am, Jannis Leidel  wrote:
>
>
>
>
>
>
>
> > Daniel,
>
> > >   "You have my sword." I want to see this happen & would love to be a
> > > part of it.
>
> > Huzzah!
>
> > > A couple questions:
>
> > > * How should patches be provided? Trac? BitBucket?
>
> > For now via Trac, that's why we've moved the changes into a SVN branch.
> > Unless anyone has a better idea I could create a Trac component "Python 3"
> > so we can track the tickets easily.
>
> > > * Where should feedback go? This mailing list? Somewhere else?
>
> > Feedback should go here, on the developers mailing list, to get as many
> > eyes on it as possible.  
>
> > > * This is further off, but once we have a ported Django, how do get
> > > the community (specifically pluggable apps) onboard? I'm assuming the
> > > docs are meant to do this but wondering if there's anything else we
> > > can be doing (like perhaps a Django-specific 2to3 (extension?) to
> > > cover common Django conventions).
>
> > Very good question, I'm uncertain as to how the "helpers" I mentioned
> > will look like in the end. Whether they will be part of Django (e.g.
> > a management command to run 2to3 on an app) or if we "only" provide the
> > necessary compatibility library (e.g. "six") so that 3rd party app
> > authors would still keep writing apps with Python 2 but would allow
> > their apps to be translated to Python 3 automatically. Documenting ways
> > of how to write a setup.py to do the conversion during install time
> > is *in* the scope of what we need to provide, IMO. Whether we need
> > Django-specific 2to3 fixers isn't clear at this time as the porting
> > has only just begun.
>
> > > * Do we have a target date? I know this is hard with a volunteer-only
> > > effort, but if we setup some sort of timeline, we'd at least have a
> > > metric & something to shoot/push for.
>
> > One assumption of the strategy I outlined was the fact that Django is
> > as close to 3.X as possible. Django 1.4 will require Python 2.5 or
> > higher, but I'm not sure how quick we can do the jump to 2.6, which
> > is recommended by the Python porting docs [1].
>
> > >   Finally, a philosophical question on approach: Should we really be
> > > doing 2to3 (leaving the Django codebase in Python 2.X for a long time)
> > > or would it be better to port Django over to Python 3 & use 3to2 for
> > > existing Python 2.X installs? I confess I don't know much about the
> > > current state of 3to2 (nor how most other Python libraries are
> > > handling the transition). But I do know Django will continue to grow
> > > over time & I worry that, at some point in the future we'll be making
> > > more even more work for someone else to do the 3-only work.
>
> > I personally haven't ported a 2.X library completely to 3.X yet, so I
> > can also only guess. But from what I've seen in the community I'm afraid
> > of a "clean cut" port because it has a high risk of leaving many projects
> > and apps behind. In that sense it seems more sensible to me to see the
> > port to Python 3 just as another step of our Python version deprecation
> > policy, which we at some point take with a complete conversion. Basically
> > a "burn bridges as soon as everyone is safe" approach :)
>
> > I don't dare to guess when that moment could be though, but it would 
> > probably
> > happen after a potential Python 2.7 only release of Django -- whenever that 
> > is.
>
> > Jannis
>
> > 1:http://docs.python.org/py3k/howto/pyporting.html#try-to-support-pytho...
>
> > > On Sep 14, 8:03 am, Jannis Leidel  wrote:
> > >> Hi all,
>
> > >> After last week's sprint I wanted to get you up-to-speed about the
> > >> current state of p

Re: Adding a signal to indicate when all models have finished loading. (#7538)

2009-04-11 Thread Daniel Lindsley

Malcolm,


   Given new information, I stand corrected. Given what I had seen in
experimenting and certain bits of Django, I was under the impression
that all models were deterministically loaded when Django started up.
Since that is not the case, it's very clear why this signal would have
implications if people started relying on it. I will simply have to
find a different means of handling what I'm looking for. I'm going to
close that ticket and detail why it isn't a good idea to the best of
my ability. Thanks.


Daniel




On Fri, Apr 10, 2009 at 6:48 PM, Malcolm Tredinnick
 wrote:
>
> On Fri, 2009-04-10 at 12:51 -0700, polarc...@gmail.com wrote:
>> I'd like to get get discussion started on the introduction of a "all
>> models loaded and ready for use" signal within AppCache. There's a
>> ticket already out there for this (http://code.djangoproject.com/
>> ticket/7538) but it looks like that forward progress on it stopped
>> because nothing was brought up here.
>>
>> The only alternative I've found it try to listen for the
>> "class_prepared" signal and repeated check to see if the AppCache is
>> loaded. This is very sub-optimal and doesn't work under certain
>> conditions.
>>
>> This would be extremely useful to me.
>
> Why? What is the use-case?
>
> Part of the reason why this isn't a slam dunk is because it's not really
> deterministic when the models will all be loaded and having people
> attempt to rely on that (and requiring us to never significantly change
> the behaviour) is going to lead to disappointment and tears.
>
> What are the big problems that require knowing this information, as
> opposed to testing AppCache.app_cache_ready() when you're executing code
> that might care about this? I'm concerned you want to kick off something
> when it happens and I can't imagine what that would be that can't be
> done in some other way. I also can't see that it's going be a reliable
> approach to a solution, particularly going forward. Locking ourselves
> into deep internal implementation behaviour such as the AppCache's
> current running approach doesn't strike me as robust engineering.
>
> Regards,
> Malcolm
>
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Possible future compatibility naming conflict with checksetup command

2013-06-19 Thread Daniel Lindsley
Russ,


Apologies for forcing the issue & for making a mess of the 
``django.core`` namespace. The goal was to help improve the user experience 
of sanity-checking their kit, but it seems I did more harm than good.

I'm fine with unifying the two (with option #2 probably being more 
likely so as not to disrupt the GSoC project any further). I am not opposed 
to either ``check`` or ``verify``. Both are perhaps a bit too generic for 
my taste ("what are you checking (or verifying)?"), but they are much more 
concise than using ``./manage.py 
checkmyapplicationsettingsandcodeforcompatibilitywiththisdjango``, so I'm 
in favor. The upshot is that since it's new, very little should be 
depending on what was committed, as I didn't get the chance to submit 
patches to other projects/apps to integrate it yet.

   In short, whatever the consensus, I'm happy to change it (or help (or 
get out of the way)).


Daniel



On Wednesday, June 19, 2013 1:34:54 AM UTC-7, Russell Keith-Magee wrote:
>
> Hi all (but especially Daniel)
>
> I've got a quick question about a recent commit and some naming 
> consequences that I think we need to sort out before we cut the 1.6 beta.
>
> tl;dr - I think we either need to rename the recently added checksetup 
> command, or do some light refactoring of it's internals so that we don't 
> make a rod for our own backs with work landing through the GSoC.
>
> In a recent commit [1], Daniel introduced a new management command - 
> checksetup - that can be used to identify upgrade-related issues in your 
> Django configuration -- in this case, the TEST_RUNNER problem, but he's set 
> up a framework that allows for other checks in the future.
>
> There's some interesting overlap here between this command, and the work 
> of my GSoC student, Christopher Medrela.
>
> Chris' GSoC project is about refactoring the `validate` management 
> command, turning the current monolithic validation mechanism into something 
> that is more easily extensible. Along the way, the aim is to introduce 
> validation warnings as well as errors, and provide ways to silence those 
> warnings if needed.
>
> As part of this work, we're addressing the fact that `validate` is a badly 
> overloaded word in Django at this point -- we have model validation, form 
> validation, and `validate` which performs system checks. This is 
> particularly problematic because we're looking at factoring 'validation' 
> system checks out into models and fields… which already have validate 
> methods to handle model validation.
>
> So - we've been looking for a new name. Two days before Daniel committed 
> his patch and new `checksetup` management command, Christopher and I were 
> having a discussion about a suitable alternative name for a new, refactored 
> `validate` command [2]. The end goal would be to deprecate validate in 
> favour of the new name.
>
> The mailing list discussion kicked around a few names, and we'd broadly 
> settled on `verify`. `check` (and some variants) was also considered. 
> Chris' preference was for verify. I'm on the fence. I don't love `verify` 
> as a name, but I'm willing to call it a bike shed on my part. 
>
> In line with this, fields, models, and anything else requiring on-startup 
> checks will be gaining a verify() entry point that users can use to add 
> their own verification checks.
>
> However, Daniel's commit has forced the issue, because we now have a 
> checksetup command whose implementation and use is a simple version of what 
> we're targeting for the GSoC. 
>  
> So, do we:
>
> 1) Keep checksetup as a name, and integrate the GSoC work under that new 
> name (including making the API entry points called check() or checksetup() 
> ). 
>
> 2) Rename `checksetup` to `verify`, setting the groundwork for the GSoC to 
> expand on the implementation that is already there
>
> 3) Keep both commands. 
>
> Opinions? I don't think (3) is the way to go - I'm not sure how we'd 
> explain the existence of both commands. I could go either way on (1) or (2) 
> though -- I'm not in love with either name, we just need to pick one. 
>
> Either way, this also means there's some scope for refactoring -- 
> django.core.compat_checks is a bit of a wart of a name against the rest of 
> the django.core namespace, IMHO. Chris is going to have a need for a bunch 
> of verify/check related utilities, not the least of which are classes for 
> defining Verify/Check Errors and Warnings, so we have an opportunity to 
> create django.core.verify.compatiblity (or django.core.check.compatibility) 
> to store the compatibility checks, and django.core.check.utils to store the 
> helper pieces.
>
> Yours,
> Russ Magee %-)
>
> [1] 
> https://github.com/django/django/commit/91f317c76d503b4bcae5a26c230425944dbf4ea8
> [2] 
> https://groups.google.com/d/msg/django-developers/fEf21dtpqDE/YqxM_OopbHcJ
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe fro

Re: App-loading reloaded - running code at startup

2013-12-30 Thread Daniel Lindsley
Aymeric,


As with the others, I'd like to echo my thanks! Having spent a lot of 
time with the old AppCache, I think these are solid improvements. Just a 
couple thoughts/questions from me. 

On Problem #1, IMO, I'd prefer to see something akin to an 
``AppConfig.setup`` (before anything is imported or built) & an 
``AppConfig.complete`` (after all classes have been setup for the app). I 
think this would make it clearer as to when the behavior is taking effect, 
as well as granting additional time between ``__init__``, which I don't 
think many people would like overriding, and the actual setup. 
Additionally, having a place to hook into that's guaranteed to be 
post-model setup would be a boon.

   On Problem #2, having been one of those people who's written many 
scripts like that, I'd prefer #2a. It's not consistent with existing 
behavior, but the existing behavior can be very unstable (for all the 
reasons others have mentioned). A stable order would be nice from an 
"easier-to-rationalize-about" perspective.


Daniel



On Monday, December 30, 2013 4:29:05 AM UTC-8, Aymeric Augustin wrote:
>
> Hello, 
>
> There’ve been lots of brainstorming on this topic. It is now tracked in 
> this ticket: https://code.djangoproject.com/ticket/21676. I’ve also sent 
> a pull request: https://github.com/django/django/pull/2128. 
>
> There’s a consensus on the general idea: 
> - AppConfigs are a good place for startup code, 
> - that code should be called when the app registry gets populated, 
> - the app registry should be populated as early as possible. 
>
> But the details are tricky. Specifically there are two problems for which 
> a design decision must be made. I need your feedback. 
>
>
> ### Problem 1 — Designing the API 
>
> I’d like to make the API as simple as possible, which means: 
>
> class AppConfig(object): 
> # … 
> def setup(self): 
> “””Override this method in subclasses to run code when Django 
> starts.””” 
>
> Logically, this would get called when the app cache is fully populated. 
> Running user code while the app cache isn’t ready will backfire. 
>
> But at this point it’s too late to register listeners for the 
> `class_prepared` signal, so we need an early entry point. I believe 
> AppConfig.__init__ does the job. 
>
> I originally suggested apps_ready() and models_ready() but I wasn’t happy 
> with this solution because it leaks internal implementation details and 
> inflicts a needlessy complicated API on the end users. 
>
> Relevant commits in the pull request: “Added AppConfig.setup() to run 
> setup code.” and “Updated advice on connecting signals at startup." 
>
>
> ### Problem 2 — Populating the app registry when not using wsgi.py or 
> manage.py 
>
> Everyone seems to agree with configuring the settings and populating the 
> app registry: 
> - before serving requests, in django.core.wsgi.get_wsgi_application() — 
> this will also prevent the server from starting with an invalid 
> configuration and crashing during the first request. 
> - before running a command, in 
> django.core.management.execute_from_command_line() — actually in 
> BaseCommand.execute() when can_import_settings = True. 
>
> Relevant commit in the pull request: “Populated the app registry earlier 
> at startup." 
>
> But these aren’t the only two entry points. It’s also possible to Django 
> code as follows: DJANGO_SETTINGS_MODULE=myproject.settings python script.py 
>
> If one makes an ORM query in script.py, the app registry gets populated 
> deep inside the ORM, as a side effect of get_models() when setting up 
> relations between models: 
>
> % DJANGO_SETTINGS_MODULE=myproject.settings python 
> >>> from gallery.models import Album 
> >>> Album.objects.all() 
> (Pdb) bt 
>   (1)() 
>   django/db/models/query.py(115)__repr__() 
> -> data = list(self[:REPR_OUTPUT_SIZE + 1]) 
>   django/db/models/query.py(140)__iter__() 
> -> self._fetch_all() 
>   django/db/models/query.py(962)_fetch_all() 
> -> self._result_cache = list(self.iterator()) 
>   django/db/models/query.py(264)iterator() 
> -> for row in compiler.results_iter(): 
>   django/db/models/sql/compiler.py(681)results_iter() 
> -> for rows in self.execute_sql(MULTI): 
>   django/db/models/sql/compiler.py(752)execute_sql() 
> -> sql, params = self.as_sql() 
>   django/db/models/sql/compiler.py(83)as_sql() 
> -> ordering, o_params, ordering_group_by = self.get_ordering() 
>   django/db/models/sql/compiler.py(409)get_ordering() 
> -> self.query.get_meta(), default_order=asc): 
>   django/db/models/sql/compiler.py(445)find_ordering_name() 
> -> field, targets, alias, joins, path, opts = self._setup_joins(pieces, 
> opts, alias) 
>   django/db/models/sql/compiler.py(477)_setup_joins() 
> -> pieces, opts, alias) 
>   django/db/models/sql/query.py(1375)setup_joins() 
> -> names, opts, allow_many) 
>   django/db/models/sql/query.py(1298)names_to_path() 
> -> field, model, direct, m2m = opts.get_field_by_name(name) 
>   django/db/models/opt