Re: Proposal/Discussion: url and template search using the directory structure of apps within django sites

[Jorge C . Leitão]

Hi Shai Berger and thanks for your quick response.

Regarding your criticisms,

On Thursday, May 30, 2013 7:54:40 AM UTC+2, Shai Berger wrote:
>
> Actually, Django doesn't. That is, an app is a Python package, and Python 
> packages can be nested, so you can have app 'b' within app 'a', and in 
> your 
> installed-apps, have both 'a' and 'a.b'. That will work fine. But as far 
> as 
> Django is concerned, they are just two separate apps. 
>
>
I completely understand your point, and I think this is a very important 
point. I agree when you say that "as far as Django is concerned, they are 
just two separate apps.". However, I believe that Django do allow nested 
apps. Allowing means not trowing an error when the developer does it. If 
django gives the developer the freedom to do it, it is because it allows it.

This motivates going back one step on my reasoning and put the following 
questions:

Why python allows nested apps?
What would be the reason for a developer to nest apps?
should django allow this?
why should django allow this?

I leave this questions open to discussion.


App names must be unique 
>

Yes, I agree with you, app names are unique. I'm not arguing that any of 
this would change.

Flat is better than nested.  


Thanks for pointing this out, and I believe this is a valid point. However, 
what I want to add is that I don't believe this renders my solution moot 
because Zen of python is used when two implementations give the same 
functionality. Otherwise python wouldn't allow nested apps...
What I'm proposing is a different functionality, which reduces to the 
current functionality if the developer doesn't do anything.

The problem you raise -- of name-collisions between elements in apps -- is 
> valid, but is a result of a long-standing decision; you will need very 
> convincing arguments to change that. 


Good that there is at least something valid! What I want to emphasize is 
that while programming this solution in my website, I ended up no longer 
having this problem, ever. I don't have any url name, any template with app 
prefix, any. What I want to try to understand with this post is: how does 
my solution can be used to improve django, and how can my solution be used 
in django without damaging anything else.

Hope this helps to clarify some of your concerns Shai Berger.

Cheers,
Jorge


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Proposal/Discussion: url and template search using the directory structure of apps within django sites

[Łukasz Rekucki]

On 30 May 2013 10:20, Jorge C. Leitão  wrote:
>
> On Thursday, May 30, 2013 7:54:40 AM UTC+2, Shai Berger wrote:
>>
>
> Why python allows nested apps?

Python allows nested *modules* because namespaces are good. Also
because without it you would have to wither write every module in a
single file or invent some weird conventions.

> What would be the reason for a developer to nest apps?

For Python modules, plenty. For Django apps? I don't really see a one
except for the sake of the namespace itself like django.contrib. But
in this case django.contrib is not an application itself, it's more of
a namespace package as defined in PEP 420.

> should django allow this?

Why it should not ?

> why should django allow this?

Because we're all consenting adults and you're free do to whatever you
want. Django documents what is possible and some major pitfalls. There
is no gain in *disallowing* this. Personally I wouldn't use this as
it's counter-intuitive and works in exact opposite to Python's import
mechanism which by default does an absolute search (since 3.x).

Regards,
Łukasz Rekucki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Proposal/Discussion: url and template search using the directory structure of apps within django sites

[Jorge C . Leitão]

Hi Russell Keith-Magee.

Thanks for your criticisms. Let's see if I can answer some of them. 
 

> I'd turn this question back onto you -- why *should* the search start 
> locally? What collisions are you seeing between apps that makes a global 
> search impractical?


Very good question! I think this question is related with your point

Well, no - I can see how you've managed to construct an example where local 
> search might be useful (although I still don't concede that it couldn't be 
> answered with global lookups). However, what I need to be convinced is to 
> see an actual use case where this is an issue -- not a synthetic example, 
> but a real world problem.

 
I.e. in my poor example I was not able to explain myself on the reason for 
using local search.

I'm sorry for the extension, but the reason is not so obvious.

Consider the case where in a an app you have an header, with the website 
logo and stuff, which you want to be consistent within your website, and 
you have smaller secondary header, (secondary_header.html) which you want 
to populate according to a specific app rendering the template. One 
frequent option is to add a block tag and put your 
(appname_secondary_header.html) with the same block tag, and call the 
appname_secondary_header.html in the specific's app view. This is the 
standard approach in Django and works great!

The problem is when you try to port this app to another website which 
already has that appname. Then you have to change your app's name (the 
directory name). However, this also means you have to change ALL the 
template names (or else the templates can also collide), all css, etc... 
you also have to change all views that call the template. Not to mention 
the disaster it it if the app is within a version control... This is not 
solved by adding a new folder on the templates directory with your app's 
name, because you would also have to change that+the all the views.

This problem is exactly the same in urls. You set a url naming convention 
to be consistent with your app, you port the app to another site 
with conflicting names and BANG.

Now, the reason why this happens is because template and url names have 
global scope. The solution people found out to solve this issue (C/C++, 
python, you name it) is to use local scope. For instance python uses 
namespaces, which uniquely define a name in relation to PATH or something.
In Django, namespaces are ill implemented (again, which is fine for most 
cases!) because they are not defined in respect to a path, they are just a 
name chosen by the developer, which is susceptible to collision.

In Python, packages are meant to be portable and the solution is: inside 
the package, you use a naming scheme assuming an arbitrary name of the 
package (directory's name). If at some moment you have to change the 
package name to avoid collision with other package, you just have to change 
the package's name and *not the content*. The names are arbitrary because 
the full name in relation to PATH will include the name of the package 
(e.g. import package.module).

So, if global scope doesn't work, what should them work? The answer (given 
by python developers) is local scope, and Django can use exactly the same 
idea as python uses in subclassing: if you want to extend an app, you put a 
sub-app within that app. If the app extends html, then the extension is 
like a subclassing: your sub-app adds some functionality, while keeping 
other constant. In my example of the secondary header, this means that if 
the sub-app has a secondary header that it would like to use, then it 
"overloads" the parent's template and calls its own secondary header. It is 
a design decision on whether the developer can act on the parent to allow 
this or not (in C++ you use "virtual" to say the method can be overloaded, 
in python every method can be overloaded).

So, my suggestion is *not just* about a funny way of searching. When I mean 
"local search", it is nothing more than a subclassing scheme, where the 
sub-apps are subclassing the parent app. Local means: "try to find the 
functionality in the class, if not found, go to the super" which python 
does internally.

And with this you answer "f*** you this is like designing a hole new django 
from scratch". My claim is that I only had to change like 10 lines of code 
in the actual Django code, produce 2 template tags and make a small 
modification on the "render" shortcut (with I named local_render) to 
include current_app. Even if this change is far from proofed to work on all 
cases (which I strongly doubt), I believe this idea is an important 
modification that increases the overall consistency of Django framework.

Russell Keith-Magee, I hope this (rather long) explanation provides 
in-depth justifications on why this modification is important and relevant 
to real problems.

Cheers,
Jorge



On Thursday, May 30, 2013 8:17:49 AM UTC+2, Russell Keith-Magee wrote:
>
>
>
> On Wed, May 29, 20

Re: Proposal/Discussion: url and template search using the directory structure of apps within django sites

[Yishai Beeri]

Hello Jorge,

On Thu, 30 May 2013 12:42:20 +0300, Jorge C. Leitão  
 wrote:



Hi Russell Keith-Magee.

Thanks for your criticisms. Let's see if I can answer some of them.
I'd turn this question back onto you -- why *should* the search start  
locally? What collisions are you seeing between apps that makes a  
global >>search impractical?


Very good question! I think this question is related with your point

Well, no - I can see how you've managed to construct an example where  
local search might be useful (although I still don't concede that it  
>>couldn't be answered with global lookups). However, what I need to be  
convinced is to see an actual use case where this is an issue -- not a  
>>synthetic example, but a real world problem.
I.e. in my poor example I was not able to explain myself on the reason  
for using local search.


I'm sorry for the extension, but the reason is not so obvious.

Consider the case where in a an app you have an header, with the website  
logo and stuff, which you want to be consistent within your website,  
>and you have smaller secondary header, (secondary_header.html) which  
you want to populate according to a specific app rendering the template.


The "a specific app rendering the template" is not phrase is not  
well-defined. Do you mean the app (in INSTALLED_APPS) that registered the  
url matched for the current request? The python module where the view  
function sits in? The python module whence the call to the template's  
render() was made? The python module which loaded the template? These can  
all be different things. Talking about "which app" makes some sense if  
you're limiting yourself to URL names - but keep in mind that templates  
can be (and often are) used outside of an HTTP request/response cycle -  
what's the "app" then?


One frequent option is to add a block tag and put your  
(appname_secondary_header.html) with the same block tag, and call the  
>appname_secondary_header.html in the specific's app view. This is the  
standard approach in Django and works great!


The problem is when you try to port this app to another website which  
already has that appname. Then you have to change your app's name (the  
>directory name). However, this also means you have to change ALL the  
template names (or else the templates can also collide), all css, etc...  
you >also have to change all views that call the template. Not to  
mention the disaster it it if the app is within a version control...  
This is not solved by >adding a new folder on the templates directory  
with your app's name, because you would also have to change that+the all  
the views.


This problem is exactly the same in urls. You set a url naming  
convention to be consistent with your app, you port the app to another  
site >with conflicting names and BANG.


Now, the reason why this happens is because template and url names have  
global scope. The solution people found out to solve this issue (C/C>++,  
python, you name it) is to use local scope. For instance python uses  
namespaces, which uniquely define a name in relation to PATH or  
>something.
In Django, namespaces are ill implemented (again, which is fine for most  
cases!) because they are not defined in respect to a path, they are just  
a >name chosen by the developer, which is susceptible to collision.


In Python, packages are meant to be portable and the solution is: inside  
the package, you use a naming scheme assuming an arbitrary name of the  
>package (directory's name). If at some moment you have to change the  
package name to avoid collision with other package, you just have to  
>change the package's name and *not the content*. The names are  
arbitrary because the full name in relation to PATH will include the  
name of the >package (e.g. import package.module).


So, if global scope doesn't work, what should them work? The answer  
(given by python developers) is local scope, and Django can use exactly  
>the same idea as python uses in subclassing: if you want to extend an  
app, you put a sub-app within that app.
How would you suggest extending a 3rd party app, then? I assume you're not  
suggesting to place my extension inside the source tree of the 3rd party  
app.
Python's subclassing is completely disconnected from physical paths - and  
you usually have to explicitly name the class you're extending - you don't  
inherit it simply by virtue of being in a subfolder. You also give the  
new, extended class a brand new name - and at least some code needs to  
know that name in order to instantiate the right thing.


If the app extends html, then the extension is like a subclassing: your  
sub-app adds some functionality, while keeping other constant. In my  
>example of the secondary header, this means that if the sub-app has a  
secondary header that it would like to use, then it "overloads" the  
parent's >template and calls its own secondary header. It is a design  
decision on whether the developer 

Re: Proposal/Discussion: url and template search using the directory structure of apps within django sites

[Jorge C . Leitão]

>
>  I must say that reading this into Russel (and other)'s replies is very 
> far fetched (language as well as content), uncalled for, and seems to 
> expose an assumption that the people here prefer the easy path of swatting 
> suggestions with a quick "nay".  I 'd say the opposite is much closer to 
> the truth.


I'm sorry, this was a joke; not something to be taken seriously. On the 
contrary, I assume people here are actually very well intended, and that's 
why I feel confident enough to put the joke here. Anyway, I'm not here to 
offend anyone or anything, so, I'm sorry for the sentence, I would retract 
it if I could.

I generally accept your criticisms and agree with your point of view that 
it is not so strait-forward in django's spirit. Let's them try to reach 
some specific points that would be worth implementing, even without the 
whole idea I'm proposing.

It seems you could achieve what you're after (at least for templates) with 
> a custom template loader. A variation on the app directories loader [0] can 
> impose a different search order than the default one that simply follows 
> the order in INSTALLED_APPS. Am I wrong?


YES!! Definitely this. However, the function get_template does not accept a 
directory to search. It is not possible to pass the directory of the search 
to the get_template: django.templates.loader.get_template only accepts one 
argument, i.e. you cannot tell which directory you want the template to be 
searched from the call. The two of the 10 lines of code I was referring to 
was change in that function:

from
get_template(template_name)
...
to
get_template(template_name*, dirs=None*)
...
template, origin = find_template(template_name*, dirs*)

this is compatible with the find_template function, which also has two 
arguments:

find_template(name, dirs=None)

For consistency, this should be implemented on the other two functions:

render_to_string
select_template

which should also receive dirs as arguments, with default=None and 
respective passing in the their calls of get_template.

This allows the developer to actually *choose* which template it is 
refering to when it uses render("secondary_header.html"), and works with 
the 

app_directories.Loader because it already has a optional argument 
"template_dirs".

So, yes, this is actually one of my proposals... change these 4 or 5 lines.

Cheers,
Jorge



On Thursday, May 30, 2013 12:50:34 PM UTC+2, Yishai Beeri wrote:
>
> Hello Jorge,
>
> On Thu, 30 May 2013 12:42:20 +0300, Jorge C. Leitão 
> > 
> wrote:
>
> Hi Russell Keith-Magee.
>
> Thanks for your criticisms. Let's see if I can answer some of them. 
>  
>
>> I'd turn this question back onto you -- why *should* the search start 
>> locally? What collisions are you seeing between apps that makes a global 
>> search impractical?
>
>
> Very good question! I think this question is related with your point
>
> Well, no - I can see how you've managed to construct an example where 
>> local search might be useful (although I still don't concede that it 
>> couldn't be answered with global lookups). However, what I need to be 
>> convinced is to see an actual use case where this is an issue -- not a 
>> synthetic example, but a real world problem.
>
>  
> I.e. in my poor example I was not able to explain myself on the reason for 
> using local search.
>
> I'm sorry for the extension, but the reason is not so obvious.
>
> Consider the case where in a an app you have an header, with the website 
> logo and stuff, which you want to be consistent within your website, and 
> you have smaller secondary header, (secondary_header.html) which you want 
> to populate according to a specific app rendering the template.
>
>
> The "a specific app rendering the template" is not phrase is not 
> well-defined. Do you mean the app (in INSTALLED_APPS) that registered the 
> url matched for the current request? The python module where the view 
> function sits in? The python module whence the call to the template's 
> render() was made? The python module which loaded the template? These can 
> all be different things. Talking about "which app" makes some sense if 
> you're limiting yourself to URL names - but keep in mind that templates can 
> be (and often are) used outside of an HTTP request/response cycle - what's 
> the "app" then?
>
> One frequent option is to add a block tag and put your 
> (appname_secondary_header.html) with the same block tag, and call the 
> appname_secondary_header.html in the specific's app view. This is the 
> standard approach in Django and works great!
>
> The problem is when you try to port this app to another website which 
> already has that appname. Then you have to change your app's name (the 
> directory name). However, this also means you have to change ALL the 
> template names (or else the templates can also collide), all css, etc... 
> you also have to change all views that call the template. Not to mention 
> the disaster it it if the app is 

Re: Perception of attitude in tickets

[Simon Litchfield]

Sorry I'm late back to the party boys and girls. 

Trivial as it may be, it's really just communication that's the only issue 
here, and I'm glad this has prompted a review. We all mean well and we're 
eager to help. The solutions Cal, Russ, Luke and co have discussed sound 
great. 

BTW- there are two Simon's here. I'm "simon29" from the original ticket, 
not "Simon" from this thread.


On Saturday, 11 May 2013 03:00:04 UTC+10, Simon wrote:
>
> Hi,
>
> When I started using Python a couple of months ago, a quick Google for 
> frameworks turned up a lot of results for Django so I decided to give it a 
> spin.
>
> I'd like to give some feedback on my experience to date. There are a lot 
> of features I really love, some that are a little quirky and some that are 
> downright inflexible. None of this will be news - it's the same for every 
> framework. That said, I started to have doubts when I was attempting to 
> find solutions/workarounds to the problems I encountered.
>
> Today was the 5th or 6th time that I've ended up at the ticket system and 
> seen people saying "This would really help me" and a core developer saying 
> "I don't see the need" (rather arbitrarily IMHO) and closing as wontfix. 
> This is invariably followed by people asking for reconsideration which in 
> turn gets a "use the mailing list" with varying degrees of rudeness.
>
> While I'm sure it's not the real reason, sending people to the mailing 
> lists feels like a way of brushing disagreement under the carpet. There's 
> no obvious way to follow on from the discussion in the ticket to the 
> relevant discussions in the mailing list (if any) and visitors coming by 
> years later now have to go and hunt through an archive to find out if 
> there's any chance of a change.
>
> I feel that the general attitude expressed in some of the tickets is poor. 
> The one which prompted this post is 
> https://code.djangoproject.com/ticket/901. I think comment 
> 20 is 
> a good demonstration of my point. A couple of users were getting frustrated 
> at the lack of discussion/progress which resulted in a fairly sanctimonious 
> rant. 
>
> Some other tickets I've ended up on have proposed patches and seem to have 
> sat in "Design decision" for years, which again gives the impression that 
> the core team didn't like it so just sort of ignored it until it went away.
>
> So, to be honest, the impression I'm getting WRT new features in Django is 
> "Don't bother proposing it 'cos it's not going to happen".
>
> There are 
> StackOverflow
>  
> questions
>  
> (another)
>  on 
> the topic and numerous other sources pointing at this particular ticket 
> wondering why it hasn't been implemented. The only reason I can see is that 
> "jacob" wasn't convinced by the (first) use case.
>
> Now, I admit that I'm probably seeing the worst side of the problem, there 
> are probably hundreds of other features which did get in (which is why 
> there's documentation not tickets for me to find) but that doesn't make the 
> situation I'm seeing better, just smaller.
>
> Perhaps the fact that people keep posting on closed tickets shows that the 
> current flow to the mailing lists isn't a good one? Maybe either add a 
> "Start a topic about this ticket" link or maybe even just allow discussion 
> to continue on the ticket as many others do?
>
> I'm unlikely to use Django moving forward. There are a number of reasons 
> and I'd be lying if I said this was the biggest but it was a factor in my 
> decision.
>
> Anyway, I wanted to take a few minutes and share the impressions I've had 
> to date - perhaps this way, others will have a better experience in future.
>
> Thanks for reading
>
> Simon
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Migrations, commands and syncdb

[Andrew Godwin]

Hi everyone,

I'm starting to plan out the commands for the new migrations stuff in
Django, and I've hit something of an impasse trying to decide which option
to go for.

Short background: South modified syncdb to just sync non-migrated apps, and
you had to go and run migrate separately to get migrations working.

Note that this proposal DOES NOT cover the commands for creating and
squashing migrations. That will come later, but will probably be
"./manage.py createmigration" and "./manage.py squashmigrations"

The proposals are:

 1. Change syncdb so that it both does the old behaviour (adds models for
unmigrated apps), and additionally runs any outstanding migrations. There
would be a separate "migrate" command for more complex operations, such as
reversing them or faking application, which is a little odd.

 2. Leave syncdb as it is, like South does, and have everything happen
through a "migrate" command. Leads to weird interactions where each command
knows about the other, and must be run in a certain order, but which isn't
immediately obvious.

 3. Do everything through a single command - migrations, non-migrated
syncing, reversal of migrations, etc. I would call this command "migrate",
and start a deprecation cycle on "syncdb" (which  would turn into an alias
to "migrate"). Calling "./manage.py migrate" would first sync unmigrated
apps, and then run migrations, but would have options so a user could
migrate (or sync!) specific apps/target migrations.

I prefer option 3, but getting rid of syncdb might be controversial, so I
want to ask for people's opinions. syncdb would continue to exist for at
least 3 versions if not forever; it would just be an alias to run "migrate"
in its default configuration, and would do exactly what you would expect
(whereas with South now, and with option 2, syncdb doesn't do enough).

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations, commands and syncdb

[Donald Stufft]

On May 30, 2013, at 2:03 PM, Andrew Godwin  wrote:

> Hi everyone,
> 
> I'm starting to plan out the commands for the new migrations stuff in Django, 
> and I've hit something of an impasse trying to decide which option to go for.
> 
> Short background: South modified syncdb to just sync non-migrated apps, and 
> you had to go and run migrate separately to get migrations working.
> 
> Note that this proposal DOES NOT cover the commands for creating and 
> squashing migrations. That will come later, but will probably be "./manage.py 
> createmigration" and "./manage.py squashmigrations"
> 
> The proposals are:
> 
>  1. Change syncdb so that it both does the old behaviour (adds models for 
> unmigrated apps), and additionally runs any outstanding migrations. There 
> would be a separate "migrate" command for more complex operations, such as 
> reversing them or faking application, which is a little odd.
> 
>  2. Leave syncdb as it is, like South does, and have everything happen 
> through a "migrate" command. Leads to weird interactions where each command 
> knows about the other, and must be run in a certain order, but which isn't 
> immediately obvious.
> 
>  3. Do everything through a single command - migrations, non-migrated 
> syncing, reversal of migrations, etc. I would call this command "migrate", 
> and start a deprecation cycle on "syncdb" (which  would turn into an alias to 
> "migrate"). Calling "./manage.py migrate" would first sync unmigrated apps, 
> and then run migrations, but would have options so a user could migrate (or 
> sync!) specific apps/target migrations.
> 
> I prefer option 3, but getting rid of syncdb might be controversial, so I 
> want to ask for people's opinions. syncdb would continue to exist for at 
> least 3 versions if not forever; it would just be an alias to run "migrate" 
> in its default configuration, and would do exactly what you would expect 
> (whereas with South now, and with option 2, syncdb doesn't do enough).
> 
> Andrew
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>  
>  

I vote #3.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Migrations, commands and syncdb

[Alex Gaynor]

I'm broadly +1 on deprecating syncbd, it's possibly the most inaccurately
named thing in all of Django (hint: it doesn't sync anything).

Alex


On Thu, May 30, 2013 at 11:03 AM, Andrew Godwin  wrote:

> Hi everyone,
>
> I'm starting to plan out the commands for the new migrations stuff in
> Django, and I've hit something of an impasse trying to decide which option
> to go for.
>
> Short background: South modified syncdb to just sync non-migrated apps,
> and you had to go and run migrate separately to get migrations working.
>
> Note that this proposal DOES NOT cover the commands for creating and
> squashing migrations. That will come later, but will probably be
> "./manage.py createmigration" and "./manage.py squashmigrations"
>
> The proposals are:
>
>  1. Change syncdb so that it both does the old behaviour (adds models for
> unmigrated apps), and additionally runs any outstanding migrations. There
> would be a separate "migrate" command for more complex operations, such as
> reversing them or faking application, which is a little odd.
>
>  2. Leave syncdb as it is, like South does, and have everything happen
> through a "migrate" command. Leads to weird interactions where each command
> knows about the other, and must be run in a certain order, but which isn't
> immediately obvious.
>
>  3. Do everything through a single command - migrations, non-migrated
> syncing, reversal of migrations, etc. I would call this command "migrate",
> and start a deprecation cycle on "syncdb" (which  would turn into an alias
> to "migrate"). Calling "./manage.py migrate" would first sync unmigrated
> apps, and then run migrations, but would have options so a user could
> migrate (or sync!) specific apps/target migrations.
>
> I prefer option 3, but getting rid of syncdb might be controversial, so I
> want to ask for people's opinions. syncdb would continue to exist for at
> least 3 versions if not forever; it would just be an alias to run "migrate"
> in its default configuration, and would do exactly what you would expect
> (whereas with South now, and with option 2, syncdb doesn't do enough).
>
> Andrew
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations, commands and syncdb

[Carl Meyer]

On 05/30/2013 12:03 PM, Andrew Godwin wrote:
> The proposals are:
> 
>  1. Change syncdb so that it both does the old behaviour (adds models
> for unmigrated apps), and additionally runs any outstanding migrations.
> There would be a separate "migrate" command for more complex operations,
> such as reversing them or faking application, which is a little odd.
> 
>  2. Leave syncdb as it is, like South does, and have everything happen
> through a "migrate" command. Leads to weird interactions where each
> command knows about the other, and must be run in a certain order, but
> which isn't immediately obvious.
> 
>  3. Do everything through a single command - migrations, non-migrated
> syncing, reversal of migrations, etc. I would call this command
> "migrate", and start a deprecation cycle on "syncdb" (which  would turn
> into an alias to "migrate"). Calling "./manage.py migrate" would first
> sync unmigrated apps, and then run migrations, but would have options so
> a user could migrate (or sync!) specific apps/target migrations.
> 
> I prefer option 3, but getting rid of syncdb might be controversial, so
> I want to ask for people's opinions. syncdb would continue to exist for
> at least 3 versions if not forever; it would just be an alias to run
> "migrate" in its default configuration, and would do exactly what you
> would expect (whereas with South now, and with option 2, syncdb doesn't
> do enough).

I much prefer option 3; I think a deprecation path for syncdb is fine.
It's always been mis-named anyway, since it doesn't really sync.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations, commands and syncdb

[Jannis Leidel]

On 30.05.2013, at 20:03, Andrew Godwin  wrote:

> Hi everyone,
> 
> I'm starting to plan out the commands for the new migrations stuff in Django, 
> and I've hit something of an impasse trying to decide which option to go for.
> 
> Short background: South modified syncdb to just sync non-migrated apps, and 
> you had to go and run migrate separately to get migrations working.
> 
> Note that this proposal DOES NOT cover the commands for creating and 
> squashing migrations. That will come later, but will probably be "./manage.py 
> createmigration" and "./manage.py squashmigrations"
> 
> The proposals are:
> 
>  1. Change syncdb so that it both does the old behaviour (adds models for 
> unmigrated apps), and additionally runs any outstanding migrations. There 
> would be a separate "migrate" command for more complex operations, such as 
> reversing them or faking application, which is a little odd.
> 
>  2. Leave syncdb as it is, like South does, and have everything happen 
> through a "migrate" command. Leads to weird interactions where each command 
> knows about the other, and must be run in a certain order, but which isn't 
> immediately obvious.
> 
>  3. Do everything through a single command - migrations, non-migrated 
> syncing, reversal of migrations, etc. I would call this command "migrate", 
> and start a deprecation cycle on "syncdb" (which  would turn into an alias to 
> "migrate"). Calling "./manage.py migrate" would first sync unmigrated apps, 
> and then run migrations, but would have options so a user could migrate (or 
> sync!) specific apps/target migrations.
> 
> I prefer option 3, but getting rid of syncdb might be controversial, so I 
> want to ask for people's opinions. syncdb would continue to exist for at 
> least 3 versions if not forever; it would just be an alias to run "migrate" 
> in its default configuration, and would do exactly what you would expect 
> (whereas with South now, and with option 2, syncdb doesn't do enough).

+1 on #3

--
Jannis

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations, commands and syncdb

[Shai Berger]

Hi all,

On Thursday 30 May 2013, Andrew Godwin wrote:
> 
> The proposals are:
> 
>  1. Change syncdb so that it both does the old behaviour (adds models for
> unmigrated apps), and additionally runs any outstanding migrations. There
> would be a separate "migrate" command for more complex operations, such as
> reversing them or faking application, which is a little odd.
> 
>  2. Leave syncdb as it is, like South does, and have everything happen
> through a "migrate" command. Leads to weird interactions where each command
> knows about the other, and must be run in a certain order, but which isn't
> immediately obvious.
> 
>  3. Do everything through a single command - migrations, non-migrated
> syncing, reversal of migrations, etc. I would call this command "migrate",
> and start a deprecation cycle on "syncdb" (which  would turn into an alias
> to "migrate"). Calling "./manage.py migrate" would first sync unmigrated
> apps, and then run migrations, but would have options so a user could
> migrate (or sync!) specific apps/target migrations.
> 
Naming aside, I see little value in an operation that runs the current syncdb 
without running pending migrations. Thus, -1 on option 2.

Between 1 and 3, I tend towards 3, assuming that the default operation (that 
is, with no flags or arguments) is the equivalent of South's current 
"syncdb --migrate" -- that is, the syncdb of option 1. I don't like the 
current situation, where syncdb without --migrate is not useful and migrate 
requires an explicit choice of apps.

Shai

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations, commands and syncdb

[Anssi Kääriäinen]

On 30 touko, 21:03, Andrew Godwin  wrote:
> I prefer option 3, but getting rid of syncdb might be controversial, so I
> want to ask for people's opinions. syncdb would continue to exist for at
> least 3 versions if not forever; it would just be an alias to run "migrate"
> in its default configuration, and would do exactly what you would expect
> (whereas with South now, and with option 2, syncdb doesn't do enough).

+1 to #3.

I haven't used South as much as I should have (instead I have painful
manual scripts to do migrations). The biggest pain point about
database schemas for me is easily test database setup. That is, sync
from scratch. I do the following currently:
  1. load schema + a little bit of data from production through
pg_dump + pg_restore. The data is mostly metadata, in particular
already applied migrations in production.
  2. run those migrations that haven't been applied already.

Will I be able to do the above somehow?  Will #3 do a similar thing
for test database setup, except that you run all migrations against
empty database? The first migration would be the initial "syncdb", and
then you run rest of the migrations in chain finally arriving to the
wanted database state. Or would it be "sync" all models, then run
migrations against that database?

Sorry if these are stupid questions... I really do not know South as
well as I should, so I need a little tutoring...

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations, commands and syncdb

[Aymeric Augustin]

+1 on option 3.

In hindsight syncdb isn't a good name. It won't be missed. Please deprecate it.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations, commands and syncdb

[Andrew Godwin]

I haven't used South as much as I should have (instead I have painful
> manual scripts to do migrations). The biggest pain point about
> database schemas for me is easily test database setup. That is, sync
> from scratch. I do the following currently:
>   1. load schema + a little bit of data from production through
> pg_dump + pg_restore. The data is mostly metadata, in particular
> already applied migrations in production.
>   2. run those migrations that haven't been applied already.
>
> Will I be able to do the above somehow?  Will #3 do a similar thing
> for test database setup, except that you run all migrations against
> empty database? The first migration would be the initial "syncdb", and
> then you run rest of the migrations in chain finally arriving to the
> wanted database state. Or would it be "sync" all models, then run
> migrations against that database?
>
> Sorry if these are stupid questions... I really do not know South as
> well as I should, so I need a little tutoring...



The way South works, and the way this will work (for new installs as well
as tests) is that if migrations are present for an app, it will always
create new databases by running through them from the very first migration
to the latest one.

The proposed "squash" feature helps stop any potential long setup times
(from hundreds of migrations) by allowing you to replace 100 old ones with
a few new ones, optimised to have less operations.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Migrations, commands and syncdb

[Russell Keith-Magee]

On Fri, May 31, 2013 at 2:03 AM, Andrew Godwin  wrote:

> Hi everyone,
>
> I'm starting to plan out the commands for the new migrations stuff in
> Django, and I've hit something of an impasse trying to decide which option
> to go for.
>
> Short background: South modified syncdb to just sync non-migrated apps,
> and you had to go and run migrate separately to get migrations working.
>
> Note that this proposal DOES NOT cover the commands for creating and
> squashing migrations. That will come later, but will probably be
> "./manage.py createmigration" and "./manage.py squashmigrations"
>
> The proposals are:
>
>  1. Change syncdb so that it both does the old behaviour (adds models for
> unmigrated apps), and additionally runs any outstanding migrations. There
> would be a separate "migrate" command for more complex operations, such as
> reversing them or faking application, which is a little odd.
>
>  2. Leave syncdb as it is, like South does, and have everything happen
> through a "migrate" command. Leads to weird interactions where each command
> knows about the other, and must be run in a certain order, but which isn't
> immediately obvious.
>
>  3. Do everything through a single command - migrations, non-migrated
> syncing, reversal of migrations, etc. I would call this command "migrate",
> and start a deprecation cycle on "syncdb" (which  would turn into an alias
> to "migrate"). Calling "./manage.py migrate" would first sync unmigrated
> apps, and then run migrations, but would have options so a user could
> migrate (or sync!) specific apps/target migrations.
>
> I prefer option 3, but getting rid of syncdb might be controversial, so I
> want to ask for people's opinions. syncdb would continue to exist for at
> least 3 versions if not forever; it would just be an alias to run "migrate"
> in its default configuration, and would do exactly what you would expect
> (whereas with South now, and with option 2, syncdb doesn't do enough).
>

+1 to (3) as well.

Russ %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Digest for django-developers@googlegroups.com - 17 Messages in 3 Topics

[Mantas]

Quoting django-developers@googlegroups.com (2013-05-31 07:39:24)
> From: Andrew Godwin 
> Date: May 30 07:03PM +0100
> Url: http://groups.google.com/group/django-developers/msg/137ab14f361fd052
>
> 2. Leave syncdb as it is, like South does, and have everything happen
> through a "migrate" command. Leads to weird interactions where each command
> knows about the other, and must be run in a certain order, but which isn't
> immediately obvious.

Order in which syncdb and migrate should be run also depends on
dependencies between apps.

For example, if you have two apps, A and B:

# A/models.py

class A(models.Model):
name = models.CharField(max_length=128)


# B/models.py

class B(models.Model):
name = models.CharField(max_length=128)
aref = models.ForeignKey(A)

Then, first you must run syncdb/migrations for A app, and *only* then
for app B.

So to fully fix migrations, Django should be aware about dependencies
between apps, which is very important not only for migrations.


--
 Mantas aka sirex
  __o   /\
_ \<,_   -- launchpad.net/~sirex --  /\/  \
___(_)/_(_)_/_/\
^

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.