Ipython 0.11 + Custom Profiles
I recently spent a lot of time trying to get my models automatically imported into IPython 0.11 from the shell. command. I finally got it working by making a new ishell command from the existing shell.py (http://greencm.blogspot.com/2011/10/django-ipython-011-models.html). The default django shell.py never calls the IPython Profile hooks. I think the one patch (https://code.djangoproject.com/attachment/ticket/14892/IPython.patch) ran shell() rather than embed. import IPython IPython.embed() That causes the default profiles to be loaded. Given that the patch was 11 months before 0.11 final was out, it's possible the API drifted along the way. -- Chris Green -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
On Mon, Oct 10, 2011 at 12:20 PM, tWoolie wrote: > I'd like to propose a project to split out the table/list > functionality and the filtering functionality in the django admin into > a seperate contrib app. This is really core functionality which is > lacking from django (in terms of being locked to the admin app). > I believe this would be beneficial because currently there are a lot > of people who love using the admin interface but find building the > same functionality into their own applications daunting at best. This > leads a lot of people to try to abuse the admin app by allowing > regular users into there. > > I propose moving a lot of the django-tables2 functionality into a > contrib app and augumenting it with filterspec. Then the admin can use > this app, or any custom subclass of Table/Filter. Currently, django- > tables2 is fantastic but has no support for filterspecs which would be > super super helpful, and would benefit from the level of [url] > abstraction that d-t2 provides. > > I'm posting to this group to gather feedback and ask if anyone would > actually like to see this happen. I believe it's something i'd love to > contribute but only if the django devs would be interested in > incorporating it. Broadly speaking, I agree with the motivation; I'm not 100% sure I agree about the implementation plan. Getting "admin-like" functionality into the mainstream was one of the prime motivations behind the introduction of class-based generic views. Django's admin contains a number of neat features (like fieldsets, views with inline formsets, filtering and searching) that can be very easily implemented as a set of class-based mixins or extensions. I've got code in private repositories that implement some of these features; I'd like to find the time to clean them up as generic utilities. To my mind, the end goal should be to replace most of the internals of Django's admin with relatively light customizations of generic form/list functionality. Admin's filters should be a specialization of a generic capability, not something specific to the admin. I haven't seen django-tables2 before, so I can't comment on whether the implementation is suitable for core as-is. However, my immediate reaction is that if we add this, it won't be as a contrib app. This is a piece of core functionality of generic views, not an "optional library implementing a defacto implementation of a common pattern" (this is the yardstick against we measure contrib proposals). I'd also say that django-tables2, by itself, appears to be only one small part of a much larger solution. I am aware of several efforts to build 'admin-like' view and form functionality on top of the CBV framework introduced in 1.3. I'd like to see a lot more consolidation in these projects before anything is proposed for trunk. The approach taken by django-messages should be the model here. When a new messages framework was proposed for Django in 1.2, an effort was made to find out all the features that were desirable, and then pull together all those features into a third party library that borrowed code from all the available implementations. The same approach should be taken here -- find out all the features that we want to have in generic views (e.g., formsets, fieldsets, filtering, ordering, table rendering), build an amalgam library that takes the best parts from all the available 3rd party libraries, and develop that into a single patch that can be applied to trunk. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
> Getting "admin-like" functionality into the mainstream was one of the > prime motivations behind the introduction of class-based generic > views. Django's admin contains a number of neat features (like > fieldsets, views with inline formsets, filtering and searching) that > can be very easily implemented as a set of class-based mixins or > extensions. I've got code in private repositories that implement some > of these features; I'd like to find the time to clean them up as > generic utilities. I think we all have code like this sitting around. It would be great to standardise this code into a common package/platform > To my mind, the end goal should be to replace most of the internals of > Django's admin with relatively light customizations of generic > form/list functionality. Admin's filters should be a specialization of > a generic capability, not something specific to the admin. Sorry if i wasn't clear but this is exactly what i want to do: namely take the magic out of admin, and turn the admin into a thin wrapper over standardised tables/filters > I haven't seen django-tables2 before, so I can't comment on whether > the implementation is suitable for core as-is. Think of the django forms declarative syntax, but for tables. It's really quite neat but in need of some refactoring that can't be done in dt2 as it would seriously break backwards compatibility. >However, my immediate > reaction is that if we add this, it won't be as a contrib app. This is > a piece of core functionality of generic views, not an "optional > library implementing a defacto implementation of a common pattern" > (this is the yardstick against we measure contrib proposals). The problem I see here is that while tables are awesome, they are in no way a view. and limiting a view to having a single table, or filterset does not sit right with me. Class based views are great, but are not a catch-all solution. > I'd also say that django-tables2, by itself, appears to be only one > small part of a much larger solution. I am aware of several efforts to > build 'admin-like' view and form functionality on top of the CBV > framework introduced in 1.3. I'd like to see a lot more consolidation > in these projects before anything is proposed for trunk. The inclusion of views that have tables "baked in" is a great idea, but really the wrapper would be so thin as to be trivial. The point of this effort is to consolidate declarative tables/filters, not build generic views. > The approach taken by django-messages should be the model here. When a > new messages framework was proposed for Django in 1.2, an effort was > made to find out all the features that were desirable, and then pull > together all those features into a third party library that borrowed > code from all the available implementations. The same approach should > be taken here -- find out all the features that we want to have in > generic views (e.g., formsets, fieldsets, filtering, ordering, table > rendering), build an amalgam library that takes the best parts from > all the available 3rd party libraries, and develop that into a single > patch that can be applied to trunk. I'm more than happy to build this as a third party app (better_admin?), but part of the appeal for me is to rebuild admin without the magic, not introduce a competing admin that nobody knows about. As far as CBV is concerned, this is a non-starter. these features extend beyond generic views. perhaps as a convenience after the majority of the work is done. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: ORM refactoring
Hi Anssi, > I would like to work on refactoring the ORM. This would be slow work, > one piece at a time, not a total rewrite. I think there is much > potential in the ORM. It needs to be cleaned first, and then we can > get hacking new and awesome features. I know there are other people > interested in this, too. > > For example I have been thinking for some time about something like > F() objects, but which could contain arbitrary SQL and references to > fields in the query. relabel_aliases and cloning would be handled for > you. These could be used in .order_by(), values_list, annotate() and > so on. SQLSnippet could be its name. This would allow us to > deprecate .extra, and would allow for some really fancy extensions. > > Using these you could implement an external SQL snippets library, the > PostgreSQL window functions in one library for example. Or custom > aggregate ModelAggregate (fetch related models using array_agg, > something like prefetch_related, but runs in single SQL query). I > really do believe this is possible. > > But we need to get the work started on refactoring the ORM first. I do > currently have some extra time, and I am willing to work on the ORM. > But we need some people reviewing the patches and most of all we need > core developers who have enough time to review and commit the ready > patches. Unfortunately the ORM-knowing core developers seem to be > really busy. One of the problems is that it can be very hard to review refactorings. For example, I recently checked in rev 16929 [1] from ivan_virabyan's patch, and reviewing it was hard, despite the fact that it was a very good quality patch. The difficulty is that there appear to be a large number of changes, when in reality the code has just moved around. But confirming that it has indeed stayed the same is very time consuming if you do it properly - in fact it seems almost faster to do the same changes yourself, and verify you get the same result. This is only thinking out loud, but I'm wondering if that approach might not be too bad. If instead of a large patch, we have a series of smaller steps (in a github branch or whatever), perhaps that would be easier to review? I'm not saying rework any already created patches to match this, BTW. We do have pretty good test coverage, which should be a good help, but one of the problems here is the mutability thing - it is very difficult to write a test that shows that shows that a new QuerySet is not sharing data structures with the old in a way that is incorrect and could lead to the previous QuerySet being modified by the new one (or subsequent ones). > I really do not know how to move forward. I guess I am asking all you > what do you think should be done with the ORM. We could take a > completely opposite direction and make it a wrapper around SQLAlchemy > (I saw this mentioned somewhere some time ago). Or decide that the ORM > is just fine now, or that there are higher importance items. Before > continuing my hacking, it would be nice to know if there is support > for ORM refactoring. It was me who suggested the idea of basing the ORM on SQLAlchemy in future, on my blog [2]. It would be a very large change, especially things related to the unit-of-work stuff, and I have no idea whether it is even feasible really. My concern is that the ORM currently leaves you high and dry when you come to the end of the queries it can generate. Some people say the answer is raw SQL at that point, but if you have built up your query dynamically in any way, or perhaps in a completely Model agnostic way (as I was doing in django-easyfilters), you are completely stuck, because you are left with a dynamically created query that you really cannot usefully manipulate in any way, but which you must manipulate in order to do the queries you need. I think the best you could do is generate the SQL from the query as a string (which is actually hard to do currently due to the way that SQL generation interacts with the different backends), parse it again with python-sqlparse, manipulate and pass to QuerySet.raw(). And then somehow cope with the backend specific stuff. Ugh. SQLA, I believe, would be very different, because 'core' provides a complete wrapper around SQL, and the ORM allows you to drop down to core to do advanced things. And it has a much nicer way of dealing with the different SQL dialects. But as I said, I don't know if any kind of rewrite to use SQLA is feasible, and I don't know whether anyone else in the community or in the core team has similar leanings - I don't know if any of the core devs happened to read that blog post. I do think that these refactorings you are doing are valuable. However, if you work on this, I think you need to be prepared for the possibility that the work may be a dead end. It could be an experiment that shows that a different approach will be needed. If it does that, it will still have been useful work, but you will need a certain kind of attitude to
Re: Splitting out admin functionality
On 10 October 2011 13:13, Russell Keith-Magee wrote: > > The approach taken by django-messages should be the model here. When a > new messages framework was proposed for Django in 1.2, an effort was > made to find out all the features that were desirable, and then pull > together all those features into a third party library that borrowed > code from all the available implementations. The same approach should > be taken here -- find out all the features that we want to have in > generic views (e.g., formsets, fieldsets, filtering, ordering, table > rendering), build an amalgam library that takes the best parts from > all the available 3rd party libraries, and develop that into a single > patch that can be applied to trunk. > On this point, I'd like to draw attention to my own set of class-based views which provide formsets functionality, mimicking the APIs of the existing CBVs as closely as possible. The ultimate goal was to attempt to reimplement some of Django's admin as a proof-of-concept. It doesn't support any higher-level functionality like fieldsets, filtering and sorting, and there's an issue with pagination (see this ticket: https://code.djangoproject.com/ticket/16432), but it's fairly feature-complete and could be a good starting point. - Andrew -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: ORM refactoring
On Oct 10, 4:13 pm, Luke Plant wrote: > One of the problems is that it can be very hard to review refactorings. > For example, I recently checked in rev 16929 [1] from ivan_virabyan's > patch, and reviewing it was hard, despite the fact that it was a very > good quality patch. The difficulty is that there appear to be a large > number of changes, when in reality the code has just moved around. But > confirming that it has indeed stayed the same is very time consuming if > you do it properly - in fact it seems almost faster to do the same > changes yourself, and verify you get the same result. Well, the patch doesn't move a lot of code - it does actually change the code. Not that it makes it any easier to review the patch :) > This is only thinking out loud, but I'm wondering if that approach might > not be too bad. If instead of a large patch, we have a series of smaller > steps (in a github branch or whatever), perhaps that would be easier to > review? I'm not saying rework any already created patches to match this, > BTW. I too support the idea of smaller patches. But the problem with the ORM is that the bigger changes tend to cascade. Or at least that was what happened to the where node refactoring. It might be that the patch is too big to swallow. But the benefits are real: django benchmark suite shows 4-5x speed improvement for chained queryset construction. And 1.5x improvements in many cases. Those are differences important to many users. Details in the ticket. > We do have pretty good test coverage, which should be a good help, but > one of the problems here is the mutability thing - it is very difficult > to write a test that shows that shows that a new QuerySet is not sharing > data structures with the old in a way that is incorrect and could lead > to the previous QuerySet being modified by the new one (or subsequent ones). The patched version actually does share structures - the leaf nodes of the where tree. They are only cloned when relabel_aliases is called, and that together with avoiding deepcopy is where the speed difference comes. Ok, the where trees are also kept as small as possible, so that too affects performance. > It was me who suggested the idea of basing the ORM on SQLAlchemy in > future, on my blog [2]. It would be a very large change, especially > things related to the unit-of-work stuff, and I have no idea whether it > is even feasible really. > > My concern is that the ORM currently leaves you high and dry when you > come to the end of the queries it can generate. Some people say the > answer is raw SQL at that point, but if you have built up your query > dynamically in any way, or perhaps in a completely Model agnostic way > (as I was doing in django-easyfilters), you are completely stuck, > because you are left with a dynamically created query that you really > cannot usefully manipulate in any way, but which you must manipulate in > order to do the queries you need. I think the best you could do is > generate the SQL from the query as a string (which is actually hard to > do currently due to the way that SQL generation interacts with the > different backends), parse it again with python-sqlparse, manipulate and > pass to QuerySet.raw(). And then somehow cope with the backend specific > stuff. Ugh. > > SQLA, I believe, would be very different, because 'core' provides a > complete wrapper around SQL, and the ORM allows you to drop down to core > to do advanced things. And it has a much nicer way of dealing with the > different SQL dialects. In the perfect world you could have a SQLA backend, or qs.as_sqla() method. That would be neat, yes. QuerySet chaining could be hard to implement, though (or does SQLA have support for something like that?). But in my opinion this is another argument for refactoring the ORM. When the ORM has better separation of concerns and better abstractions between compiler and sql/query.py it will be easier to write different compilers - be it a compiler for Cassandra or SQLA. > I do think that these refactorings you are doing are valuable. However, > if you work on this, I think you need to be prepared for the possibility > that the work may be a dead end. It could be an experiment that shows > that a different approach will be needed. If it does that, it will still > have been useful work, but you will need a certain kind of attitude to > still consider it useful work if we end up switching to SQLA :-) I am fine with the code being discarded, no problems with that. I just want it to be at least somewhat useful... My biggest worry is not getting any response at all. That means I do work which is not useful at all. And trust me, I have already met some dead ends in my refactorings. You may not have seen them, but they do exist... :) - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe fro
Re: Custom transform in django docs
On 08/10/11 19:04, Luke Plant wrote: > Would we consider removing this custom transform? If we did, in addition > to the 32 instance of indentation that already need fixing, there are 12 > other instances that would need fixing. (These places can be found > easily by grepping the output for ' transform applied). These relative figures on their own suggest the > confusion added is greater than the benefit received - we are only > making correct use of this feature in 12 places, and messing up in 32 > places. I was wrong about this - there are actually about 60 places where we are making use this feature in the docs, in less than 10 files. I still think we should fix them. Any objections? Luke -- "My capacity for happiness you could fit into a matchbox without taking out the matches first." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Custom transform in django docs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/10/2011 09:40 AM, Luke Plant wrote: > I was wrong about this - there are actually about 60 places where we are > making use this feature in the docs, in less than 10 files. I still > think we should fix them. > > Any objections? +1 from me - I think the bar should be pretty high for what's worth a departure from standard ReST syntax, and this doesn't seem to meet that bar as far as I can tell. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6TFi4ACgkQ8W4rlRKtE2cb2QCggHtMm2Fz9dvvolEb4VmYLV9q pe8AnjLamoVNAHqJ5PxcS8FIcTPAIC/0 =S+Kk -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Russell, On 10/10/2011 06:13 AM, Russell Keith-Magee wrote: > Getting "admin-like" functionality into the mainstream was one of the > prime motivations behind the introduction of class-based generic > views. Django's admin contains a number of neat features (like > fieldsets, views with inline formsets, filtering and searching) that > can be very easily implemented as a set of class-based mixins or > extensions. I've got code in private repositories that implement some > of these features; I'd like to find the time to clean them up as > generic utilities. FWIW, TemplateResponse was introduced in the same release as the CBVs, and it allows these same generic utilities (sorting, filtering, pagination) to be easily implemented as reusable decorators on top of TemplateResponse-using views. Having tried both approaches in different projects, I find the code using TemplateResponse and decorators more concise, readable and maintainable than the equivalent using CBV mixins; there are not nearly as many subtle ordering and what-calls-what-when issues, method names to memorize, and extraneous super() noise. If you do get the CBV versions cleaned-up and public, I'd be happy to put together the equivalent in decorator style for a head-to-head comparison :-) I also agree with tWoolie that as much of the functionality as possible should be implemented separate from anything that's tied to views, so it's more easily reusable for e.g. filtering two different lists in the same view. The decorators and/or mixins should be a relatively light glue layer between views and some kind of Sort/Filter objects (roughly analogous to the Paginator object). Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6TGP0ACgkQ8W4rlRKtE2coFACffzpv6SZ621Qe0TARJlNYP0bX AdEAniAeOrZNHjxTtpxz04avwZ1vA/Nj =t1A5 -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: ORM refactoring
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Anssi, Thanks for your work on #17025 - sounds like an impressive improvement. On 10/09/2011 01:28 PM, Anssi Kääriäinen wrote: > I really do not know how to move forward. I guess I am asking all you > what do you think should be done with the ORM. We could take a > completely opposite direction and make it a wrapper around SQLAlchemy > (I saw this mentioned somewhere some time ago). Or decide that the ORM > is just fine now, or that there are higher importance items. Before > continuing my hacking, it would be nice to know if there is support > for ORM refactoring. I am not sure the "wrapper around SQLAlchemy" idea is at all feasible, for any number of reasons; it's certainly not an idea that should hold up other improvements. I do think there are parts of the ORM that could really benefit from the kind of cleanup and refactoring you're talking about, so consider me in favor. I can't make specific promises about review, but I have #17025 high on my list to look at, and would make an effort to review any patches you work on in this area. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6TGvEACgkQ8W4rlRKtE2dphgCfa+u2GsqbNuQbZCHjgNkuwKre +JsAoICOuSAQ82lrE6oPNHmkDIrqKhSA =K6a2 -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Ipython 0.11 + Custom Profiles
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Chris, On 10/10/2011 05:52 AM, Chris Green wrote: > I recently spent a lot of time trying to get my models automatically > imported into IPython 0.11 from the shell. command. I finally got it > working by making a new ishell command from the existing shell.py > (http://greencm.blogspot.com/2011/10/django-ipython-011-models.html). > > The default django shell.py never calls the IPython Profile hooks. I > think the one patch > (https://code.djangoproject.com/attachment/ticket/14892/IPython.patch) > ran shell() rather than embed. > > import IPython > IPython.embed() > > That causes the default profiles to be loaded. Given that the patch > was 11 months before 0.11 final was out, it's possible the API drifted > along the way. I had the same trouble you did, only didn't pursue it very far. I don't see any reason "manage.py shell" shouldn't support IPython profiles, and would support a ticket/patch to this end. If it uses an IPython API that's 0.11-only, though, it needs to have a fallback to still support 0.10 as well. Carl -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6THKQACgkQ8W4rlRKtE2c8/ACgmMdvnesM4X1zZlcoDw+/TDtt 2qIAn19W0bExqZE/AWhVnJJVQLQqARsi =/vI5 -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
On 10/10/11 17:10, Carl Meyer wrote: > FWIW, TemplateResponse was introduced in the same release as the CBVs, > and it allows these same generic utilities (sorting, filtering, > pagination) to be easily implemented as reusable decorators on top of > TemplateResponse-using views. Having tried both approaches in different > projects, I find the code using TemplateResponse and decorators more > concise, readable and maintainable than the equivalent using CBV mixins; > there are not nearly as many subtle ordering and what-calls-what-when > issues, method names to memorize, and extraneous super() noise. I haven't tried using TemplateResponse much, but I have tried CBVs, and I've also found with relatively small projects that mixin ordering etc can be a real pain. Example: http://goo.gl/pzEVF (I learnt about the type.mro() method, which was fun, but that kind of thing should *not* be needed!) Regards, Luke -- "My capacity for happiness you could fit into a matchbox without taking out the matches first." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Custom transform in django docs
On Sat, Oct 8, 2011 at 1:04 PM, Luke Plant wrote: > The reason for this appears to be that it allows you to write > restructured text with extra levels of indentation which would normally > be interpreted as a blockquote. Heh, so actually funny story. Turns out I copied this hack was over from the code I used for the Django book. We did it there to surpress a difference between the HTML conversion we used on the website and the OpenOffice conversion we used for the print publication. So it's actually an artifact of our hacked together book toolchain. I'm pretty surprised that it's still there -- I'd totally forgotten about this hack. By all means rip it out. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
TEMPLATE_DIRS in django.contrib.auth.tests.views
contrib.auth.tests.views line 28 overrides settings.TEMPLATE_DIRS causing tests to fail for signal hooks that use templates in locations specified in settings.TEMPLATE_DIRS. Is this a bug? I can't immediately see the reason for overriding here. I understand that the test runner needs its own templates, but why not just add this location to the tuple? At very least, it causes in an unexpected result. -- Justin Holmes Head Instructor, SlashRoot Collective SlashRoot: Coffee House and Tech Dojo 60 Main Street New Paltz, NY 12561 845.633.8330 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Corporate CLA
I am interested in getting the SlashRoot membership all filed with CLA's. However, people who work on django at SlashRoot aren't "employees" - they're members. Can we simply change the language of the CLA or are the legal wranglings more complex than that? Might it be simpler for each of us to simply fill out the individual form? Or does that not cover the proper basis of members be able to work on django on paid company time? -- Justin Holmes Head Instructor, SlashRoot Collective SlashRoot: Coffee House and Tech Dojo 60 Main Street New Paltz, NY 12561 845.633.8330 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Cleaning up manage.py and import paths
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, In the spirit of making Django behave better as a Python library (c.f. Glyph's keynote at djangocon.us), I'd like to finally tackle removing the sys.path hacks in django.core.management.setup_environ. I'll give the full detailed rundown here on the current behavior and how I propose to change it. Fortunately, the fix isn't that complicated, and I think it's a no-brainer. For the impatient, you can skip straight to my proposed patch [2]. The tl;dr summary is that I think with some small changes to manage.py and the default project layout, we can fix some very common bugs and deployment problems, dramatically reduce the extent to which manage.py is "unknown magic" for newcomers to Django, and take a significant step towards being "just a Python library." This issue is tracked in ticket #15372 [1], though there's a lot more detailed info in this message than on that ticket. The problem === Right now "django-admin.py startproject mysite" generates this: mysite/ __init__.py manage.py settings.py urls.py This layout is the root of the problem, because it commingles manage.py (a command-line script entry point that should not be imported) with importable modules inside a package. When Python runs a command-line script it puts the script's directory onto sys.path (even if you run "python /full/path/to/mysite/manage.py" from somewhere else entirely). So with this layout, via manage.py, "import settings" and "import urls" will both work, and if we add any other modules or packages to this directory, they are also importable as top-level modules/packages. This is a standard feature of Python. But "mysite" is a package, and Django wants us to be able to import "mysite.settings" and "mysite.urls" (in fact, the latter is the default ROOT_URLCONF setting). In order to make that possible, setup_environ does some magic: it finds the parent directory of "mysite", temporarily adds it to sys.path, imports mysite, and then reverts sys.path. Now we can either "import settings" or "import mysite.settings", and both will work. Same is true for any other modules or packages (e.g. apps) we add under mysite/. How nice! Not really. This bit of clever causes a boat-load of problems: 1. Python's importer doesn't know that the same module imported under two different names is the same module, so it imports it twice. Which means module-level code can run twice, which causes really confusing bugs. 2. People write code that imports things inconsistently (sometimes with the project prefix, sometimes without it), and then when they go to deploy (without manage.py or setup_environ), their imports break and they don't understand why. This comes up frequently in #django. In order to fix it they either have to make all their imports consistent, or add both the inner and outer directories to sys.path/PYTHONPATH in their deployment setup. Inconsistent imports should fail fast, not when you move to production. 2a. Even worse, our tutorial now encourages people to make their imports inconsistent! ROOT_URLCONF is "mysite.urls" but we tell people to use just "polls" rather than "mysite.polls" - so any tutorial-based project now requires the double import paths in order to even run. 3. Since it's not obvious what Django is doing, people don't always realize that the name of the outer directory matters, they think it's just a container (which is what it ought to be). If they commit the startproject layout at the top of a VCS repo, then check it out under a different name, imports break. (If they check it out under a name that isn't a valid Python module name, setup_environ breaks entirely). 4. The temporary addition to sys.path can cause unrelated packages adjacent to the project to be imported, if the import of the project module triggers an import of a name that happens to exist adjacent. This is extremely difficult to track down; since Django restores sys.path, it appears the adjacent package is somehow imported without even being on sys.path. This can catch very experienced Python devs; I once spent an hour on IRC working with Ned Batchelder to track this down in his project. 5. Understanding Python import paths is hard enough for beginners. The Django tutorial could help them on the road to understanding, with no loss in usability; pulling a trick like this behind the scenes helps them stay confused instead. The solution The key is to fix the "startproject" layout: mysite/ manage.py mysite/ __init__.py settings.py urls.py The outer "mysite/" is now just a container, not a Python package. It can be renamed without consequence. The inner "mysite/" is the Python package, and contains settings and urls. This means they are no longer importable as top-level modules; they must always be "mysite.settings" and "mysite.urls", which is better than having them pollute the to
Re: ORM refactoring
On 10/10/2011 04:59 PM, Anssi Kääriäinen wrote: In the perfect world you could have a SQLA backend, or qs.as_sqla() method. That would be neat, yes. QuerySet chaining could be hard to implement, though (or does SQLA have support for something like that?). But in my opinion this is another argument for refactoring the ORM. When the ORM has better separation of concerns and better abstractions between compiler and sql/query.py it will be easier to write different compilers - be it a compiler for Cassandra or SQLA. What's required to make different database backends easy are correct abstractions -- currently the ORM backend abstraction is pretty messed up, at least it's not written with non-SQL backends in mind. /Everything/ SQL related should be moved into a base SQL backend. The code in django.db.models should be completely query language/database agnostic. Currently, SQL generation is scattered over multiple places and abstraction layers, django.db.models.sql.* and django.db.backends etc. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Cleaning up manage.py and import paths
On Mon. 2011-10-10 at 06:05 PM EDT, Carl Meyer wrote: > In the spirit of making Django behave better as a Python library (c.f. > Glyph's keynote at djangocon.us), I'd like to finally tackle removing > the sys.path hacks in django.core.management.setup_environ. Yes! I can't wait. I cannot count the hours I wasted in confusion caused by these inconsistencies. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Corporate CLA
On Tue, Oct 11, 2011 at 5:57 AM, Justin Holmes wrote: > I am interested in getting the SlashRoot membership all filed with CLA's. > > However, people who work on django at SlashRoot aren't "employees" - > they're members. Can we simply change the language of the CLA or are > the legal wranglings more complex than that? > > Might it be simpler for each of us to simply fill out the individual > form? Or does that not cover the proper basis of members be able to > work on django on paid company time? I'm not sure that the employee/member distinction that you're making here is significant from the point of view of the CLA. We need a corporate CLA to cover those jurisdictions where companies implicitly or contractually own the work of anyone working for them. Whether those people are called "members" or "employees" doesn't really matter; we need the corporate entity to release their copyright, and enumerate the people covered by that release. However, it's certainly something worth clarifying. I'd also be wary of just altering the language to say "member" -- certain words can be loaded when it comes to legal documents, and I don't know the status of "member" when it comes to US law. I'll add this to the list of things we need to ask the lawyers. It looks like we need to do some work here anyway, because in reviewing the corporate CLA to answer your question, I noticed that it references a "Schedule B" that doesn't exist... Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Corporate CLA
On Mon, Oct 10, 2011 at 7:51 PM, Russell Keith-Magee < russ...@keith-magee.com> wrote: > On Tue, Oct 11, 2011 at 5:57 AM, Justin Holmes > wrote: > > I am interested in getting the SlashRoot membership all filed with CLA's. > > > > However, people who work on django at SlashRoot aren't "employees" - > > they're members. Can we simply change the language of the CLA or are > > the legal wranglings more complex than that? > > > > Might it be simpler for each of us to simply fill out the individual > > form? Or does that not cover the proper basis of members be able to > > work on django on paid company time? > > I'm not sure that the employee/member distinction that you're making > here is significant from the point of view of the CLA. We need a > corporate CLA to cover those jurisdictions where companies implicitly > or contractually own the work of anyone working for them. Whether > those people are called "members" or "employees" doesn't really > matter; we need the corporate entity to release their copyright, and > enumerate the people covered by that release. > > However, it's certainly something worth clarifying. I'd also be wary > of just altering the language to say "member" -- certain words can be > loaded when it comes to legal documents, and I don't know the status > of "member" when it comes to US law. I'll add this to the list of > things we need to ask the lawyers. It looks like we need to do some > work here anyway, because in reviewing the corporate CLA to answer > your question, I noticed that it references a "Schedule B" that > doesn't exist... > > Yours, > Russ Magee %-) > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > To be (layman's understanding) the purpose of the corporate CLA is that in many jurisdictions an employer owns the product of work done by it's employees, even if it's not done directly as a part of employment. Unless this same relationship exists between you and your members, you cannot have such a CLA, because you don't legally own their work and thus can't license it. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: TEMPLATE_DIRS in django.contrib.auth.tests.views
On Tue, Oct 11, 2011 at 5:32 AM, Justin Holmes wrote: > contrib.auth.tests.views line 28 overrides settings.TEMPLATE_DIRS > causing tests to fail for signal hooks that use templates in locations > specified in settings.TEMPLATE_DIRS. > > Is this a bug? I can't immediately see the reason for overriding > here. I understand that the test runner needs its own templates, but > why not just add this location to the tuple? At very least, it causes > in an unexpected result. The reason for overriding TEMPLATE_DIRS is to keep the tests isolated. The theory is that it shouldn't matter what templates you have in your production setup; the auth tests should be self contained. The traditional way this manifests is with the login page template -- if you had a project that *didn't* deploy the standard login view, the auth tests would fail because they couldn't find the login template for the standard login view when that view was tested. We can't just add an extra location to the tuple because we still end up with an unpredictable environment -- consider the case where we are testing the fact that a template *doesn't* exist, but your production environment provides one. The full fix here is to provide better environment isolation during testing. Ideally, we should be constructing a completely clean settings object for every test run, explicitly introducing the extra settings that we need (rather than the current model taking system settings, and 'fixing' the settings that are problematic). Ideally, this would be a full 'system flush', including settings app caches, and every other runtime singleton that Django uses. However, this is a non-trivial piece of work. A very *useful*, non-trivial piece of work, but still non-trivial :-) Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Corporate CLA
On Tue, Oct 11, 2011 at 7:54 AM, Alex Gaynor wrote: > > > On Mon, Oct 10, 2011 at 7:51 PM, Russell Keith-Magee > wrote: >> >> On Tue, Oct 11, 2011 at 5:57 AM, Justin Holmes >> wrote: >> > I am interested in getting the SlashRoot membership all filed with >> > CLA's. >> > >> > However, people who work on django at SlashRoot aren't "employees" - >> > they're members. Can we simply change the language of the CLA or are >> > the legal wranglings more complex than that? >> > >> > Might it be simpler for each of us to simply fill out the individual >> > form? Or does that not cover the proper basis of members be able to >> > work on django on paid company time? >> >> I'm not sure that the employee/member distinction that you're making >> here is significant from the point of view of the CLA. We need a >> corporate CLA to cover those jurisdictions where companies implicitly >> or contractually own the work of anyone working for them. Whether >> those people are called "members" or "employees" doesn't really >> matter; we need the corporate entity to release their copyright, and >> enumerate the people covered by that release. >> >> However, it's certainly something worth clarifying. I'd also be wary >> of just altering the language to say "member" -- certain words can be >> loaded when it comes to legal documents, and I don't know the status >> of "member" when it comes to US law. I'll add this to the list of >> things we need to ask the lawyers. It looks like we need to do some >> work here anyway, because in reviewing the corporate CLA to answer >> your question, I noticed that it references a "Schedule B" that >> doesn't exist... >> >> Yours, >> Russ Magee %-) >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers" group. >> To post to this group, send email to django-developers@googlegroups.com. >> To unsubscribe from this group, send email to >> django-developers+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-developers?hl=en. >> > > To be (layman's understanding) the purpose of the corporate CLA is that in > many jurisdictions an employer owns the product of work done by it's > employees, even if it's not done directly as a part of employment. Unless > this same relationship exists between you and your members, you cannot have > such a CLA, because you don't legally own their work and thus can't license > it. And there's the rub. Is a "member" an "employee" for the purposes of corporate copyright assignment? I can certainly see the argument that you're making, but unfortunately IANAL doesn't get the job done here. :-) Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Cleaning up manage.py and import paths
On 10/10/11 23:05, Carl Meyer wrote: > Unless there are significant objections, I'd like to commit this in the > next few days and get it into 1.4. I'll also modify the patch on #16360 > to take advantage of the new layout; hopefully that can get into 1.4 as > well. +1. Spent hours cleaning up a project with inconsistent imports recently. And everything you suggested is how I'd like to see it done. Luke -- "My middle name is 'Organised'! My first name is 'Poorly'." Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
Hi, On 10 October 2011 15:20, tWoolie wrote: > I'd like to propose a project to split out the table/list > functionality and the filtering functionality in the django admin into > a seperate contrib app. This is really core functionality which is > lacking from django (in terms of being locked to the admin app). > I believe this would be beneficial because currently there are a lot > of people who love using the admin interface but find building the > same functionality into their own applications daunting at best. This > leads a lot of people to try to abuse the admin app by allowing > regular users into there. > > I propose moving a lot of the django-tables2 functionality into a > contrib app and augumenting it with filterspec. Then the admin can use > this app, or any custom subclass of Table/Filter. Currently, django- > tables2 is fantastic but has no support for filterspecs which would be > super super helpful, and would benefit from the level of [url] > abstraction that d-t2 provides. > > I'm posting to this group to gather feedback and ask if anyone would > actually like to see this happen. I believe it's something i'd love to > contribute but only if the django devs would be interested in > incorporating it. > > Thomas Making the admin's codebase more flexible and reusable is definitely something that we should aim for. In fact, there have been lots of group discussions about this during DjangoCon. Idan and I will try to post a detailed summary to this list soon. In a nutshell, the aim is to make the admin more lightweight and modular so that its various components can easily be replaced or used in different contexts. It's quite likely that a full rewrite will eventually be necessary, following the same path as oldforms/newforms: "newadmin". However, I think at this stage one great way to start would be to refactor the current admin to make use of class-based views. This would be good for a number of reasons: - Dogfooding: having a concrete real-world app in Django core would help drive improvements and extensions to the CBVs themselves. - The admin codebase would reach a position where abstracting its core functionality out gets easier. - This would help quickly anticipate issues (both from a technical and usability point of view) to do with ravioli code [1]. - A complete rewrite would likely take time and would realistically not ship in Django before the next 2-3 releases -- unless someone pulls out the work to make me lie :-). Adding CBVs would quickly open the door to a lot of possible improvements in oldadmin, which would in turn inform the parallel development of newadmin. - This would make the future transition and upgrade path to newadmin easier. If someone is willing to take this on then I'd be happy to provide some support by reviewing patches, giving feedback etc. Cheers, Julien [1] http://lukeplant.me.uk/blog/posts/class-based-views-and-dry-ravioli/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
> If someone is willing to take this on then I'd be happy to provide some > support by reviewing patches, giving feedback etc. Cheers. Would you be willing to review code that swaps out entire swathes of admin functionality? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
On 11 October 2011 12:22, tWoolie wrote: > > If someone is willing to take this on then I'd be happy to provide some > > support by reviewing patches, giving feedback etc. > > Cheers. Would you be willing to review code that swaps out entire > swathes of admin functionality? Yep. Could you outline your plan of attack for this? Julien -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
What would be the value in replacing the admin with a new version with abstracted, reusable tools... that still resides in the new admin? IMO, duplicating the admin's features would be better than that (assuming you can't simply change the admin's guts in-place, which is probably the case). On Oct 10, 9:04 pm, tWoolie wrote: > > Yep. Could you outline your plan of attack for this? > > 1) create "newadmin" project to be a dropin replacement for admin. > (Clone current admin and slowly replace features/trim dead code) > > 2) Split out filterspec functionality into newadmin.filter. > - Give it a nicer wrapper such that a FilterSet can be declared and > instantiantiated much like newforms > - Move API for registering filters out of ModelAdmin and into > newadmin.filter > > 3) Split out table generating/sorting/display into newadmin.table > - clone django_tables2 and do some refactoring as necessary > - replace admin functionality with admin.table functionality > > 4) Develop CBV Wrappers for the core functionality > - replace admin views with CVB style views -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
On 11 October 2011 14:04, tWoolie wrote: > > Yep. Could you outline your plan of attack for this? > > 1) create "newadmin" project to be a dropin replacement for admin. >(Clone current admin and slowly replace features/trim dead code) > > 2) Split out filterspec functionality into newadmin.filter. > - Give it a nicer wrapper such that a FilterSet can be declared and > instantiantiated much like newforms > - Move API for registering filters out of ModelAdmin and into > newadmin.filter > > 3) Split out table generating/sorting/display into newadmin.table > - clone django_tables2 and do some refactoring as necessary > - replace admin functionality with admin.table functionality > > 4) Develop CBV Wrappers for the core functionality > - replace admin views with CVB style views Actually, my proposal at this stage simply is to start with step #4 (i.e. Turn admin views into CBVs, in a backwards-compatible way and without changing existing functionality yet) with the hope that the core functionality would then be more easily extractable and could be made more generic. I'd be reluctant to start a "newadmin" in core just yet without extensive design discussions first taking place on this topic. However, nothing prevents you or anyone who's interested to push this along by starting it outside of Django and then potentially merge it to core once it reaches a satisfactory state. I see that step #4 could be done in core regardless, though, and my point is that this work would benefit other efforts that are put into the full newadmin rewrite as it'd give us some empirical evidence of what does or doesn't work, quickly highlighting any major snag in the process. Julien -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Splitting out admin functionality
> Yep. Could you outline your plan of attack for this? 1) create "newadmin" project to be a dropin replacement for admin. (Clone current admin and slowly replace features/trim dead code) 2) Split out filterspec functionality into newadmin.filter. - Give it a nicer wrapper such that a FilterSet can be declared and instantiantiated much like newforms - Move API for registering filters out of ModelAdmin and into newadmin.filter 3) Split out table generating/sorting/display into newadmin.table - clone django_tables2 and do some refactoring as necessary - replace admin functionality with admin.table functionality 4) Develop CBV Wrappers for the core functionality - replace admin views with CVB style views -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
