Re: Multi-Threaded Dev Server
I'm a bit confused how multi-threading has anything to do with AJAX? Your requests will be slower, but they will still get processed (at least I've never had any issues). On Nov 16, 1:03 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > If the patch that's currently on there works with no changes at all, > I'd be hesitantly in favor of it, however if there any bugs, or other > complications associated with it, my answer is to use a real server, > setting up something like CherryPy locally is almost no effort, as > seen > here:http://www.oebfare.com/blog/2008/nov/03/writing-custom-management-com... > > On Nov 16, 1:39 am, "Calvin Spealman" <[EMAIL PROTECTED]> wrote: > > > Does it need to be multi-threaded? Can you get a multi-process dev > > server instead/also? > > > On Sun, Nov 16, 2008 at 1:26 AM, Chris <[EMAIL PROTECTED]> wrote: > > > > I thinkhttp://code.djangoproject.com/ticket/3357shouldbe given > > > another look at enabling optional multi-threading on the dev server. > > > > Jacob previously closed this ticket, noting the patch could introduce > > > threading bugs, and would provide functionality too similar to that of > > > a production environment. > > > > This is my rationale to accept this ticket: > > > 1. Ajax is very common place, and often requires a multi-threaded > > > server. The dev server is very convenient, but by not providing multi- > > > threading support, we're preventing it from being even more useful. > > > 2. Since multi-threading would be an optional setting, the default > > > scenario would still be single-threaded and would not break anything. > > > Only those specifically desiring multi-threading would subject > > > themselves to any unforeseen bugs. > > > 3. Fear of multi-threading bugs shouldn't be a reason to avoid multi- > > > threading, especially when it could be very useful. We don't even know > > > if there are multi-threading bugs. And even if there are, they can be > > > fixed. > > > > Regards, > > > Chris > > > -- > > Read my blog! I depend on your acceptance of my opinion! I am > > interesting!http://techblog.ironfroggy.com/ > > Follow me if you're into that sort of > > thing:http://www.twitter.com/ironfroggy --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multi-Threaded Dev Server
On Nov 16, 6:26 am, Chris <[EMAIL PROTECTED]> wrote: > I thinkhttp://code.djangoproject.com/ticket/3357should be given > another look at enabling optional multi-threading on the dev server. > > Jacob previously closed this ticket, noting the patch could introduce > threading bugs, and would provide functionality too similar to that of > a production environment. > > This is my rationale to accept this ticket: > 1. Ajax is very common place, and often requires a multi-threaded > server. The dev server is very convenient, but by not providing multi- > threading support, we're preventing it from being even more useful. > 2. Since multi-threading would be an optional setting, the default > scenario would still be single-threaded and would not break anything. > Only those specifically desiring multi-threading would subject > themselves to any unforeseen bugs. > 3. Fear of multi-threading bugs shouldn't be a reason to avoid multi- > threading, especially when it could be very useful. We don't even know > if there are multi-threading bugs. And even if there are, they can be > fixed. > > Regards, > Chris I would second what David Cramer said. You don't need, in the general case, need multi-threading on the server side for Ajax - it's really about browser-side multi-threading. I'm not sure where your point 1 comes from: I use AJAX with the standard, single-threading development server all the time, and I have not encountered any issues at all. I'd say leave it as it is, unless of course you have encountered some specific problem relating to the single-threaded nature of the server. In that case, please post some details. It's theoretically possible that with the server tied up with some long-running request, the browser might time out before the server got around to servicing an AJAX request which was next in line. However, you should be able to configure client-side time-outs appropriately to mitigate this. Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multi-Threaded Dev Server
Some months ago I tried a snippet from djangosnippets.org which shows a upload progress bar. I think some people want to use snippets like that and a multi-threaded server would help you developing such applications. 2008/11/16 Vinay Sajip <[EMAIL PROTECTED]> > > > > On Nov 16, 6:26 am, Chris <[EMAIL PROTECTED]> wrote: > > I thinkhttp://code.djangoproject.com/ticket/3357should be given > > another look at enabling optional multi-threading on the dev server. > > > > Jacob previously closed this ticket, noting the patch could introduce > > threading bugs, and would provide functionality too similar to that of > > a production environment. > > > > This is my rationale to accept this ticket: > > 1. Ajax is very common place, and often requires a multi-threaded > > server. The dev server is very convenient, but by not providing multi- > > threading support, we're preventing it from being even more useful. > > 2. Since multi-threading would be an optional setting, the default > > scenario would still be single-threaded and would not break anything. > > Only those specifically desiring multi-threading would subject > > themselves to any unforeseen bugs. > > 3. Fear of multi-threading bugs shouldn't be a reason to avoid multi- > > threading, especially when it could be very useful. We don't even know > > if there are multi-threading bugs. And even if there are, they can be > > fixed. > > > > Regards, > > Chris > > I would second what David Cramer said. You don't need, in the general > case, need multi-threading on the server side for Ajax - it's really > about browser-side multi-threading. I'm not sure where your point 1 > comes from: I use AJAX with the standard, single-threading development > server all the time, and I have not encountered any issues at all. I'd > say leave it as it is, unless of course you have encountered some > specific problem relating to the single-threaded nature of the server. > In that case, please post some details. > > It's theoretically possible that with the server tied up with some > long-running request, the browser might time out before the server got > around to servicing an AJAX request which was next in line. However, > you should be able to configure client-side time-outs appropriately to > mitigate this. > > Regards, > > Vinay Sajip > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multi-Threaded Dev Server
Julian wrote: > [...] I think some people want to use snippets like that [...] Wouldn't you agree that's a pretty feeble use case for something as potentially disruptive as multi-threaded serving? Particularly when the CherryPy alternative is so readily available. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
> (and obviously I meant 'setup.py sdist') > I don't know how setup.py bundles a tarball, but it's doing it wrong -- GeoDjango is broken in the 1.0.1 release tarball. In particular, at least the following directories were completely _omitted_ from this release: http://code.djangoproject.com/browser/django/branches/releases/1.0.X/django/contrib/gis/templates http://code.djangoproject.com/browser/django/branches/releases/1.0.X/django/contrib/gis/tests/geoapp/sql While the missing test data is OK, not having the GIS template directory breaks a lot of functionality, including the geographic admin and geo sitemaps. I ran `python setup.py sdist` and watched it skip over the directories myself -- those with superior setuptools skills can hopefully inform me as to how this can happen. -Justin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
On Nov 16, 2008, at 15:37, Justin Bronn wrote: > http://code.djangoproject.com/browser/django/branches/releases/1.0.X/django/contrib/gis/templates > http://code.djangoproject.com/browser/django/branches/releases/1.0.X/django/contrib/gis/tests/geoapp/sql > > While the missing test data is OK, not having the GIS template > directory breaks a lot of functionality, including the geographic > admin and geo sitemaps. I ran `python setup.py sdist` and watched it > skip over the directories myself -- those with superior setuptools > skills can hopefully inform me as to how this can happen. The reason is that MANIFEST.in doesn't tell setuptools to include those directories in the distribution. http://code.djangoproject.com/browser/django/trunk/MANIFEST.in - Ludvig Ericson --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
> The reason is that MANIFEST.in doesn't tell setuptools to include those > directories in the distribution. Thanks, I eventually figured this out on my own -- it was the problem. Fixed in r9473 and r9474. -Justin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multi-Threaded Dev Server
On Sun, Nov 16, 2008 at 1:26 AM, Chris <[EMAIL PROTECTED]> wrote: > > I think http://code.djangoproject.com/ticket/3357 should be given > another look at enabling optional multi-threading on the dev server. > 3. Fear of multi-threading bugs shouldn't be a reason to avoid multi- > threading, especially when it could be very useful. We don't even know > if there are multi-threading bugs. And even if there are, they can be > fixed. > Yet, the time spent identifying and fixing those buds means less time developing real features that I need and can use on my production sites. I'd say the developers time is better spent elsewhere. Especially considering there are already working solutions out there. I seem to recall at least one management command someone put together that runs a multithreaded cherrypy server. Why reinvent the wheel? Lets focus on real, useful features. -- Waylan Limberg [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
@#$%!.will this composite primary key feature be included in Django 2.0? Or will this be ever included? Let me explain my frustration a bit. My company only allows Oracle db and the legacy tables span hundreds of schemas and hundreds of tables in each schema, in multiple databases. This limitation on single primary key and single databse connection single schema is really making me feel that Django is only catering to the simple bloggers and social networkers. However, to django's credit, Django is very well designed. But without having the aforementioned resolved, i will never be able to get my team to deploy django formally. It only leaves me with a sigh of not being able to use all of this great work but having to suffer through php (yes currently our legacy reporting tool is written in php, in >1000 lines of code (5 different lanuagues) per file). I feel the only reason i check the user group now is just to see the status on this proposal...and lo and behold, it didn't even make it into 1.1 compiled feature list for rejection. What a bummer. How much longer do we have to wait. I mean can someone just take a look at sqlalchemy and emulate it? I mean come on, you people are smart, why hasn't anyone done anything. In addition, opinionated as you are, it is not hard to just opinionate your way into a decision about how the admin portion should be handled and get on with the rest. Finally, dont the great JKM and Malcolms use composite primary keys...?!! (#U%)QU Alright, i feel better now. But still, come on, surprise our community a bit. (plus i mean the only reason anyone ever goes for Sqlalchemy, isn't it just for the composite key, and multi db support? (at least for me)) Frank On Nov 13, 9:25 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Fri, 2008-11-14 at 00:17 -0500, Steve Holden wrote: > > [...] > > > Sure. Perhaps it's not common terminology: by "recursive" I meant a > > self-referential relationship, though now you mention it I see that the > > same issue can occur if the order of model definitions is sub-optimal. > > The recursive case is just a collapsed form of forwards references, so I > didn't address that specifically. You don't even need sub-optimal > ordering to require forwards references. For example, circular reference > loops (particularly with NULL relations) aren't unheard of. Even in > Django, you need forwards reference to implement the "through" option > for ManyToMany fields, since you need to define the intermediate model > with a ForeignKey to model A and then define model A with a many-to-many > through the intermediate model. So one of them has to be defined second. > All I'm saying is that forwards references crop up naturally; they're > not a wart (and references to the same model are the other obvious case > they crop up naturally). > > Regards, > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django 1.1, app() and ticket #3591
>>> Well, what are those features you wanted, explicitly?
>>
>> Mostly what has been written down
>> athttp://code.djangoproject.com/wiki/InstalledAppsRevision
>
> Thank you for your response. If you mean
>
>* Allow change of name of third-party app
>* Allow change of db_prefix of third-party app
>* Allow multiple instances of an app with different names,
> db_prefix, etc.
>> Using an app multiple times is pretty difficult since it requires
>> changes in the way the models are registered. To be honest, I'm not
>> sure what the use case is for that. Could someone give an example
>> please?
>
> I'm not sure there is a particularly good case for worrying
> excessively about multiple instances of the same app, as it's hardly
> the common case. Are not the first two of the three goals sufficient
> justification?
There was a case for multiple instances of apps when it was discussed
at the Pycon sprint and I just forgot it.
> Given that there is an ever-increasing body of
> applications out there, the chances of a name clash in the label is
> also ever-increasing. AFAIK, the #3591 patch meets the first two
> goals, assuming that by "name of third-party app" you mean a verbose
> name which can be e.g. internationalised. I would rather not bang on
> about the multiple-instances-of-the-same-app case because it would be
> a side benefit rather than a benefit which would be widely enjoyed.
> I'm not quite sure what you mean by "wasn't convinced of how it's
> bound to the settings." Did it work, in the sense that you could
> disambiguate apps and apply verbose names to them?
I mean with that sentence: I don't like the current implementation
because it extends the settings system when it really should have its
own place. I doubt for example that translated verbose_name arguments
would work -- just like it doesn't work for the LANGUAGES setting
where a dummy gettext function is needed to prevent a recursive
import. And that's just one example why it makes sense to put app
class definitions outside the settings.
> In my
> implementation, I went for minimal changes to the Django source,
> because I thought it would make it easier for people to scan,
> understand, review and hopefully accept the changes.
It's not a matter of the amount of changes, as far as I understand it.
The app handling is a sensitive topic that deserves more than a fast
fix (yes, I know the ticket is old).
> If the basic
> premise of an app class - instances of which can live in
> settings.INSTALLED_APPS - is acceptable (and, of course, this means
> instances of subclasses of app can live in settings.INSTALLED_APPS
> too) then the precise location of an implementation (e.g.
> django.core.apps) can be refined.
I'd also love to hear something from the core devs on that.
> Also, the functionality of that app
> class (the base class) can be enhanced over time. What's important as
> the first step is to have a place to hang your hat when talking about
> an app instance in a Django site.
Your patch adds app classes to the settings system while apps aren't
just a matter of setting things up.
> Previous objections by the committers have been about the apparent
> complexity of proposed changes, and I have tried to keep the design/
> implementation as minimal as possible in deference to these worries.
> However, the approach allows app-centric functionality to be refined
> over time, and I can't see any specific problems with the current
> approach which would hamper this refinement. I believe your idea of
> using app class names in INSTALLED_APPS is weaker than using app class
> instances because it does not allow you to parametrise entries in the
> simplest possible way.
Having app instances in settings.py is in sharp contrast to how the
settings.py is used currently. Extensible features like
AUTHENTICATION_BACKENDS, FILE_UPLOAD_HANDLERS, MIDDLEWARE_CLASSES,
SESSION_ENGINE, TEMPLATE_CONTEXT_PROCESSORS and TEMPLATE_LOADERS all
use tuples containing paths to Python functions or classes -- just
like INSTALLED_APPS does with module paths. The AppCache should also
be able to take paths to app classes and create instances of them on
the fly while startup.
A strong paradigm in Django for extensible features is to inherit from
base classes and/or implement a simple API and then adding it to the
settings. I don't see why this shouldn't also be done for apps.
> For example, if I have a couple of third-
> party applications whose package paths end in 'auth' and 'comments',
> and I want to also use 'django.contrib.auth' and
> 'django.contrib.comments' on my site, then I have a problem with
> app_labels 'auth' and 'comments'. With app instances in the mix, I can
> do for example
>
> INSTALLED_APPS = (
>'django.contrib.auth',
>'django.contrib.comments',
>...
>app('third.party.app1.package.path.ends.in.auth', 'tp_auth'),
>app('third.party.app2.package.path.ends.in.comments',
> 'tp_comme
Re: Proposal: Composite Foreign Keys
On Sun, Nov 16, 2008 at 2:42 PM, Frank Liu <[EMAIL PROTECTED]> wrote: > > @#$%!.will this composite primary key feature be included in > Django 2.0? Or will this be ever included? Let me explain my > frustration a bit. > > [snipped] Did I miss something? Your note makes it sound like the core team actively prevented this proposal from being considered for 1.1. I don't believe that is the case. Jacob posted this note: http://groups.google.com/group/django-developers/msg/0b517e283421034b to this list on September 15. The last paragraph of that note stated: Proposing features for 1.1 == To propose features for 1.1, make a post on django-dev, and add a link to your message at http://code.djangoproject.com/wiki/Version1.1Features. That's not an onerous procedure; anyone could propose anything for 1.1 consideration, and that wiki page was available for editing for something like two months (actually a bit longer than the published schedule had said it would be). If you wanted this feature so much, why did you not do something to make sure it got posted as a proposal on the wiki page? Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django 1.1, app() and ticket #3591
On Nov 16, 7:48 pm, Jannis Leidel <[EMAIL PROTECTED]> wrote:
> >>> Well, what are those features you wanted, explicitly?
>
> There was a case for multiple instances of apps when it was discussed
> at the Pycon sprint and I just forgot it.
>
Ok - I'm not saying there's no case for it, just that it is a rare
case and so appropriate weighting should be given to it.
> I mean with that sentence: I don't like the current implementation
> because it extends the settings system when it really should have its
> own place. I doubt for example that translated verbose_name arguments
> would work -- just like it doesn't work for the LANGUAGES setting
> where a dummy gettext function is needed to prevent a recursive
> import. And that's just one example why it makes sense to put app
> class definitions outside the settings.
>
> It's not a matter of the amount of changes, as far as I understand it.
> The app handling is a sensitive topic that deserves more than a fast
> fix (yes, I know the ticket is old).
>
I agree with you that django.conf is not the best place for the app
class to live, and the implementation is not like it is because of
wanting a fast fix. As you point out, the ticket and the patch are
pretty old. If the core developers say the basic idea is fine, and to
refactor it into e.g. django.core.apps, I'll get on it right away. But
up till now there's been absolutely no review from the core devs or
anyone else where specific points like the one you bring up have been
made; I've been waiting a long time for any specific comments at all,
so thanks for those.
> > If the basic
> > premise of an app class - instances of which can live in
> > settings.INSTALLED_APPS - is acceptable (and, of course, this means
> > instances of subclasses of app can live in settings.INSTALLED_APPS
> > too) then the precise location of an implementation (e.g.
> > django.core.apps) can be refined.
>
> I'd also love to hear something from the core devs on that.
Yep, let's see what they say. Malcolm has already voted -1, so does
this mean it's not going ahead for 1.1? According to Jacob's original
post, this would be killed now. But Malcolm has not given any specific
criticisms of the patch, and spoken only in generalities. And Jacob
asked me in private mail to start this thread, to which he expects to
contribute.
>
> Your patch adds app classes to the settings system while apps aren't
> just a matter of setting things up.
>
Sure, and I'd be fine with putting app-related stuff in e.g.
django.core.apps and importing those into settings.py. I see that as a
minor refactoring issue. The placement of the app class in the conf
system was pragmatic, to see if the idea would fly. No core dev ever
commented on this patch, so it stayed as it was.
>
> Having app instances in settings.py is in sharp contrast to how the
> settings.py is used currently. Extensible features like
> AUTHENTICATION_BACKENDS, FILE_UPLOAD_HANDLERS, MIDDLEWARE_CLASSES,
> SESSION_ENGINE, TEMPLATE_CONTEXT_PROCESSORS and TEMPLATE_LOADERS all
> use tuples containing paths to Python functions or classes -- just
> like INSTALLED_APPS does with module paths. The AppCache should also
> be able to take paths to app classes and create instances of them on
> the fly while startup.
It's a debatable point, and I'm happy to see discussion and comments
from others. I tend to the pragmatic, so how (in more detail) would
you propose tackling the specific use case I mentioned with clashing
auth and comments apps? How would settings.py look in your ideal
scenario?
An analogy with using app instances in settings.INSTALLED_APPS might
arise from looking at how the use of either functions or paths to
functions in allowed in urlconfs. I don't think anyone is insisting
that urlconfs should contain only paths to functions - the utility of
using functions there is well-established, I would say.
> A strong paradigm in Django for extensible features is to inherit from
> base classes and/or implement a simple API and then adding it to the
> settings. I don't see why this shouldn't also be done for apps.
>
I don't see how my suggestion precludes that. I even gave an example
with a derived app class.
>
> Not true if each app instance knows it's full module path. The
> AppCache should use that full python path when handling the list of
> installed apps.
>
The point is, I don't want to touch a third-party app's code if I
don't need to. What if I want to use an existing app which doesn't
care what its full path is? Can you give some code snippets like I
did, to show how things should work, in your view?
>
>
> > However, if you want to have specific app classes, then you can import
> > them into settings.py and have entries such as
>
> > from my.package import MyCustomAppSubclass
>
> > INSTALLED_APPS = (
> >'django.contrib.auth',
> >'django.contrib.comments',
> >...
> >app('third.party.app1.package.path.ends.in.auth', 'tp_auth'),
> >app('third.party.app2.package.path.en
Re: Multi-Threaded Dev Server
On Nov 16, 5:21 pm, "Waylan Limberg" <[EMAIL PROTECTED]> wrote: > On Sun, Nov 16, 2008 at 1:26 AM, Chris <[EMAIL PROTECTED]> wrote: > Especially considering there are already working solutions out there. > I seem to recall at least one management command someone put together > that runs a multithreaded cherrypy server. Why reinvent the wheel? > Lets focus on real, useful features. > Especially since Alex Gaynor helpfully posted a link to that specific solution earlier in this thread. Here it is again: http://www.oebfare.com/blog/2008/nov/03/writing-custom-management-command/ Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Denormalisation Magic, Round Two
i just implemented something like this. it useses decorator-syntax to create a denormalized field from a callable. i think the advantages of this aproach are: 1) it's very flexible 2) all code related to the denormalisiation is located in one central place, and not one place for the field definition and another place for the actual code i also implemented a management command for rebuilding all the denormalisations. a (working) example: http://github.com/initcrash/django-denorm/tree/master/example/gallery/models.py the actual implementation of the decorator contains quite a few ugly hacks, maybe this can be done in a cleaner way by changing some code inside django itself. http://github.com/initcrash/django-denorm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
I guess i thought the core developers would've added this proposal to the list (judging from the title of this thread). Regardless, even if this had been added, it would've been given a -1 anyway, judging from the readiness of this feature. More importantly though, my gripe about this is that if you go through a search for this feature, you would see entries from 2006. Over the last 2 years, there were multiple threads on this and yet nothing was really done to it. How many time have we seen attempts at integrating sqlalchemy and django? And really what features did people intend to add? It's really obvious (at least to me) that it's for the multi db support and composite primary key (I mean exactly what else does sqlalchemy do better, ok more backends, what else?). Why don't people get this? And let me correct myself a bit. It's not that no one has done anything, it's just that people have been only writing bits and pieces of hacks here and there to patch the flaws, but this problem has proven to be more difficult than just patches. For example, there are numerous suggestions in this thread in the API of the model. However, SO WHAT? does it matter? I like both. Both have merits. THEN WHAT? what about the admin layer, what this and that relationship, what about these recursive thing. Almost every one of these composite key threads had been stalled somewhere either in admin or the model api that nothing substantial was done. (usually a thread like this goes on a few 10s of exchanges and finally left cold...just like this one). And I think this is not the developers' fault. I think fundamentally there are 2 reasons for this: 1. the original design of the orm is not up to the task so that extending it proved to be too difficult. As have been mentioned in numerous places, the orm is one place where loose coupling should be taken with a grain of salt. 2 lack of leadership. To deal with this difficult situation, it is important to have the big guys like JKM (ok i don't know any other obvious names, but JKM here represent the core team) to say, "ok now let's address the orm issues and this has to be done before release of 1.1" and so the smart minds of everyone else will start churning AND making those opinionated decisions that django is so famous for (which most of time is really brilliant). But no, JKM never has any interest in this because either he doesn't work in an enterprise environment anymore (because he's too successful unlike the rest of us still working in an Office Space envrionement) or django had seen tremendous growth even without it, why bother. (what i mean to say here is JKM, please post a note on the main site and mobilize your goons to address this issue. I am pretty sure it would be done before i go to work tomorrow). Finally, don't get me wrong, i love django and python. It is simply the best language and best web framework out there for me (where enterprise reporting fit beautifully into a micro-app type of architecture instead of pylons or turbogears full blown application architecture for each report). But the orm really drives me crazy. Of course we all wrote custom managers, and overriding save to deal with the composite key short comings, but come on, there's gotta be some more pythonic (battery-included, there's only 1 obvious way) way of dealing with this. Frank On Nov 16, 12:12 pm, "Karen Tracey" <[EMAIL PROTECTED]> wrote: > On Sun, Nov 16, 2008 at 2:42 PM, Frank Liu <[EMAIL PROTECTED]> wrote: > > > @#$%!.will this composite primary key feature be included in > > Django 2.0? Or will this be ever included? Let me explain my > > frustration a bit. > > [snipped] > > Did I miss something? Your note makes it sound like the core team actively > prevented this proposal from being considered for 1.1. I don't believe that > is the case. > > Jacob posted this note: > > http://groups.google.com/group/django-developers/msg/0b517e283421034b > > to this list on September 15. The last paragraph of that note stated: > > Proposing features for 1.1 > == > > To propose features for 1.1, make a post on django-dev, and add a link to > your message athttp://code.djangoproject.com/wiki/Version1.1Features. > > That's not an onerous procedure; anyone could propose anything for 1.1 > consideration, and that wiki page was available for editing for something > like two months (actually a bit longer than the published schedule had said > it would be). If you wanted this feature so much, why did you not do > something to make sure it got posted as a proposal on the wiki page? > > Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~
Re: Proposal: Composite Foreign Keys
Frank Liu wrote: > I guess i thought the core developers would've added this proposal to > the list (judging from the title of this thread). > > Regardless, even if this had been added, it would've been given a -1 > anyway, judging from the readiness of this feature. > > More importantly though, my gripe about this is that if you go through > a search for this feature, you would see entries from 2006. Over the > last 2 years, there were multiple threads on this and yet nothing was > really done to it. How many time have we seen attempts at integrating > sqlalchemy and django? And really what features did people intend to > add? It's really obvious (at least to me) that it's for the multi db > support and composite primary key (I mean exactly what else does > sqlalchemy do better, ok more backends, what else?). Why don't people > get this? > Whether people "get it" or not, proposing the integration of an entirely different database front-end (it would be unfair to characterize SQLAlchemy as an ORM: it's actually quite a lot more than that) is almost certainly way too radical a step for a point release. It may well be way too radical a step for release 2. OK, so it's an important use case for you, but surely you can see that puts you in the minority, and the development team have to think about the majority first. > And let me correct myself a bit. It's not that no one has done > anything, it's just that people have been only writing bits and pieces > of hacks here and there to patch the flaws, but this problem has > proven to be more difficult than just patches. For example, there are > numerous suggestions in this thread in the API of the model. However, > SO WHAT? does it matter? I like both. Both have merits. THEN WHAT? > what about the admin layer, what this and that relationship, what > about these recursive thing. Almost every one of these composite key > threads had been stalled somewhere either in admin or the model api > that nothing substantial was done. (usually a thread like this goes on > a few 10s of exchanges and finally left cold...just like this one). > > And I think this is not the developers' fault. I think fundamentally > there are 2 reasons for this: 1. the original design of the orm is not > up to the task so that extending it proved to be too difficult. As > have been mentioned in numerous places, the orm is one place where > loose coupling should be taken with a grain of salt. > > Well whether this is true or not you can surely see that some kind of radical change would be needed for you to have your pony. Or would you disagree? > 2 lack of leadership. To deal with this difficult situation, it is > important to have the big guys like JKM (ok i don't know any other > obvious names, but JKM here represent the core team) to say, "ok now > let's address the orm issues and this has to be done before release of > 1.1" and so the smart minds of everyone else will start churning AND > making those opinionated decisions that django is so famous for (which > most of time is really brilliant). But no, JKM never has any interest > in this because either he doesn't work in an enterprise environment > anymore (because he's too successful unlike the rest of us still > working in an Office Space envrionement) or django had seen tremendous > growth even without it, why bother. (what i mean to say here is JKM, > please post a note on the main site and mobilize your goons to address > this issue. I am pretty sure it would be done before i go to work > tomorrow). > > I think this not only mischaracterizes Jacob's leadership of the project, it also mischaracterizes the relationship between Jacob and the other core developers. Further, it does so in a somewhat unpleasant way which I hope you will retract, as none of the core developer I've come across could be described as "Jacob's goons". We don't need this unpleasantness on a list which is intended to discuss the future of a large software framework, and it can serve no useful purpose. > Finally, don't get me wrong, i love django and python. It is simply > the best language and best web framework out there for me (where > enterprise reporting fit beautifully into a micro-app type of > architecture instead of pylons or turbogears full blown application > architecture for each report). But the orm really drives me crazy. Of > course we all wrote custom managers, and overriding save to deal with > the composite key short comings, but come on, there's gotta be some > more pythonic (battery-included, there's only 1 obvious way) way of > dealing with this. > > Well, you've certainly expanded on your frustration. To say that the required changes could be made in less than 24 hours is a breathtaking oversimplification, however. If you believe it, then roll your own sleeves up and you could surely have it done by the end of the week. Open source, remember? This means that leadership is a bit like herding cats. I think you just took a good shot at
Re: Proposal: Composite Foreign Keys
Hi Frank -- It's hard for me to figure out how to answer this: if you've got a problem with my leadership skills, I don't really see how anything I say makes much of a difference. Frankly, your tone is completely inappropriate and I feel I'm enforcing absurdly out-of-line behavior simply by responding. However, I'm just going to ignore those parts of your post and focus on the real question. Support for composite keys has indeed been requested before. In fact, it's ticket #373; opened about three years ago! On July 20th, 2006, I commented: """ [He]re are the issues [...] that would need to be solved to make this work: There's three basic problems in dealing with composite primary keys in Django. The first is that a number of APIs use "obj._meta.pk" to access the primary key field (for example, to do "pk=whatever" lookups). A composite PK implementation would need to emulate this in some way to avoid breaking everything. Second, a number of things use (content_type_id, object_pk) tuples to refer to some object -- look at the comment framework, or the admin log API. Again, a composite PK system would need to somehow not break this. Finally, there's the issue of admin URLs; they're of the form "/app_label/module_name/pk/"; there would need to be a way to map URLs to objects in the absence of a primary key. """ (http://code.djangoproject.com/ticket/373#comment:3) That's a pretty clear list, and it's been sitting out there for over two years. I've pointed folks at that comment any number of times since then, and at some point someone expanded somewhat into a wiki page (http://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) that could serve as a simple spec. And yet three years on I've yet to see a patch appear on #373. Yes, this gets asked for time and time again, but nobody seems to want it enough to write even a *partial* fix. Why should this be? I think the main reason is that the lack of the feature is quite easy to work around in most cases. So most people who could fix it just don't feel like it's worth the time and move on. Somehow, despite the strum und drang there isn't really enough energy here to prompt anyone to work up a patch. Patches are the unit of currency in this community. With very few exceptions, every one of Django's thousands of commits began life as a patch posted by someone in the community. We committers can be a lot more effective when we review and integrate other peoples' patches — I can review a dozen patches from a dozen different people in the time it takes me to fix one bug on my own — so by necessity we have to rely on our community. If there's a feature you need, implement it. If you can't figure out where to start, ask — I'm on #django-dev during most of the work week, and I'd be happy to help anyone who wants to hack on this feature. If you don't want to or can't implement it yourself, there's a legion of options available ranging from asking around for help to organizing a team to contracting someone qualified. Finally, please keep in mind that the feature list we're drafting right now isn't set it stone. Anything that gets finished between now and the feature freeze date for 1.1 (2/15/09) is a candidate for inclusion. We develop these feature lists to help people figure out what to work on; nobody's gonna tell anyone not to scratch their own itch — that's what open source is all about. Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
> Thanks, I eventually figured this out on my own -- it was the > problem. Fixed in r9473 and r9474. > I wish I had known about MANIFEST.in sooner, is there any way we re- tag and re-release 1.0.1 with the missing files? Or am I forced to instruct GeoDjango users to avoid easy_install until 1.0.2? If the latter is the only option, I propose we set up a 1.0.2 timeline ASAP. -Justin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
Also keep in mind that work has been done, but as I haven't had a lot of spare time to continue it (attempt #3?). It's a very complex problem as well, like Jacob mentioned. You have to deal w/ the legacy use of single primary keys, as well as the new composites. While I almost have a fully functioning version (barring ForeignKey support) it's still going to take a little bit. On Sun, Nov 16, 2008 at 4:19 PM, Jacob Kaplan-Moss < [EMAIL PROTECTED]> wrote: > > Hi Frank -- > > It's hard for me to figure out how to answer this: if you've got a > problem with my leadership skills, I don't really see how anything I > say makes much of a difference. Frankly, your tone is completely > inappropriate and I feel I'm enforcing absurdly out-of-line behavior > simply by responding. > > However, I'm just going to ignore those parts of your post and focus > on the real question. > > Support for composite keys has indeed been requested before. In fact, > it's ticket #373; opened about three years ago! On July 20th, 2006, I > commented: > > """ > [He]re are the issues [...] that would need to be solved to make this work: > > There's three basic problems in dealing with composite primary keys in > Django. > > The first is that a number of APIs use "obj._meta.pk" to access the > primary key field (for example, to do "pk=whatever" lookups). A > composite PK implementation would need to emulate this in some way to > avoid breaking everything. > > Second, a number of things use (content_type_id, object_pk) tuples to > refer to some object -- look at the comment framework, or the admin > log API. Again, a composite PK system would need to somehow not break > this. > > Finally, there's the issue of admin URLs; they're of the form > "/app_label/module_name/pk/"; there would need to be a way to map URLs > to objects in the absence of a primary key. > """ > > (http://code.djangoproject.com/ticket/373#comment:3) > > That's a pretty clear list, and it's been sitting out there for over > two years. I've pointed folks at that comment any number of times > since then, and at some point someone expanded somewhat into a wiki > page (http://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) > that could serve as a simple spec. > > And yet three years on I've yet to see a patch appear on #373. Yes, > this gets asked for time and time again, but nobody seems to want it > enough to write even a *partial* fix. Why should this be? > > I think the main reason is that the lack of the feature is quite easy > to work around in most cases. So most people who could fix it just > don't feel like it's worth the time and move on. Somehow, despite the > strum und drang there isn't really enough energy here to prompt anyone > to work up a patch. > > Patches are the unit of currency in this community. With very few > exceptions, every one of Django's thousands of commits began life as a > patch posted by someone in the community. We committers can be a lot > more effective when we review and integrate other peoples' patches — I > can review a dozen patches from a dozen different people in the time > it takes me to fix one bug on my own — so by necessity we have to rely > on our community. > > If there's a feature you need, implement it. If you can't figure out > where to start, ask — I'm on #django-dev during most of the work week, > and I'd be happy to help anyone who wants to hack on this feature. If > you don't want to or can't implement it yourself, there's a legion of > options available ranging from asking around for help to organizing a > team to contracting someone qualified. > > Finally, please keep in mind that the feature list we're drafting > right now isn't set it stone. Anything that gets finished between now > and the feature freeze date for 1.1 (2/15/09) is a candidate for > inclusion. We develop these feature lists to help people figure out > what to work on; nobody's gonna tell anyone not to scratch their own > itch — that's what open source is all about. > > Jacob > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
On Sun, Nov 16, 2008 at 5:19 PM, David Cramer <[EMAIL PROTECTED]> wrote: > Also keep in mind that work has been done, but as I haven't had a lot of > spare time to continue it (attempt #3?). It's a very complex problem as > well, like Jacob mentioned. You have to deal w/ the legacy use of single > primary keys, as well as the new composites. While I almost have a fully > functioning version (barring ForeignKey support) it's still going to take a > little bit. If you'll share your code — #373 is begging for a patch... — others can use it as a starting place. Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
On Sun, Nov 16, 2008 at 6:07 PM, Justin Bronn <[EMAIL PROTECTED]> wrote: > > > Thanks, I eventually figured this out on my own -- it was the > > problem. Fixed in r9473 and r9474. > > > > I wish I had known about MANIFEST.in sooner, is there any way we re- > tag and re-release 1.0.1 with the missing files? Or am I forced to > instruct GeoDjango users to avoid easy_install until 1.0.2? If the > latter is the only option, I propose we set up a 1.0.2 timeline ASAP. > I didn't know MANIFEST.in needed data directories listedI know from bug fixing in setup.py that it generates a list of data files to pass into setup -- I wonder what that is used for? I guess it's for install but not distribution tarfile building? I do think we need to get a fixed release tarball out ASAP, though I don't know what's best. I don't think a 2nd tarball with the 1.0.1 name is an option -- the whole point of having a label like 1.0.1 is so you know what you've got. Two different 1.0.1 files with different contents defeats the purpose of that. So I think the next file has to have a 1.0.2 name, and then the question is what does it contain? One option is exactly what was in 1.0.1 plus the MANIFEST.in change, plus the version file(s?) changes that are necessary to change what version is reported. I'm not sure what all SVN gyrations are necessary to achieve that, but I'm sure it's possible. (Branch from the 1.0.1 rev, apply the necessary fixes to this branch, tag that as 1.0.2?) The other alternative is building 1.0.2 from 'traditionally' from the current 1.0.X branch contents. So 1.0.2 would have a handful of fixes in addition to the missing gis files. The fixes that have gone in are small & minor and I don't see any reason whey any of them couldn't go out in a release immediately. If this is the route taken, I'd suggest #9612 be added as well -- it's a translation update that fixes the date format translation strings which were misunderstood during first translation. Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
On Sun, Nov 16, 2008 at 5:33 PM, Karen Tracey <[EMAIL PROTECTED]> wrote: > The other alternative is building 1.0.2 from 'traditionally' from the > current 1.0.X branch contents. So 1.0.2 would have a handful of fixes in > addition to the missing gis files. The fixes that have gone in are small & > minor and I don't see any reason whey any of them couldn't go out in a > release immediately. If this is the route taken, I'd suggest #9612 be added > as well -- it's a translation update that fixes the date format translation > strings which were misunderstood during first translation. I'll defer to James to make the final call, but I'd prefer to do this and release 1.0.2 on Monday or Tuesday with a note that it's basically 1.0.1 plus the GeoDjango stuff we forgot and a couple of new bug fixes that got slipped in. If you look around you'll see other projects do the same all the time; I see no reason to be different. Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
http://github.com/dcramer/django-compositepks/tree/master# On Nov 16, 5:25 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote: > On Sun, Nov 16, 2008 at 5:19 PM, David Cramer <[EMAIL PROTECTED]> wrote: > > Also keep in mind that work has been done, but as I haven't had a lot of > > spare time to continue it (attempt #3?). It's a very complex problem as > > well, like Jacob mentioned. You have to deal w/ the legacy use of single > > primary keys, as well as the new composites. While I almost have a fully > > functioning version (barring ForeignKey support) it's still going to take a > > little bit. > > If you'll share your code — #373 is begging for a patch... — others > can use it as a starting place. > > Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
Hi Jacob, Let me apologize first for using JKM as an alias to the whole core team. I guess the problem here is that adding this feature really entail changes in too many areas. It would be nice if you and the other core team members can set a direction for changes on this issue. The various patches suggested in those previous threads all seem to fix one problem while giving rise to another somewhere else. So having a direction that fundamentally address this issue i think would be a better approach. On the other hand, i have to say that I am not too familiar with this open source style of working on a project. At my company, issues like this that span across multiple sub sections usually can only be resolved by management stepping in. From what I've read here in the user/dev mailing list, it seems that the only reason that this can't be resolved is due to lack of leadership from above. (Totalitarian states usually are pretty efficient in execution...military, another example...perhaps sqlalchemy could be a third example with MB being the strong leader). Anyway, i guess by seeing your email I've already achieved quite a bit. Simply having you recognizing such a problem again after 2 years i think it's already going somewhere. Perhaps in a few days we will see some patches coming in. I am glad to see that you are willing to help. I will try to contribute if i can although i don't do this by trade. Oh yeah, as for the tone...I was pretty disappointed not to see this proposal being included in your list earlier today (it's like Santa had forgotten you the day of christmas) and it somehow seemed to me that this list is a final list for 1.1 release. Plus, if you had a chance to look at the type of databases that I have to deal with by day, you will see why this is so important to me. Composite primary is the only non-issue issue at work; you gotta see the kind of lack of constraints, primary key, proper privilege, tests/ verification/process, bizarre division of schemas and all the other database craziness at my work, you would understand my frustration. The django orm + admin/admindocs, with all these beautiful scaffolding/ self-documenting/constraints generating/best practice imposing goodness, would be a great way to rectify some of that craziness. But no, it's not too usable because it doesn't handle composite pk (ok, hacking it would make some parts available--it is working for me right now, but imagine what a Lumbergh-from Office Space- type of manager would say...and imagine what all the Java IT guys would say). And finally, i don't think my frustration is particular to my company/ group, everyone i know to who work in a similar enterprise environment in the silicone valley seems to have the exact same problems (unless your company is a hip facebook/web app development company). Leaving people like us behind I think is a huge missed opportunity for django. Frank On Nov 16, 2:19 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote: > Hi Frank -- > > It's hard for me to figure out how to answer this: if you've got a > problem with my leadership skills, I don't really see how anything I > say makes much of a difference. Frankly, your tone is completely > inappropriate and I feel I'm enforcing absurdly out-of-line behavior > simply by responding. > > However, I'm just going to ignore those parts of your post and focus > on the real question. > > Support for composite keys has indeed been requested before. In fact, > it's ticket #373; opened about three years ago! On July 20th, 2006, I > commented: > > """ > [He]re are the issues [...] that would need to be solved to make this work: > > There's three basic problems in dealing with composite primary keys in Django. > > The first is that a number of APIs use "obj._meta.pk" to access the > primary key field (for example, to do "pk=whatever" lookups). A > composite PK implementation would need to emulate this in some way to > avoid breaking everything. > > Second, a number of things use (content_type_id, object_pk) tuples to > refer to some object -- look at the comment framework, or the admin > log API. Again, a composite PK system would need to somehow not break > this. > > Finally, there's the issue of admin URLs; they're of the form > "/app_label/module_name/pk/"; there would need to be a way to map URLs > to objects in the absence of a primary key. > """ > > (http://code.djangoproject.com/ticket/373#comment:3) > > That's a pretty clear list, and it's been sitting out there for over > two years. I've pointed folks at that comment any number of times > since then, and at some point someone expanded somewhat into a wiki > page (http://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) > that could serve as a simple spec. > > And yet three years on I've yet to see a patch appear on #373. Yes, > this gets asked for time and time again, but nobody seems to want it > enough to write even a *partial* fix. Why should this be? > > I think the main reason is t
Re: Proposal: Composite Foreign Keys
So far it allows declaration and creation of primary key models. You
declare them in class Meta: primary_key = ('field', 'field', 'field').
There is no ForeignKey/Relational? handlers as of right now, but the
admin is mostly working.
Alex Gaynor pointed out there is one unrelated change I accidently
committed. The list_select_related changes I proposed.
There is also an issue with URL resolving in the admin (its a 1.1
proposal) so it's using a , vs a / for the delimiter, but it's a
simply setting change.
On Nov 16, 5:50 pm, David Cramer <[EMAIL PROTECTED]> wrote:
> http://github.com/dcramer/django-compositepks/tree/master#
>
> On Nov 16, 5:25 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]>
> wrote:
>
> > On Sun, Nov 16, 2008 at 5:19 PM, David Cramer <[EMAIL PROTECTED]> wrote:
> > > Also keep in mind that work has been done, but as I haven't had a lot of
> > > spare time to continue it (attempt #3?). It's a very complex problem as
> > > well, like Jacob mentioned. You have to deal w/ the legacy use of single
> > > primary keys, as well as the new composites. While I almost have a fully
> > > functioning version (barring ForeignKey support) it's still going to take
> > > a
> > > little bit.
>
> > If you'll share your code — #373 is begging for a patch... — others
> > can use it as a starting place.
>
> > Jacob
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
On Sun, Nov 16, 2008 at 5:50 PM, David Cramer <[EMAIL PROTECTED]> wrote: > http://github.com/dcramer/django-compositepks/tree/master# Cool; I'll take a look tomorrow. Do you want me to make changes in my own tree, or would you rather patches back to you? Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
> I didn't know MANIFEST.in needed data directories listedI know from bug > fixing in setup.py that it generates a list of data files to pass into setup > -- I wonder what that is used for? I guess it's for install but not > distribution tarfile building? Neither did I -- and those files _are_ in the `data_files` list used by setup.py :) From what I gather, if the files aren't in MANIFEST.in and aren't Python source files then they'll be pruned. [1] > I don't think a 2nd tarball with the 1.0.1 name is an option -- the whole > point of having a label like 1.0.1 is so you know what you've got. Two > different 1.0.1 files with different contents defeats the purpose of that. Agreed. My only worry was that there would be a significant period of time between 1.0.1 and 1.0.2 (like the time between 1.0 and 1.0.1). If the timeline is for next week (like Jacob suggested) rather than two months from now, it's a moot point and I'm +1 for fixing this issue in a 1.0.2 release. -Justin [1] http://www.python.org/doc/2.5.2/dist/manifest.html --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
I think forking it on github would be the best. I really like how easy it makes things. There's still quite a bit that will need done, mostly with code cleanup, and tests, but I think it will give everyone a pretty good start. If you'd like I can give you just direct commit access to the branch as well. On Sun, Nov 16, 2008 at 6:00 PM, Jacob Kaplan-Moss < [EMAIL PROTECTED]> wrote: > > On Sun, Nov 16, 2008 at 5:50 PM, David Cramer <[EMAIL PROTECTED]> wrote: > > http://github.com/dcramer/django-compositepks/tree/master# > > Cool; I'll take a look tomorrow. Do you want me to make changes in my > own tree, or would you rather patches back to you? > > Jacob > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
On Sun, Nov 16, 2008 at 5:42 PM, Jacob Kaplan-Moss <[EMAIL PROTECTED]> wrote: > I'll defer to James to make the final call, but I'd prefer to do this > and release 1.0.2 on Monday or Tuesday with a note that it's basically > 1.0.1 plus the GeoDjango stuff we forgot and a couple of new bug fixes > that got slipped in. If you look around you'll see other projects do > the same all the time; I see no reason to be different. That sounds like the best plan. Let's aim for Tuesday (Monday I've got to babysit some site launches), and add a couple caveats to contributing instructions for patch authors and committers reminding them to run 'setup.py sdist' and check that the package includes any new directories they've added (if not, patch should be amended to include changes to setup.py or the manifest). -- "Bureaucrat Conrad, you are technically correct -- the best kind of correct." --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
On Sun, 2008-11-16 at 15:55 -0800, Frank Liu wrote: > Hi Jacob, > > Let me apologize first for using JKM as an alias to the whole core > team. I guess the problem here is that adding this feature really > entail changes in too many areas. It would be nice if you and the > other core team members can set a direction for changes on this issue. > The various patches suggested in those previous threads all seem to > fix one problem while giving rise to another somewhere else. So having > a direction that fundamentally address this issue i think would be a > better approach. > I'm sorry, but you're essentially 100% incorrect about the maintainership techniques and a large number of your other claims about the ORM. The changes required for composite primary keys are relatively localised. There's more than one place, but there's a limited number of them. For example, since a lot of the logic is already encapsulated by having a "pk" property already, it's mostly those places that need attention. Similarly, despite your beliefs, all the maintainers have have been setting directions for this work in the sense of encouraging, guiding and providing regular detailed feedback, as the most casual regular observer on this mailing list could see. But it's not code production by fiat in the open source world. We aren't going to tell people what to work on, but we do keep an eye on things and also have our own interests that we push along. For a couple of years, there's been fairly coordinated and steady activity towards improvement in various areas, including the ORM. They remain ongoing. Again, as with Jacob, I've been reluctant to respond to this thread because you're just so out of line with your criticisms and the ad hominem attacks are fairly insulting (and since you mentioned me by name and by implication in your attacks on Jacob, I am definitely going to take offence at your mischaracterisations). You happen to want composite primary keys for your particular situation. But in the whole broad scheme of every possible web application and database usage, they just aren't the most common use-case in the world, so will happen in due course when they're done. Yes, they're on the table; have been for quite a while. Which is why they're always part of the equation in ORM design work. The latest round of large ORM changes last year used this case as one influence over quite a lot of the design, contrary to your observations (and I can speak with some authority to that point). If this was/is so blindingly important to you, where's your patch? You are the one with the apparently show stopping need for this and you get paid at least 10 times as much as me to work on Django (even if you get paid $0), so if it's so important to your job, put up or shut up. You are welcome to use Django. We are glad you do. But that doesn't mean you get to post crap like this when your favorite feature isn't yet implemented. > On the other hand, i have to say that I am not too familiar with this > open source style of working on a project. Clearly. > At my company, issues like > this that span across multiple sub sections usually can only be > resolved by management stepping in. One of the reasons why most corporate software is of much lower quality than Open Source software. > From what I've read here in the > user/dev mailing list, it seems that the only reason that this can't > be resolved is due to lack of leadership from above. Nonsense. You're mischaracterising the entire situation. It isn't "not resolved" (whatever definition of "resolved" you happen to want to choose). It's ongoing work that hasn't been finished yet. The bar for quality is high and there are other, simultaneous priorities and limited resources. > (Totalitarian > states usually are pretty efficient in execution...military, another > example...perhaps sqlalchemy could be a third example with MB being > the strong leader). Anyway, i guess by seeing your email I've already > achieved quite a bit. Simply having you recognizing such a problem > again after 2 years i think it's already going somewhere. Perhaps in a > few days we will see some patches coming in. Seriously, put down the weed and step away from the keyboard. Have you followed any of the discussion about this at all? David, myself and a number of others have been having ongoing discussions for a few months in the latest round. It's progressing fairly nicely, given the time we have available and all the other pieces we're all working on as well. As mentioned above, it's also been one of the many, many considerations when designing other pieces of the infrastructure and has been duly waiting in the queue for the appropriate moment to start making the changes (which is around now). The reason the ticket has been open is to indicate it's ongoing work and we haven't been unaware of it. It's just not done yet (see, again, priorities and lack of a complete patch that meets the quality standards). This w
Re: Django 1.1, app() and ticket #3591
>>> If the basic premise of an app class - instances of which can
>>> live in
>>> settings.INSTALLED_APPS - is acceptable (and, of course, this means
>>> instances of subclasses of app can live in settings.INSTALLED_APPS
>>> too) then the precise location of an implementation (e.g.
>>> django.core.apps) can be refined.
>>
>> I'd also love to hear something from the core devs on that.
>
> Yep, let's see what they say. Malcolm has already voted -1, so does
> this mean it's not going ahead for 1.1? According to Jacob's original
> post, this would be killed now. But Malcolm has not given any specific
> criticisms of the patch, and spoken only in generalities. And Jacob
> asked me in private mail to start this thread, to which he expects to
> contribute.
The two -1 from core devs veto the feature for the next version, not
the whole feature. We can go on discussing it here. I still hope they
chime in though :)
> Sure, and I'd be fine with putting app-related stuff in e.g.
> django.core.apps and importing those into settings.py. I see that as a
> minor refactoring issue. The placement of the app class in the conf
> system was pragmatic, to see if the idea would fly. No core dev ever
> commented on this patch, so it stayed as it was.
Indeed, my idea though is to dodge imports in settings.py and just use
dotted module names.
>> Having app instances in settings.py is in sharp contrast to how the
>> settings.py is used currently. Extensible features like
>> AUTHENTICATION_BACKENDS, FILE_UPLOAD_HANDLERS, MIDDLEWARE_CLASSES,
>> SESSION_ENGINE, TEMPLATE_CONTEXT_PROCESSORS and TEMPLATE_LOADERS all
>> use tuples containing paths to Python functions or classes -- just
>> like INSTALLED_APPS does with module paths. The AppCache should also
>> be able to take paths to app classes and create instances of them on
>> the fly while startup.
>
> It's a debatable point, and I'm happy to see discussion and comments
> from others. I tend to the pragmatic, so how (in more detail) would
> you propose tackling the specific use case I mentioned with clashing
> auth and comments apps? How would settings.py look in your ideal
> scenario?
The settings.py would contain a list of one of the following:
1. a dotted module name to a module representing an app, like
'django.contrib.admin'
2. a dotted module name to an App subclass representing an app, like
'tagging.app.TaggingApp'
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.admin',
'tagging.app.TaggingApp',
'timezones.app.TimezoneApp',
'satchmo',
)
This is how the AppCache is filled with App instances:
1. On startup the AppCache tries to import each of the entries in the
INSTALLED_APPS setting.
a. If the result is an App subclass: yay!
b. If the result is a module (like ): look in the 'app' submodule for App
subclass(es)
c. If neither was found use the base App class
2. Things like get_model and get_app would take the full dotted module
name instead of the app_label.
> An analogy with using app instances in settings.INSTALLED_APPS might
> arise from looking at how the use of either functions or paths to
> functions in allowed in urlconfs. I don't think anyone is insisting
> that urlconfs should contain only paths to functions - the utility of
> using functions there is well-established, I would say.
Ah I see, I see the analogy more with forms, models and the admin.
>> A strong paradigm in Django for extensible features is to inherit
>> from
>> base classes and/or implement a simple API and then adding it to the
>> settings. I don't see why this shouldn't also be done for apps.
>
> I don't see how my suggestion precludes that. I even gave an example
> with a derived app class.
Imports are required for no other extensible feature of Django --
while they are required to use derived app classes in your idea. I
think that has a simple reason: it keeps the implementations separate
and the settings clean of complex code. Something that is very helpful
for users, I think.
> The point is, I don't want to touch a third-party app's code if I
> don't need to. What if I want to use an existing app which doesn't
> care what its full path is? Can you give some code snippets like I
> did, to show how things should work, in your view?
Somewhere in your site specific code (preferable app.py) you would
create something this to override thing from 3rd party apps:
from tagging.app import TaggingApp
from django.utils.translation import gettext_lazy as _
class MyTaggingApp(TaggingApp):
db_prefix = 'my_tagging'
verbose_name = _('my tagging')
#..
In your settings.py you would add:
INSTALLED_APPS = (
# ..
'tralala.app.MyTaggingApp',
# ..
)
>>> However, if you want to have specific app classes, then you can
>>> import
>>> them into settings.py and have entries such as
>>
>>> from my.package import MyCustomAppSubclass
>>
>>> I
Re: ANN: Django 1.0.1 released
On Sun, 2008-11-16 at 17:42 -0600, Jacob Kaplan-Moss wrote: > On Sun, Nov 16, 2008 at 5:33 PM, Karen Tracey <[EMAIL PROTECTED]> wrote: > > The other alternative is building 1.0.2 from 'traditionally' from the > > current 1.0.X branch contents. So 1.0.2 would have a handful of fixes in > > addition to the missing gis files. The fixes that have gone in are small & > > minor and I don't see any reason whey any of them couldn't go out in a > > release immediately. If this is the route taken, I'd suggest #9612 be added > > as well -- it's a translation update that fixes the date format translation > > strings which were misunderstood during first translation. > > I'll defer to James to make the final call, but I'd prefer to do this > and release 1.0.2 on Monday or Tuesday with a note that it's basically > 1.0.1 plus the GeoDjango stuff we forgot and a couple of new bug fixes > that got slipped in. If you look around you'll see other projects do > the same all the time; I see no reason to be different. Yeah, I don't see a problem with doing a 1.0.2 like that. Point releases for non-security reasons are cheap. Recommend we do that. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
This: http://code.djangoproject.com/ticket/9613 seems to be of the same vein. I haven't confirmed it however. On Nov 16, 7:54 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Sun, 2008-11-16 at 17:42 -0600, Jacob Kaplan-Moss wrote: > > On Sun, Nov 16, 2008 at 5:33 PM, Karen Tracey <[EMAIL PROTECTED]> wrote: > > > The other alternative is building 1.0.2 from 'traditionally' from the > > > current 1.0.X branch contents. So 1.0.2 would have a handful of fixes in > > > addition to the missing gis files. The fixes that have gone in are small > > > & > > > minor and I don't see any reason whey any of them couldn't go out in a > > > release immediately. If this is the route taken, I'd suggest #9612 be > > > added > > > as well -- it's a translation update that fixes the date format > > > translation > > > strings which were misunderstood during first translation. > > > I'll defer to James to make the final call, but I'd prefer to do this > > and release 1.0.2 on Monday or Tuesday with a note that it's basically > > 1.0.1 plus the GeoDjango stuff we forgot and a couple of new bug fixes > > that got slipped in. If you look around you'll see other projects do > > the same all the time; I see no reason to be different. > > Yeah, I don't see a problem with doing a 1.0.2 like that. Point releases > for non-security reasons are cheap. Recommend we do that. > > Regards, > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django 1.1, app() and ticket #3591
On Mon, 2008-11-17 at 01:50 +0100, Jannis Leidel wrote: > >>> If the basic premise of an app class - instances of which can > >>> live in > >>> settings.INSTALLED_APPS - is acceptable (and, of course, this means > >>> instances of subclasses of app can live in settings.INSTALLED_APPS > >>> too) then the precise location of an implementation (e.g. > >>> django.core.apps) can be refined. > >> > >> I'd also love to hear something from the core devs on that. > > > > Yep, let's see what they say. Malcolm has already voted -1, so does > > this mean it's not going ahead for 1.1? According to Jacob's original > > post, this would be killed now. But Malcolm has not given any specific > > criticisms of the patch, My -1 is because of basically the same thing Jannis has pointed out (and as I mentioned in my comment). There's a big ticket with various proposals and at some point last year Adrian mentioned he had another idea and that led to a group discussion at PyCon this year which raised a bunch of issues to be resolved. I don't feel there's a great consensus about the approach yet. This isn't something that should be half-baked, so if there's still unresolved items with no buy-in from any maintainer (and I haven't seen that yet), then I'm not going to be in favour. I'm examining the concept as a whole, not the lines of code at the moment. Remember that "no, right now" doesn't mean "no, forever". Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Notification on new database connections (ticket #6064)
Hi all, I've been playing with adding sqlite3 back-end support to GeoDjango, using the SpatiaLite extension. This requires executing some magic SQL each time you connect to the database, to enable the spatial extensions. Ticket #6064 seems like this right way to do this, by causing the database to send a signal each time a new connection is opened, which GeoDjango can catch. This works like a charm, and I just uploaded an updated patch to that ticket, since it hadn't been touched for a year and had gotten quite stale. I'm curious if anyone thinks there's a better way to do this (i.e. cause magic SQL to be executed on each database connection). I'm also curious whether the test in my patch passes for other back-ends, like oracle and whatnot. Finally, it occurred to me that a signal on *cursor* creation could also be useful, and could easily be added at the same time. (In fact in the old patch the connection_created signal actually erroneously behaved like a cursor_created signal for some back-ends.) Thoughts? Matt Matt Hancher Intelligent Systems Division NASA Ames Research Center [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
[PATCH] Proposal: connection pooling with psycopg2
PsycoPg2 with a remote DB over a WAN is deadly slow. Since PsycoPg2
supports connection pooling, I have decided to patch the Django DB
adapter to use this. I could see this implemented as below (this is a
very rough draft). For example, min and max connections should not be
hardcoded.
Let me know what you think,
Jonathan
diff -uprN /usr/share/python-support/python-django/django/db/backends/
postgresql_psycopg2/base.py psycopg2_pool/base.py
--- /usr/share/python-support/python-django/django/db/backends/
postgresql_psycopg2/base.py 2008-08-28 02:49:00.0 -0400
+++ psycopg2_pool/base.py 2008-11-16 14:51:02.465706723 -0500
@@ -15,6 +15,7 @@ from django.utils.safestring import Safe
try:
import psycopg2 as Database
import psycopg2.extensions
+from psycopg2.pool import PersistentConnectionPool as PoolClass
except ImportError, e:
from django.core.exceptions import ImproperlyConfigured
raise ImproperlyConfigured("Error loading psycopg2 module: %s" %
e)
@@ -25,6 +26,23 @@ IntegrityError = Database.IntegrityError
psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
psycopg2.extensions.register_adapter(SafeString,
psycopg2.extensions.QuotedString)
psycopg2.extensions.register_adapter(SafeUnicode,
psycopg2.extensions.QuotedString)
+pools = {}
+
+def getconn(conn_str, opts):
+try:
+pool = pools[conn_str];
+except KeyError:
+pool = pools[conn_str] = PoolClass(3, 5, conn_str)
+
+return pool.getconn()
+
+def putconn(conn_str, conn):
+try:
+pool = pools[conn_str];
+except KeyError:
+return
+
+return pool.putconn(conn)
class DatabaseFeatures(BaseDatabaseFeatures):
needs_datetime_string_cast = False
@@ -72,16 +90,16 @@ class DatabaseWrapper(BaseDatabaseWrappe
if settings.DATABASE_NAME == '':
from django.core.exceptions import
ImproperlyConfigured
raise ImproperlyConfigured("You need to specify
DATABASE_NAME in your Django settings file.")
-conn_string = "dbname=%s" % settings.DATABASE_NAME
+self.conn_string = "dbname=%s" % settings.DATABASE_NAME
if settings.DATABASE_USER:
-conn_string = "user=%s %s" % (settings.DATABASE_USER,
conn_string)
+self.conn_string = "user=%s %s" %
(settings.DATABASE_USER, self.conn_string)
if settings.DATABASE_PASSWORD:
-conn_string += " password='%s'" %
settings.DATABASE_PASSWORD
+self.conn_string += " password='%s'" %
settings.DATABASE_PASSWORD
if settings.DATABASE_HOST:
-conn_string += " host=%s" % settings.DATABASE_HOST
+self.conn_string += " host=%s" %
settings.DATABASE_HOST
if settings.DATABASE_PORT:
-conn_string += " port=%s" % settings.DATABASE_PORT
-self.connection = Database.connect(conn_string,
**self.options)
+self.conn_string += " port=%s" %
settings.DATABASE_PORT
+self.connection = getconn(self.conn_string, self.options)
self.connection.set_isolation_level(1) # make
transactions transparent to all cursors
self.connection.set_client_encoding('UTF8')
cursor = self.connection.cursor()
@@ -94,3 +112,8 @@ class DatabaseWrapper(BaseDatabaseWrappe
# No savepoint support for earlier version of
PostgreSQL.
self.features.uses_savepoints = False
return cursor
+
+def close(self):
+if self.connection is not None:
+putconn(self.conn_string, self.connection)
+self.connection = None
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Proposal: connection pooling with psycopg2
This sounds like a good feature, however, it's very difficult to read
a patch in either email or on google groups, so could you please open
a ticket in trac, and post this as a patch there.
On Nov 16, 2:57 pm, jothan <[EMAIL PROTECTED]> wrote:
> PsycoPg2 with a remote DB over a WAN is deadly slow. Since PsycoPg2
> supports connection pooling, I have decided to patch the Django DB
> adapter to use this. I could see this implemented as below (this is a
> very rough draft). For example, min and max connections should not be
> hardcoded.
>
> Let me know what you think,
> Jonathan
>
> diff -uprN /usr/share/python-support/python-django/django/db/backends/
> postgresql_psycopg2/base.py psycopg2_pool/base.py
> --- /usr/share/python-support/python-django/django/db/backends/
> postgresql_psycopg2/base.py 2008-08-28 02:49:00.0 -0400
> +++ psycopg2_pool/base.py 2008-11-16 14:51:02.465706723 -0500
> @@ -15,6 +15,7 @@ from django.utils.safestring import Safe
> try:
> import psycopg2 as Database
> import psycopg2.extensions
> + from psycopg2.pool import PersistentConnectionPool as PoolClass
> except ImportError, e:
> from django.core.exceptions import ImproperlyConfigured
> raise ImproperlyConfigured("Error loading psycopg2 module: %s" %
> e)
> @@ -25,6 +26,23 @@ IntegrityError = Database.IntegrityError
> psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
> psycopg2.extensions.register_adapter(SafeString,
> psycopg2.extensions.QuotedString)
> psycopg2.extensions.register_adapter(SafeUnicode,
> psycopg2.extensions.QuotedString)
> +pools = {}
> +
> +def getconn(conn_str, opts):
> + try:
> + pool = pools[conn_str];
> + except KeyError:
> + pool = pools[conn_str] = PoolClass(3, 5, conn_str)
> +
> + return pool.getconn()
> +
> +def putconn(conn_str, conn):
> + try:
> + pool = pools[conn_str];
> + except KeyError:
> + return
> +
> + return pool.putconn(conn)
>
> class DatabaseFeatures(BaseDatabaseFeatures):
> needs_datetime_string_cast = False
> @@ -72,16 +90,16 @@ class DatabaseWrapper(BaseDatabaseWrappe
> if settings.DATABASE_NAME == '':
> from django.core.exceptions import
> ImproperlyConfigured
> raise ImproperlyConfigured("You need to specify
> DATABASE_NAME in your Django settings file.")
> - conn_string = "dbname=%s" % settings.DATABASE_NAME
> + self.conn_string = "dbname=%s" % settings.DATABASE_NAME
> if settings.DATABASE_USER:
> - conn_string = "user=%s %s" % (settings.DATABASE_USER,
> conn_string)
> + self.conn_string = "user=%s %s" %
> (settings.DATABASE_USER, self.conn_string)
> if settings.DATABASE_PASSWORD:
> - conn_string += " password='%s'" %
> settings.DATABASE_PASSWORD
> + self.conn_string += " password='%s'" %
> settings.DATABASE_PASSWORD
> if settings.DATABASE_HOST:
> - conn_string += " host=%s" % settings.DATABASE_HOST
> + self.conn_string += " host=%s" %
> settings.DATABASE_HOST
> if settings.DATABASE_PORT:
> - conn_string += " port=%s" % settings.DATABASE_PORT
> - self.connection = Database.connect(conn_string,
> **self.options)
> + self.conn_string += " port=%s" %
> settings.DATABASE_PORT
> + self.connection = getconn(self.conn_string, self.options)
> self.connection.set_isolation_level(1) # make
> transactions transparent to all cursors
> self.connection.set_client_encoding('UTF8')
> cursor = self.connection.cursor()
> @@ -94,3 +112,8 @@ class DatabaseWrapper(BaseDatabaseWrappe
> # No savepoint support for earlier version of
> PostgreSQL.
> self.features.uses_savepoints = False
> return cursor
> +
> + def close(self):
> + if self.connection is not None:
> + putconn(self.conn_string, self.connection)
> + self.connection = None
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Multi-Threaded Dev Server
I think I may have to actually agree that multi-threading is somewhat needed, since it does limit these kind of features from working. I, however, would deem it less a priority than most thing. If you can write a patch for it (or if one exists, I didn't look) though, I don't see any reason to not extend runserver to use it. On Nov 16, 6:45 am, Steve Holden <[EMAIL PROTECTED]> wrote: > Julian wrote: > > [...] I think some people want to use snippets like that [...] > > Wouldn't you agree that's a pretty feeble use case for something as > potentially disruptive as multi-threaded serving? Particularly when the > CherryPy alternative is so readily available. > > regards > Steve > -- > Steve Holden +1 571 484 6266 +1 800 494 3119 > Holden Web LLC http://www.holdenweb.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: [PATCH] Proposal: connection pooling with psycopg2
On Sun, 2008-11-16 at 11:57 -0800, jothan wrote:
> PsycoPg2 with a remote DB over a WAN is deadly slow. Since PsycoPg2
> supports connection pooling, I have decided to patch the Django DB
> adapter to use this. I could see this implemented as below (this is a
> very rough draft). For example, min and max connections should not be
> hardcoded.
>
> Let me know what you think,
You've made it non-optional, which isn't ideal. The connection pooling
in psycopg2 is purely manual, in-memory stuff that shares connections
within the same process (so, for multi-threaded operations). Once you
switch to multi process operations, it's time to graduate to the "real
stuff" and use, e.g., pgpool to the connection pooling. At that point,
the extra overhead from the psycopg2 classes is probably not helping.
A different approach would be to work out which hooks are really needed
to support this (i.e. where we might need to call a function in a
subclass where the subclass is the thing that supports psycopg2's
pooling, for example) and we can write those in. Then, as hinted, we can
put the pooling in a subclass and provide it that way (maybe).
A couple of comments on the patch inline below...
> diff -uprN /usr/share/python-support/python-django/django/db/backends/
> postgresql_psycopg2/base.py psycopg2_pool/base.py
> --- /usr/share/python-support/python-django/django/db/backends/
> postgresql_psycopg2/base.py 2008-08-28 02:49:00.0 -0400
> +++ psycopg2_pool/base.py 2008-11-16 14:51:02.465706723 -0500
> @@ -15,6 +15,7 @@ from django.utils.safestring import Safe
> try:
> import psycopg2 as Database
> import psycopg2.extensions
> +from psycopg2.pool import PersistentConnectionPool as PoolClass
> except ImportError, e:
> from django.core.exceptions import ImproperlyConfigured
> raise ImproperlyConfigured("Error loading psycopg2 module: %s" %
> e)
> @@ -25,6 +26,23 @@ IntegrityError = Database.IntegrityError
> psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
> psycopg2.extensions.register_adapter(SafeString,
> psycopg2.extensions.QuotedString)
> psycopg2.extensions.register_adapter(SafeUnicode,
> psycopg2.extensions.QuotedString)
> +pools = {}
> +
> +def getconn(conn_str, opts):
> +try:
> +pool = pools[conn_str];
> +except KeyError:
> +pool = pools[conn_str] = PoolClass(3, 5, conn_str)
> +
> +return pool.getconn()
> +
> +def putconn(conn_str, conn):
> +try:
> +pool = pools[conn_str];
> +except KeyError:
> +return
> +
> +return pool.putconn(conn)
>
> class DatabaseFeatures(BaseDatabaseFeatures):
> needs_datetime_string_cast = False
> @@ -72,16 +90,16 @@ class DatabaseWrapper(BaseDatabaseWrappe
> if settings.DATABASE_NAME == '':
> from django.core.exceptions import
> ImproperlyConfigured
> raise ImproperlyConfigured("You need to specify
> DATABASE_NAME in your Django settings file.")
> -conn_string = "dbname=%s" % settings.DATABASE_NAME
> +self.conn_string = "dbname=%s" % settings.DATABASE_NAME
You're making this sort of change all throughout this function, which
suggests there's an easier way: put a single line at the end that says
self.conn_string = conn_string
Less changes, shorter lines in the function and it's also slightly more
efficient (since local variable lookups are faster than having to
dereference the attribute all the time). All of which are extremely
minor, but it simultaneously helps cut down on the code churn.
> if settings.DATABASE_USER:
> -conn_string = "user=%s %s" % (settings.DATABASE_USER,
> conn_string)
> +self.conn_string = "user=%s %s" %
> (settings.DATABASE_USER, self.conn_string)
> if settings.DATABASE_PASSWORD:
> -conn_string += " password='%s'" %
> settings.DATABASE_PASSWORD
> +self.conn_string += " password='%s'" %
> settings.DATABASE_PASSWORD
> if settings.DATABASE_HOST:
> -conn_string += " host=%s" % settings.DATABASE_HOST
> +self.conn_string += " host=%s" %
> settings.DATABASE_HOST
> if settings.DATABASE_PORT:
> -conn_string += " port=%s" % settings.DATABASE_PORT
> -self.connection = Database.connect(conn_string,
> **self.options)
> +self.conn_string += " port=%s" %
> settings.DATABASE_PORT
> +self.connection = getconn(self.conn_string, self.options)
This would seem to be the place for a hook. We call, for example, a
"connect()" function on the current class which is normally aliased to
Database.connect, but can be overridden in the pooling subclass.
> self.connection.set_isolation_level(1) # make
> transactions transparent to all cursors
> self.connection.set_client_encoding('UTF8')
> cursor = self.connection.cursor()
> @@ -94,3 +112,8 @@ class Databa
Re: Notification on new database connections (ticket #6064)
On Sun, 2008-11-16 at 17:14 -0800, Matthew D. Hancher wrote: > Hi all, > > I've been playing with adding sqlite3 back-end support to GeoDjango, > using the SpatiaLite extension. This requires executing some magic > SQL each time you connect to the database, to enable the spatial > extensions. Ticket #6064 seems like this right way to do this, by > causing the database to send a signal each time a new connection is > opened, which GeoDjango can catch. This works like a charm, and I > just uploaded an updated patch to that ticket, since it hadn't been > touched for a year and had gotten quite stale. Modulo my question about signal overhead below, this seems like a decent approach. It's kind of the reason for signals existing. A random thought: is there any other information worth sending along with the signal? Right now, the receiver is told "a connection was created". Anything that's likely to vary and that could be useful as a trigger for other actions? I can't immediately think of anything, but I'll throw it out there in case I've overlooked something. > I'm curious if anyone thinks there's a better way to do this (i.e. > cause magic SQL to be executed on each database connection). I'm also > curious whether the test in my patch passes for other back-ends, like > oracle and whatnot. > > Finally, it occurred to me that a signal on *cursor* creation could > also be useful, and could easily be added at the same time. (In fact > in the old patch the connection_created signal actually erroneously > behaved like a cursor_created signal for some back-ends.) I'm not up to speed these days on the overhead for signal emission. We create a lot of database connections, for better or worse (as you no doubt realise, our connection management strategy is "one per request"). Is the impact noticeable for those doing nothing with the signals? I suspect it's not hard to test, but I'm going to be lazy and not do it myself (I suspect my laptop isn't quite production-quality hardware in any case). I'm generally in favour of the idea, though. Looks like a reasonable use for signals and your use-case seems like a typical situation that will need this. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Notification on new database connections (ticket #6064)
In terms of signal overhead, the most likely case here is probably no receivers, and in the signal refactor, that particular case is now hugely faster. On Nov 16, 8:55 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Sun, 2008-11-16 at 17:14 -0800, Matthew D. Hancher wrote: > > Hi all, > > > I've been playing with adding sqlite3 back-end support to GeoDjango, > > using the SpatiaLite extension. This requires executing some magic > > SQL each time you connect to the database, to enable the spatial > > extensions. Ticket #6064 seems like this right way to do this, by > > causing the database to send a signal each time a new connection is > > opened, which GeoDjango can catch. This works like a charm, and I > > just uploaded an updated patch to that ticket, since it hadn't been > > touched for a year and had gotten quite stale. > > Modulo my question about signal overhead below, this seems like a decent > approach. It's kind of the reason for signals existing. > > A random thought: is there any other information worth sending along > with the signal? Right now, the receiver is told "a connection was > created". Anything that's likely to vary and that could be useful as a > trigger for other actions? I can't immediately think of anything, but > I'll throw it out there in case I've overlooked something. > > > I'm curious if anyone thinks there's a better way to do this (i.e. > > cause magic SQL to be executed on each database connection). I'm also > > curious whether the test in my patch passes for other back-ends, like > > oracle and whatnot. > > > Finally, it occurred to me that a signal on *cursor* creation could > > also be useful, and could easily be added at the same time. (In fact > > in the old patch the connection_created signal actually erroneously > > behaved like a cursor_created signal for some back-ends.) > > I'm not up to speed these days on the overhead for signal emission. We > create a lot of database connections, for better or worse (as you no > doubt realise, our connection management strategy is "one per request"). > Is the impact noticeable for those doing nothing with the signals? I > suspect it's not hard to test, but I'm going to be lazy and not do it > myself (I suspect my laptop isn't quite production-quality hardware in > any case). > > I'm generally in favour of the idea, though. Looks like a reasonable use > for signals and your use-case seems like a typical situation that will > need this. > > Regards, > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Notification on new database connections (ticket #6064)
On Sun, 2008-11-16 at 18:11 -0800, [EMAIL PROTECTED] wrote: > In terms of signal overhead, the most likely case here is probably no > receivers, and in the signal refactor, that particular case is now > hugely faster. "Hugely" being, of course, a highly scientific measurement lending itself to accurate comparison against future changes. :-) Yes, I know it's faster now and I realise that's the common case. But what's the impact of adding one or two new signal calls per request. I suspect it's minimal, but I haven't measured it and this is the first time it's come up since the refactor, so it's probably time to know what we're agreeing to for each new signal. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Notification on new database connections (ticket #6064)
On Mon, Nov 17, 2008 at 11:20 AM, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > > On Sun, 2008-11-16 at 18:11 -0800, [EMAIL PROTECTED] wrote: >> In terms of signal overhead, the most likely case here is probably no >> receivers, and in the signal refactor, that particular case is now >> hugely faster. > > "Hugely" being, of course, a highly scientific measurement lending > itself to accurate comparison against future changes. :-) > > Yes, I know it's faster now and I realise that's the common case. But > what's the impact of adding one or two new signal calls per request. I > suspect it's minimal, but I haven't measured it and this is the first > time it's come up since the refactor, so it's probably time to know what > we're agreeing to for each new signal. This sort of benchmarking would be helpful to have kicking around. One of the features on the table for v1.1 is to add signals for activity on m2m operations. The biggest impediment to introducing these signals is the overhead (or, at least, the perception that there will be an overhead) associated with adding a signal for a common operation. I would like to see m2m signals introduced (overhead permitting), so I'll probably have to work up a set of benchmarks at some point so that we can have the debate over some concrete numbers rather than perceptions and speculation. If someone else were to save me the effort of having to write these benchmarks myself, I would buy them a lollipop* :-) Russ Magee %-) * Lollipop offer valid while stocks last. ISO standard size and flavor. Offer void where prohibited. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
> This:http://code.djangoproject.com/ticket/9613seems to be of the > same vein. I haven't confirmed it however. Yeah I confirmed it. I went through the contrib apps one-by-one to see if anything else was missing. In addition to auth's missing files, the templates for formtools are omitted. I've got a patch on #9613; what other places should we be looking for missing dirs? -Justin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: ANN: Django 1.0.1 released
On Sun, 2008-11-16 at 18:42 -0800, Justin Bronn wrote: > > This:http://code.djangoproject.com/ticket/9613seems to be of the > > same vein. I haven't confirmed it however. > > Yeah I confirmed it. I went through the contrib apps one-by-one to > see if anything else was missing. In addition to auth's missing > files, the templates for formtools are omitted. > > I've got a patch on #9613; what other places should we be looking for > missing dirs? I applied that patch and then diffed the filename listings between an sdist tarball and the raw tree. The only missing file of note is simplejson's LICENSE file, which we should be shipping. I'll add that and commit the change. Thanks for finding those other ones, Justin. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Notification on new database connections (ticket #6064)
Well, Jeremy Dunck was kind enough to do a benchmark against the old system: http://groups.google.com/group/django-developers/browse_thread/thread/815f76ad7e823cbf/b008a757fbdefa2b On Nov 16, 9:49 pm, "Russell Keith-Magee" <[EMAIL PROTECTED]> wrote: > On Mon, Nov 17, 2008 at 11:20 AM, Malcolm Tredinnick > > > > <[EMAIL PROTECTED]> wrote: > > > On Sun, 2008-11-16 at 18:11 -0800, [EMAIL PROTECTED] wrote: > >> In terms of signal overhead, the most likely case here is probably no > >> receivers, and in the signal refactor, that particular case is now > >> hugely faster. > > > "Hugely" being, of course, a highly scientific measurement lending > > itself to accurate comparison against future changes. :-) > > > Yes, I know it's faster now and I realise that's the common case. But > > what's the impact of adding one or two new signal calls per request. I > > suspect it's minimal, but I haven't measured it and this is the first > > time it's come up since the refactor, so it's probably time to know what > > we're agreeing to for each new signal. > > This sort of benchmarking would be helpful to have kicking around. One > of the features on the table for v1.1 is to add signals for activity > on m2m operations. The biggest impediment to introducing these signals > is the overhead (or, at least, the perception that there will be an > overhead) associated with adding a signal for a common operation. > > I would like to see m2m signals introduced (overhead permitting), so > I'll probably have to work up a set of benchmarks at some point so > that we can have the debate over some concrete numbers rather than > perceptions and speculation. If someone else were to save me the > effort of having to write these benchmarks myself, I would buy them a > lollipop* :-) > > Russ Magee %-) > > * Lollipop offer valid while stocks last. ISO standard size and > flavor. Offer void where prohibited. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Notification on new database connections (ticket #6064)
On Sun, 2008-11-16 at 19:21 -0800, [EMAIL PROTECTED] wrote: > Well, Jeremy Dunck was kind enough to do a benchmark against the old > system: > http://groups.google.com/group/django-developers/browse_thread/thread/815f76ad7e823cbf/b008a757fbdefa2b Since we're no longer using the old system, that doesn't tell us how much slower adding a new signal to the new system is. You seem to be missing the point: It's not a huge deal, but we might as well work out this impact now since, as Russell and I have both noted, it's the first time we're looking at adding a new signal and there are other cases on the table (I'm also in favour of the many-to-many case, for example). Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Notification on new database connections (ticket #6064)
On Mon, Nov 17, 2008 at 12:21 PM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > Well, Jeremy Dunck was kind enough to do a benchmark against the old > system: > http://groups.google.com/group/django-developers/browse_thread/thread/815f76ad7e823cbf/b008a757fbdefa2b I knew about these. While these are great for establishing the relative improvement offered by the new signals, they don't really address the question of absolute performance. For an signal with no listeners, the new signals may be 67% faster, but faster than what? I know this is a horribly nebulous question (like all benchmarking), and it's completely dependent on the speed of your machine and a million other factors. However, if we are going to start adding signals to very common operations (like m2m and opening connections), we need to know what sort of overhead we are adding in absolute terms. Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Notification on new database connections (ticket #6064)
Right, I understand the need for absolute numbers, I just meant that we don't need to start rewriting new benchmarks if Jeremy still has the benchmark script. Alex On Nov 16, 11:03 pm, "Russell Keith-Magee" <[EMAIL PROTECTED]> wrote: > On Mon, Nov 17, 2008 at 12:21 PM, [EMAIL PROTECTED] > > <[EMAIL PROTECTED]> wrote: > > > Well, Jeremy Dunck was kind enough to do a benchmark against the old > > system:http://groups.google.com/group/django-developers/browse_thread/thread... > > I knew about these. While these are great for establishing the > relative improvement offered by the new signals, they don't really > address the question of absolute performance. For an signal with no > listeners, the new signals may be 67% faster, but faster than what? > > I know this is a horribly nebulous question (like all benchmarking), > and it's completely dependent on the speed of your machine and a > million other factors. However, if we are going to start adding > signals to very common operations (like m2m and opening connections), > we need to know what sort of overhead we are adding in absolute terms. > > Yours, > Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
What is the big deal? I was on a gig recently where Django was used with Oracle. To support model partitioning for selected models, we munged the generated SQL for model creation to add composite keys and the partitioning logic. It was a little bit hairy, but once the model creation was correct, everything else "just worked". No server hacks or avoidance of the Django ORM was required to run the apps. Sadly the code was special cased for the specific form of partitioning used and therefore not suitable for release as open source, but the code was not exceptionally hard to write. So get to it. If it really is too tough for you, there are some hotshot Django consultants around who may have some free cycles on rare occasions and might be willing to help you. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
