Re: Buildbot failure
If buildbot is failing for you could I suggest hudson? Yeah i know it's java - but I've been using it for python CI for the last few months and it's been great. It's a snap to set up, (especially with buildout) and the unittest tracking over time is really good too. Ben 2009/10/27 Jacob Kaplan-Moss : > > On Mon, Oct 26, 2009 at 9:49 PM, Karen Tracey wrote: >> I don't know who maintains or has access to the buildbot machine, > > It's me, probably. Buildbot's in bad shape, currently; it's basically > another part-time job just keeping the damn thing running. > > I've more or less given up and am working on pony-build instead. Watch > this space. > > Jacob > > > > -- Regards, Ben Ford ben.for...@gmail.com +447540722690 Charles de Gaulle - "The better I get to know men, the more I find myself loving dogs." - http://www.brainyquote.com/quotes/authors/c/charles_de_gaulle.html --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Transactions
Hi All, I'm looking into the transaction module due to an app that I'm working on doing something very odd (and mostly indefensible) with long running transactions. It would seem that the docs are totally off base WRT to transactions - specifically the DISABLE_TRANSACTION_MANAGEMENT setting. I'm extremely confused looking through the code as I can find no reference to this setting anywhere in the code... ever. When magic removal branch was merged (at revision 2809) the TRANSACTIONS_MANAGED setting appears in global_settings.py, but I've followed back further than that. Transactions seem to have first appeared in http://code.djangoproject.com/changeset/2457 when the TRANSACTIONS_MANAGED setting and a load of other transaction related stuff was committed. Ticket http://code.djangoproject.com/ticket/2304 (from July 06!) refers to this so I'm posting here and reopening rather than opening another similar ticket. I would suggest the best fix being changing the transaction documentation to replace DISABLE_TRANSACTION_MANAGEMENT with TRANSACTIONS_MANAGED. My problem has come as the app isn't using the transaction middleware and therefore _rollback_on_exception (in django.db.__init__) is being fired by the got_request_exception signal, despite DISABLE_TRANSACTION_MANAGEMENT being set. As expected from reading the code when I replace it with TRANSACTIONS_MANAGED, the app behaves as per the documentation I'm really curious though -- can anyone remember far back enough to know where this came from? I haven't been able to find a single mention of DISABLE_TRANSACTION_MANAGEMENT anywhere in django's history! Cheers, Ben P.S. I'm fully aware that the way our app is working is causing our problems, and that it should probably should be changed forthwith - however it's being put into an existing infrastructure and there isn't the freedom at this point to change it. Even with this being the case, I think this could quite easily bite others, and it's pretty trivial to fix. -- Regards, Ben Ford ben.for...@gmail.com +447540722690 Joan Crawford - "I, Joan Crawford, I believe in the dollar. Everything I earn, I spend." - http://www.brainyquote.com/quotes/authors/j/joan_crawford.html --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Oracle inspectdb
Hi, I've been playing with an Oracle database at work today and after a bit of hacking I got inspectdb to work. I have to qualify this with the fact that I know the square root of very little about Oracle, and I'm pretty sure what I've done is a nasty hack at best. Could someone please clarify how the DATA_TYPES_REVERSE works, specifically how you 're supposed to get from get_table_description(table_name) to the numbers which are the keys in the dictionary? FYI what I ended up doing was having get_table_description return cursor.describe on the query "SELECT * FROM table_name WHERE ROWNUM > 1" and using cx_Oracle. (i.e. row[1]) as the keys in DATA_TYPES_REVERSE (like the mysql backend does it). One thing that I did find slightly confusing was that cx_Oracle only found 4 different types of fields when I wrote a little script to to iterate through the fields in all of my tables... This isn't an insignificant database (57 tables and 317 fields by my count) so it seems strange to have so few data types in the schema? Once again I'll qualify that with my lack of familiarity with Oracle (and database design in general) and the fact that I didn't have time to dig in a bit more thoroughly. Any information gratefully received :) Ben -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Oracle inspectdb
Hi Mathias, Yes I did come see the boulder-sprint branch, but I'd read in the wiki that insectdb wasn't fully working...? I'm also need to read my svn docs again so I can do a proper merge (I'm using the gis bracnh at the moment). I'd definately like to have a crack at getting it working to save myself the work of defining all of the relationships etc!! Thanks for the comments :) Ben On 26/03/07, Masida <[EMAIL PROTECTED]> wrote: > > > Hi Ben, > > Are you aware of the Boulder Oracle Sprint branch? > > http://code.djangoproject.com/browser/django/branches/boulder-oracle-sprint > > And more specifically, about the support for inspectdb it already has? > > http://code.djangoproject.com/browser/django/branches/boulder-oracle-sprint/django/db/backends/oracle/introspection.py > > If you are, I'm sorry for bothering you. But I want to make sure you > don't duplicate work ;-) > > Ciao, > - Matthias > > > On Mar 25, 5:53 pm, "Ben Ford" <[EMAIL PROTECTED]> wrote: > > Hi, > > I've been playing with an Oracle database at work today and after a bit > of > > hacking I got inspectdb to work. I have to qualify this with the fact > that I > > know the square root of very little about Oracle, and I'm pretty sure > what > > I've done is a nasty hack at best. Could someone please clarify how the > > DATA_TYPES_REVERSE works, specifically how you 're supposed to get from > > get_table_description(table_name) to the numbers which are the keys in > the > > dictionary? > > > > FYI what I ended up doing was having get_table_description return > > cursor.describe on the query "SELECT * FROM table_name WHERE ROWNUM > 1" > and > > using cx_Oracle. (i.e. row[1]) as the keys in > DATA_TYPES_REVERSE > > (like the mysql backend does it). One thing that I did find slightly > > confusing was that cx_Oracle only found 4 different types of fields when > I > > wrote a little script to to iterate through the fields in all of my > > tables... This isn't an insignificant database (57 tables and 317 fields > by > > my count) so it seems strange to have so few data types in the schema? > Once > > again I'll qualify that with my lack of familiarity with Oracle (and > > database design in general) and the fact that I didn't have time to dig > in a > > bit more thoroughly. > > > > Any information gratefully received :) > > Ben > > > > -- > > Regards, > > Ben Ford > > [EMAIL PROTECTED] > > +628111880346 > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +6281388434435 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Sorry...
Hi there, First off sorry for asking a 'non-django-dev' question here... I've been playing with a number of different branches recently (gis, boulder-oracle-sprint and now multiple-db-support). My working copy is the gis branch with a lot of the boulder-sprint stuff merged in (in fact come to think about it I think I actually achieved this with a switch operation); it works well for my requirements. The problem is that I now need to get at the functionality provided by multiple-db to make the most of the oracle stuff I've been doing for the last few days. I tried to merge the multiple-db-support branch onto my working copy, merging r3198:HEAD as 3198 was when the branch was created, but it failed and i ended up with a lot of <<<<< .working and similar dotted around the files. I'm a bit of a subversion newb and I'm not sure what I'm doing wrong. I'm guessing that I might be trying to merge over the top of a previous merge of multiple-db-support back to trunk..? if anyone could help me out I'd be eternally grateful! The command I used was: svn merge -r3198:HEAD http://code.djangoproject.com/svn/django/branches/multiple-db-support . #this done while I was in the django-gis directory (where I checked out the gis branch) Once again sorry for the OT-ness of my email. Ben -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Sorry...
Thanks very much guys, Wouldn't you know it I have a very good book on svn (the pragmatic one) but it's in New Zealand and I'm in Indonesia! Doh! A ray of hope might be that I don't think that muliple-db-support changed that many files so it might not to soul destroying to fix the merge. If someone who was involved in that effort could point me in the right direction, I'd be happy to help with any improvements or other stuff that needs doing..? I just had a quick look at SVK and it looks pretty promising, I will have to investigate a bit further assuming I get a second at some point in the next decade ;) Cheers again, Ben On 27/03/07, Robert Coup <[EMAIL PROTECTED]> wrote: > > > Russell Keith-Magee wrote: > > The branches are all independent, and depending on how active the > > branch is, a branch may not be up to date with the trunk, let alone > > other branches. If one branch were to cleanly merges into another, I > > would be extremely surprised - doubly so for the more complex branches > > like multi-db. > > > > That's not to say you can't get multi-db support into a checkout of > > the gis branch - just that you will need to manually resolve a lot of > > merges. Time for you to break out the Subversion manual :-) > > > Or look into SVK, which can potentially make this a lot easier since it > tracks changes across branches/repositories. It takes some time to get > your head around, but there are a number of reasonable tutorials and you > can't break *too* much :) > > http://svk.bestpractical.com/view/HomePage > > Rob :) > > -- > One Track Mind Ltd. > PO Box 1604, Shortland St, Auckland, New Zealand > Phone +64-9-966 0433 Mobile +64-21-572 632 > Web http://www.onetrackmind.co.nz > > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +6281388434435 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Oracle patch is ready
Hi guys,
Great work on the oracle stuff!! I've been using it with the multiple-db
branch for the last month or so and it's been great!
A couple of things I had to mess with (mainly to use introspect db):
- in django.core.management (line 842) I had to get my tables like this:
for table_name in [list_of_desired_table_names>]:
#introspection_module.get_table_list(cursor):
because my get_table_list in oracle.introspection looks like this:
def get_table_list(cursor):
"Returns a list of table names in the current database."
cursor.execute("SELECT TABLE_NAME FROM USER_TABLES")
return [row[0].upper() for row in cursor.fetchall()]
I know nothing about oracle so I don't really know what the function of
USER_TABLES is, but it wasn't getting me what I needed.
I can't think of any other major things I changed at right now, but suffice
to say my svn check out is looking pretty tatty at the moment (I'm also
using the gis branch) and if I can do anything to help getting this into
trunk I will :)
Regards,
Ben
On 20/04/07, Matt Boersma <[EMAIL PROTECTED]> wrote:
>
>
> The "boulder-oracle-sprint" branch is ready to come home.
>
> We made a wiki page describing the goals and status of the code (quick
> summary: Oracle works and we think it's done, unless you tell us
> otherwise). Attached to that page is a patch against rev 5036 of the
> trunk. See here:
> http://code.djangoproject.com/wiki/OracleBranch
>
> The diff itself is here:
>
> http://code.djangoproject.com/attachment/wiki/OracleBranch/django-oracle-rev5036.diff
>
> This is a "non-trivial patch," we believe, so rather than attaching it
> to the closed Trac bug #1990, we're advertising it here as recommended
> on the "Contributing to Django" page. There are some minor
> architectural changes in there, although we've made every effort to
> minimize the overall deltas.
>
> Please review the patch if you're able to and let us know what you
> think.
>
>
> >
>
--
Regards,
Ben Ford
[EMAIL PROTECTED]
+628111880346
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Disable admin pagination completely: discussion for ticket #4027
Hi Marek, I wrote a patch a while back that implemented a 'number per page widget' for admin. It never really went anywhere as it needed a better patch and some tests, which I didn't have the time to supply. I don't know if you would find this useful, let me know if you like and I can send you the patch. Regards Ben On 30/04/07, Marek Kubica <[EMAIL PROTECTED]> wrote: > > > Hi all, > > I created a new ticket with the need to disable pagination in the admin > object listing altogether. The current default value for pagination is 100 > objects per page, which can of course be extended to sys.maxint and is > surely enough but this leaves an odd feeling. In the ticket I put up > different proposals how to solve that: > > - list_per_page could accept 0 or None as "do not paginate" > - same as above, but rename this setting - list_per_page = 0 seems >strange > - create a setting like disable_pagination > > As far as I see, the oldform-admin is now finishing it's life, so there > isn't probably much sense in patching that. A patch (which I volunteer to > write) would need to go against newforms-admin, maybe even before the > merge to trunk, so backwards incompatible changes would hit with one > merge. > > The ticket: http://code.djangoproject.com/ticket/4027 > > So, what do you think about this? I'll be happy to hear your ideas. > > regards, > Marek > > P.S.: I'll try to write a patch for #4027, so expect many questions ;) > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Subversion cliff notes
Carole, Weren't you involved with the multiple-db branch before? With regards to using branches someone on this list advised me to have a look at SVK (sorry don't have a link to hand). I've looked at it and it's a nice way to go for working with SVN. You may be well served to have a look yourself as it covers all of the functionality of SVN in addition to making merges/branches etc less painful. Ben On 10/05/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > My apologies if this was the incorrect place to post this. I'm just > trying to get started on hopefully contributing to the project, and > had questions on getting started. I thought that was what this group > was for. > > On May 9, 5:01 pm, "Joseph Heck" <[EMAIL PROTECTED]> wrote: > > This is really the wrong list for this. Django-users would be more > > appropriate, or better yet a list that's about subversion. > > > > On 5/9/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > > > > > > > Can someone point me to a good site for a quick-guide to subversion? > > > We pretty much just do vanilla update/commit where I work...and I've > > > never dealt with branches. > > > > > I've checked out the trunk to one directory, and a patch which hasn't > > > been updted since 12/10/06 to another directory. > > > > > I'd basically just like to know how I find the diff for each trunk > > > revision, so I can start incorporating them into the branch code to > > > get it up to date, and then start bug fixing and hopefully, > > > eventually, get it closer to being incorporated into the trunk. > > > > > Yes...I could go search around on the subversion site, but I thought > > > someone might have a link to a good site with a list of common how-to > > > bullets that might be quickeror that someone might just be kind > > > enough to paste the command they use into a reply. > > > > > Thanks for any information. > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] SQLAlchemy branch questions
Hi all, I don't wish to rake over the debate on the database connection - newbie question<http://groups.google.com/group/django-developers/browse_thread/thread/99821103689f8787/910a12675c8b03f3?lnk=gst&q=database+connection+user+-+newbie+question&rnum=1#910a12675c8b03f3>debate which I followed with interest over the last few days. __BUT__ having used multiple-db it seems to fall down in a few areas. Specifically multiple-db as it stands at the moment doesn't seem to support FK relationships across databases properly. Ticket 3885<http://code.djangoproject.com/ticket/3885>and Ticket 3642 <http://code.djangoproject.com/ticket/3642> both refer to this issue. With these issues in mind, I'd like to ask some questions about multiple databases, specifically how they pertain to the SQLAlchemy branch. I've just had a quick google and there doesn't appear to be a great deal of activity in the SQLAlchemy branch at the moment. I'm aware that the mantle has passed, in a manner of speaking, from Robin to Brian. but I'm not sure what the current state of play is. From the log it looks like the last activity on the branch was a merge on the 2nd of Feb, and from all of the log messages it would appear that all of the check ins so far have been merges. However playing with my shiny new install of SVK it appears that trying to merge from 4456 to HEAD comes up with a number of conflicts, so I'm taking it that the merges so far have also been mixed in with a number of commits of code? Questions then: 1. Is the current thinking on the design of SQLAlchemy pretty much the same as Adrian laid out in his Intergating SQLAlchemy and Django message<http://groups.google.com/group/django-developers/browse_thread/thread/5149e1c60dc65bff/a177bb34cfde1ec7> ? 2. Would SQLAlchemy make using multiple databases easier/more possible? And if so could this be the default way to support multi-db? 3. Failing 2. How about a OtherDatabaseForeignKey and OtherDatabaseM2M field to overcome the problems with django's db-api breaking down over multiple databases? 3. What would people think of the concept of 'casting' a Model to the equivalent SQLAlchemy concept (a Table?) for any processing over and above django ORM and then - if necessary - translating back to django models. I say if necessary because it would appear that the Table object would have the same kind of structure as a django Model ( i.e. Model.field == Table.field, provided that db_column isn't changed). Caveat, I'm not a computer scientist, so appologies if I've used the wrong terminology here! Multiple databases is certainly a reality for me, and I would suggest it would be for a lot of people who have to integrate with legacy systems; the number of responses to the 'database connection' thread I mentioned above would appear to bear this out. I'd be very interested to hear peoples' opinions on this subject. Thanks for reading, Ben -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Caching querysets
How about caching the return of qs.iterator()? Would that work?
Ben
On 15/05/07, Jeremy Dunck <[EMAIL PROTECTED]> wrote:
>
>
> I'd like to propose an addition to the low-level caching API that
> fills the queryset _result_cache if the value being set (or contained
> in a sequence or dictionary being set) is a queryset.
>
> Consider that before magic-removal, the ORM returned actual lists, so
> that something like this was productive:
>
> cache.set('key', somemodel.get_list())
>
> The equivalent after magic removal might appear to be this:
>
> cache.set('key', somemodel.objects.all())
>
> But actually, it's not; since querysets are lazy, the cached queryset
> will execute each time it's iterated after fetching it from the cache.
>
> To get the previous behavior, you'd need to do this:
>
> cache.set('key', list(somemodel.objects.all()))
>
> The problem with this approach is that it's pretty easy to forget that
> list() have a non-obvious source of cache misses.
>
> Locally, I've changed our code to fill the qs._result_cache, but I
> feel a bit dirty doing it since django.core.cache really doesn't have
> much to do with django.db.model.query.QuerySet.
>
> Do people think it's worth doing?
>
> (I do, I just wish there was a nicer way to accomplish the goal...)
>
> -Jeremy
>
> Here's the code for backends/memcache:
>
> def _fill_qs_cache(self, value):
> """
> QuerySets are lazy and have an internal cache.
> It doesn't make much sense to cache them without this result
> cache being filled
> since you'll pay the query overhead each time you pull it out
> of the cache.
> Any time a QS is cached, you *could* force _get_data, but it
> breaks
> the abstraction of QS's being list-like.
> So, hack here to force QS cache to fill before storing.
> """
>
> from django.db.models.query import QuerySet
> if isinstance(value, QuerySet):
> value._get_data()
> elif isinstance(value, (set, list, tuple)):
> [self._fill_qs_cache(part) for part in value]
> elif isinstance(value, dict):
> [self._fill_qs_cache(part) for part in value.values()]
> else:
> pass #not queryset
>
> def set(self, key, value, timeout=0):
> self._fill_qs_cache(value)
>
> self._cache.set(key, value, timeout or self.default_timeout)
>
> >
>
--
Regards,
Ben Ford
[EMAIL PROTECTED]
+628111880346
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Runtime initialisation of db_table from context
Hi Chris, You could copy the whole model, ala DynamicModels on the wiki [ http://code.djangoproject.com/wiki/DynamicModels], I've sketched out a more complete example here: http://www.djangosnippets.org/snippets/442/. This example (I say example as it's a bit rough around the edges - especially the field copying code!) was needed when working with the old multi-db branch to specify a different database, so there are some things that won't work with trunk (the references to "connections" for example). The advantage of copying the whole model is that you can then do anything you want with it and the django API doesn't change. Cheers, Ben 2008/5/29 ChrisStoyles <[EMAIL PROTECTED]>: > > Hi Everyone, > > I've been doing some prototyping with django to see how good of a fit > it will be for an upcoming project, and would like to run an idea past > you guys. I'll try and explain my situation first, and then ask the > question afterward. > > At a very high level, I have an application, which holds many > organisations, and these organisations each contain many products. I > may need to at times, partition the product data, so that some > organisation products are stored in table 'a' and others are stored in > table 'b'. However the Product as a model class is the same between > both of these, and I won't know the names of these tables until > runtime (they may be based on organisation names, which would be > created by users). > > So as you might already have guessed, I would like to be able to use > my "Product" model class across many tables. I would do this by > changing the value of db_table. This however only works for saving an > instance of Product (I can change the db_table value for that > instance, prior to saving, and it will be committed to said table). I > cannot however change the db_table property for a single thread, so as > to fetch a Product from any arbitrary table name. > > I believe what would be needed is the ability to supply a context at > runtime, from which db_table could be determined, i.e. > Product.objects.all(ctx) where ctx.db_table is the name of the product > table I wish to use > > That is a very quick and dirty example, however I hope I have still > made sense and might be able to get some feedback from the developer > community. > > Thanks! > Chris > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Bug in mod_python
Hi all, I've come across some inconsistent behavior when running django under different conditions. This is in some existing code that has has some problems and isn't really well written, so I'm going to ask the question here first to avoid noise on trac before I raise a ticket. In this code there's a context processor that checks the path (uri) of the request and adds something if it's equal to a certain value. The code is: if request.META['PATH_INFO'] == : Under fastcgi, and runserver this is fine and we only noticed as we started to migrate to mod_python. It would appear that request.META['PATH_INFO'] is always equal to '/' when running under mod_python. I've fixed it by changing the code to check request.path, which seems to work. Questions then: 1) Is it better to use 'PATH_INFO' or .path (I'm guessing the latter). However ticket 285 <http://code.djangoproject.com/ticket/285> refers to inconsistent behaviour WRT self.path in the WSGI environment. 2) Is it a bug in django.core.handlers.modpython.ModPythonRequest to set META['PATH_INFO'] to self._req.path_info rather than self.path? 3) Does this need a ticket or is the use of request.META['PATH_INFO'] discouraged? (django's documentation would suggest the use of request.path) Cheers, Ben -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: a new middleware class for django
Hi Mark, This sounds really useful, I'd suggest sticking this on djangosnippets.orgfor others to try! Ben 2008/6/16 lep_man <[EMAIL PROTECTED]>: > > Hi Guys, > > I did not know to refer to. > > i have developed a middleware that helps to debug django application > in runtime (at production) > > the basic idea is if a specific useragent is accessing the site the > middleware class detects and append debug inforamtion that was added > to the request object into the web page as a remark. > > and than the user can see it in the view source. > > i found it vary useful in production in when debugging error that can > not be recreated in the django development server. > > Thanks > > -Mark Zitnik > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Bug in mod_python
Thanks Graham, I though that might be the case, I won't bother to raise a ticket then. Ben 2008/6/17 Graham Dumpleton <[EMAIL PROTECTED]>: > > > > On Jun 17, 1:28 am, "Ben Ford" <[EMAIL PROTECTED]> wrote: > > Hi all, > > > > I've come across some inconsistent behavior when running django under > > different conditions. This is in some existing code that has has some > > problems and isn't really well written, so I'm going to ask the question > > here first to avoid noise on trac before I raise a ticket. > > > > In this code there's a context processor that checks the path (uri) of > the > > request and adds something if it's equal to a certain value. The code is: > > if request.META['PATH_INFO'] == : > > > > Under fastcgi, and runserver this is fine and we only noticed as we > started > > to migrate to mod_python. It would appear that request.META['PATH_INFO'] > is > > always equal to '/' when running under mod_python. I've fixed it by > changing > > the code to check request.path, which seems to work. > > > > Questions then: > > 1) Is it better to use 'PATH_INFO' or .path (I'm guessing the latter). > > However ticket 285 <http://code.djangoproject.com/ticket/285> refers to > > inconsistent behaviour WRT self.path in the WSGI environment. > > 2) Is it a bug in django.core.handlers.modpython.ModPythonRequest to set > > META['PATH_INFO'] to self._req.path_info rather than self.path? > > 3) Does this need a ticket or is the use of request.META['PATH_INFO'] > > discouraged? (django's documentation would suggest the use of > request.path) > > When using mod_python, you cannot rely on req.path_info/PATH_INFO > being set in the traditional way that PATH_INFO would be for CGI > scripts. > > If you need this functionality with Apache with mod_python like > feature of embedding, you would have to use mod_wsgi. > > If mounting Django application at non root URL, do take heed though of > issues with Django and SCRIPT_NAME/PATH_INFO as described in: > > http://code.google.com/p/modwsgi/wiki/IntegrationWithDjango > > Graham > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Experimental Git repository available
Hi Jacob, This sounds like a nice idea... Any plans to do a mercurial repo (a-la the documentation refactor)? Am I right in thinking that support of SVN <=> HG is less complete than GIT <=> SVN? Cheers, Ben 2008/6/19 [EMAIL PROTECTED] <[EMAIL PROTECTED]>: > > Is this only going to offer the trunk branch? > > On Jun 19, 11:39 am, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> > wrote: > > Hi folks -- > > > > I've published an experimental Git clone of Django's SVN repository > > (created with git-svn). If you're a Git user and want to use this > > repository, be my guest. Please still upload patches to Trac for > > review, but mention you're using my clone so I can more easily apply > > patches from you. Also let me know if you publish your clones > > anywhere; I'll add 'em as remotes. > > > > Note that this is in no way official, this is just me fooling around > > with new tech. It's entirely possible that this won't actually save > > anyone any time, but since I'm fooling with Git I thought I'd give > > this a shot. Please don't try to turn this into a "Django should use > > Git!" thread; if you do I'll just ignore you. We're not switching from > > SVN any time in the foreseeable future. > > > > OK, so the details: > > > > Repository: git://djangoproject.com/django > > Gitweb:http://code.djangoproject.com/git/?p=django > > > > Jacob > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Experimental Git repository available
> Something like this? > http://hg.dpaste.com/django/ <http://hg.dpaste.com/django/> Yeah I use that one at the moment, I was getting at a more thing. I'm sure that's more difficult to do with HG then GIT and like Jacob said, it's a moot point anyway :-) Ben 2008/6/19 Ariel Mauricio Nunez Gomez <[EMAIL PROTECTED]>: > > <http://hg.dpaste.com/django/> > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
Hi Jan, It sounds like you've made great progress. We have an informal trac and hg repo set up at trac and hg dot woe-beti.de respectively. you're more than welcome to add your documentation there! Let me know if you want an hg repo tp play with too and I'll sort it out for you. Cheers, Ben 2008/7/8 Jan Oberst <[EMAIL PROTECTED]>: > > Hi guys, > > I've been heavily swamped with work for college, so I missed this > thead and the few others on multiple databases. Sorry. > > I have implemented a proof-of-concept database scaling solution for > Django. It tackles all kind of scaling issues I have seen in Django. > It's purpose is mainly to find out if we could scale up Django at all. > I didn't worry too much about syntax and the way it's supposed to > integrate into Django - I just hacked away in Django code to make it > work the fastest possible way I could think of. > > > The solution covers the largest part of Simon's #2 problem. I added a > few attributes and config parameters to the ORM so you can decide > which models are hosted on which server. One model can be hosted on 20 > servers with the actual location depending on a foreign key value. > > We're using it to store data for different groups on different servers > for a more horizontal scaling. For example if a photo got a ForeignKey > to group A it will be routed to server 15 because of some logic. > > You can also route objects 1-1000 to server 1 and 1001-2000 to server > 2. > > > I have also added database denormalization, caching foreign key > querysets to the DB, bulk prefetching, in-model privacy checks and a > few other things. > > A large percentage of the stuff probably isn't suitable for Django- > trunk. Most of it tackles quite specific and hard scaling issues, but > I guess there's a way to build it more modular and make it work for > more people. After all I'm new to Django-developers and also to > opening up my work. > > If some of you are interested in the code and would benefit from it I > would be more than happy to share. > > Just posting a big pile of code probably won't help you too much, so I > thought I'd write a few lines documentation about each part and post > them here. Does that sound reasonable? > > Jan > > On May 22, 4:59 pm, Simon Willison <[EMAIL PROTECTED]> wrote: > > I have to admit I'm slightly worried about the multi-database > > proposal, because at the moment it doesn't seem to solve either of the > > multi-db problems I'm concerned about. > > > > The proposal at the moment deals with having different models live in > > different databases - for example, the Forum application lives on DB1 > > while the Blog application lives on DB2. > > > > I can see how this could be useful, but the two database problems that > > keep me up at night are the following: > > > > 1. Replication - being able to send all of my writes to one master > > machine but spread all of my reads over several slave machines. > > Thankfully Ivan Sagalaev's confusingly named mysql_cluster covers this > > problem neatly without modification to Django core - it's just an > > alternative DB backend which demonstrates that doing this isn't > > particularly hard:http://softwaremaniacs.org/soft/mysql_cluster/en/ > > > > 2. Sharding - being able to put User entries 1-1000 on DB1, whereas > > User entries 1001-2000 live on DB2 and so on. > > > > I'd love Django to have built-in abilities to solve #1 - it's a really > > important first-step onscalingup to multiple databases, and it's > > also massively easier than any other part of the multi-db problem. > > > > I wouldn't expect a magic solution to #2 because it's so highly > > dependent on the application that is being built, but at the same time > > it would be nice to see a multi-db solution at least take this in to > > account (maybe just by providing an easy tool to direct an ORM request > > to a specific server based on some arbitrary logic). > > > > I may have misunderstood the proposal, but I think it's vital that the > > above two use cases are considered. Even if they can't be solved > > outright, providing tools that custom solutions to these cases can be > > built with should be a priority for multi-db support. > > > > Cheers, > > > > Simon > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Sprinting
Damn gmail, I wasn't ready to send that!!! What I was also going to say is that time is relatively flexible, so if anyone who is interested could get in touch we could talk about possible dates. I don't know if there are any committers in the Oxford area who might be available, but that would obviously help! Cheers, Ben 2008/7/22 Ben Ford <[EMAIL PROTECTED]>: > Hi all, > > Appologies for being a couple of weeks late with this... I'm currently > working for in a team developing a django site and we'd like to offer our > facilities (office space, wireless, tea making etc..) to hold a sprint in > Oxford, UK. I've noticed that there are a few people/companies using django > in the Oxford area > > -- > Regards, > Ben Ford > [EMAIL PROTECTED] > +447792598685 > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Sprinting
That's cool! We don't really have any dates in mind yet... it would seem to make sense to do it on one of those dates - especially if there aren't any committers available in Oxford. These are the date's left: August 1 Push to beta sprint in Washington, DC and in IRC. August 5 *1.0 beta 1.* August 8 Push to beta 2 sprint in Lawrence, KS. August 12 *1.0 beta 2.* August 15 Release candidate sprint in Austin, TX. August 19 *1.0 rc 1.* August 22 Final release sprint in Portland, OR August 26 Earliest possible 1.0 release date, or perhaps rc2. September 2 *1.0* I guess any of those would be good! Ben 2008/7/22 Tom Dyson <[EMAIL PROTECTED]>: > Hi Ben > > Some of us (at Torchbox) would be interested, depending on dates. Are you > planning to coincide it with one of the main Django sprints? > > Cheers > > Tom > > > On 22 Jul 2008, at 09:51, Ben Ford wrote: > > Damn gmail, I wasn't ready to send that!!! What I was also going to say is >> that time is relatively flexible, so if anyone who is interested could get >> in touch we could talk about possible dates. I don't know if there are any >> committers in the Oxford area who might be available, but that would >> obviously help! >> >> Cheers, >> Ben >> > > -- > tom dyson > +44 (0)1608 811870 > http://torchbox.com > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Sprinting
Hi all, Appologies for being a couple of weeks late with this... I'm currently working for in a team developing a django site and we'd like to offer our facilities (office space, wireless, tea making etc..) to hold a sprint in Oxford, UK. I've noticed that there are a few people/companies using django in the Oxford area -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: RFC: django.template refactoring (#7806)
I take it that most are aware of: http://lucumr.pocoo.org/cogitations/2008/09/16/why-jinja-is-not-django-and-why-django-should-have-a-look-at-it/ It seems like a very well thought out and thorough write up. Ben 2008/9/17 Johannes Dollinger <[EMAIL PROTECTED]> > > > Would @register.tag(token_stream=True) work instead, or am I missing > > something? > > Yes, that would work. > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Python 2.6 abstract base classes
Named tuples looked like good juju in Raymond Hettinger's talk at pyconuk
too! not sure if they're in 2.6 or 3.0 though
Ben
2008/10/2 Marty Alchin <[EMAIL PROTECTED]>
>
> Looking over the new things in Python 2.6, I wonder if there are plans
> to inherit from any of the new abstract base classes in Django. It
> looks (to me) like it could be backwards-compatible, using an import
> like so:
>
> try:
>from collections import Sequence
> except ImportError:
># Fallback for Python 2.5
>Sequence = type('Sequence', (object,), {})
>
> Django obviously works just fine without these at the moment, so
> there's no pressing need to do so, but if they catch on and 2.6+
> libraries start checking for them, it might be beneficial if some of
> our custom types, like QuerySets, inherited the appropriate classes.
>
> I'm not really in favor of using or not using them, but I'd just like
> to know if they're in the plan, in case I should add some notes in my
> ... documentation.
>
> -Gul
>
> >
>
--
Regards,
Ben Ford
[EMAIL PROTECTED]
+447792598685
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Declarative fixtures for django
etter integration with django's test framework. So I have a few questions on where to go next... 1. What's the best way to integrate with django's test framework (Transactions, the test client etc). Subclass django's TestCase or something else? 2. I've written some validation code, I'd appreciate it if someone could have a look through this module<http://bitbucket.org/boothead/fixture-django/src/tip/fixture/loadable/django_loadable.py>particularly the field_is_required function and the _check_schema method and tell me if there's a better way to do it. 3. At the moment you can only specify a relationship from the direction that the field is defined. i.e as we have Book -> fk -> Author you can't write a dataset the does: class app__Author(DataSet): class auth1: ... books = [app__Book.book1, ...] I made it work this way to simplify development, but I'm wondering if it's the right decisoin. Is this desirable behaviour? I hope this is usefull to everyone, and in the spirit of Malcolms recent blog post, a big thank you to all of you :-). If you have any improvement suggestions please let me know too. Cheers, Ben -- Regards, Ben Ford ben.for...@gmail.com +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Declarative fixtures for django
Hi Malcolm This is one of those cases where I have to ask why you're using a class > instead of a dictionary. You're mapping keys (initialisation parameters) > to values (the values to set the parameters to). Using a map object > seems like the way to go. It doesn't carry the necessary indentation > requirements with it, for a start. > This is just the way fixture does it, I found it a bit of a paradigm shift at first, but I actually really like it now, for one thing it allows you to do: class app__Model(DataSet): class generic_instance: field_one = ... lots of other generic fields here class another_instance(generic_instance): different_field = ... other_overridden_field = ... In this instance you can define fields on generic_instance that will be present unless overridden in another_instance. I also allows for the mechanism of referring between DataSets. See http://code.google.com/p/fixture/source/browse/trunk/presentations/ for more of that back story, usage and motivation behind fixture. > > > I'd look at that approach first. Looks like you'd need to override the > _pre_setup() method. This seems like a very easy question to answer. Why > isn't the "yes" just the obvious answer? Is there some hidden shoal in > there? The testing landscape has changed a bit since I looked in detail, and DataSets are loaded in a transaction as well so I guess the waters are potentially a bit cloudy WRT transactions and fixture loading. Fixture also provides some niceties for loading fixtures so it may come down to a choice of a fairly vanilla subclass of unittest's TestCase as opposed to a subclass of django's overriding _pre_setup() > > This is one of those things I'm going to have to postpone for a while > until I have more time. You're on the boundary of repeating a bunch of > internal code (particularly from django.db.models.base.Model) by the > looks of it, without constructing models. I'd be looking at closely > whether you can reuse more of the existing stuff. Yes this was what I was getting at really, it's been some time since I played around with the internals of django models, (pre qsrf) so things have changed a bit. I'd much rather do this with existing django code the roll my own stuff which is overly familiar with django's privates so to speak! > > One general observation: The existing fixtures code, for all its > edge-case fragility, doesn't seem to have any real design constraints > stemming from the requirement that it creates Model instances and saves > them (calling Model.save_base(raw=True)). Any particular reason you're > not going down that path? You'd get all the error checking for free. This was what I was after... I wasn't going down that path because I didn't know about it, thanks for the steer! :-) > > > You also don't avoid the problems such as MySQL + InnoDB not having > deferrable constraint checking: none of that is a restriction of the > Model layer. Sound far preferable to the way I've done it then. > > Try to implement the reverse direction as well, but don't lose sleep if > you can't, I'd suggest. :-) > > When you have access to the models, you know the names Django uses to > refer to them, so you can use the same names in the fixture > descriptions. For example, If Child -> Parent, exists, then the Parent > fixture could define stuff for "child_set" (or the right related name). That's pretty much what I thought. I started off this way to constrain what I would have to cater for, but it would seem reasonable and more flexible to let people so it either way. The one case where it might not work would be a non-nullable ForeignKey. I will investigate, I have a suspicion that fixture might have catered for this already. > > In the past (not Django, but in general coding), I've found it > beneficial to write unit tests for third-party code to ensure it doesn't > change some piece of behaviour we were relying on (or that our patched > version didn't forget to be patched). Being able to specify fixtures for > all related models in those cases could be useful. It's a question of > how strictly you want to define the word "unit" in "unittest" (and > Django takes a pretty liberal position there, which isn't necesasarily a > bad thing). You've currently said that people can test an app and > everything it refers to. But not test things that depend on that app. > That's not a horrible restriction: facing only in the direction of > upstream dependencies. But if you can make it worth both ways, why not? Yep, this would be quite a nice aim, if an upstream app already had te
Re: Model validation
Hi Honza, Would this code be an approach I could take with my work on fixture? (See my message from a couple of days ago). Cheers, Ben 2009/3/10 Honza Král > > Hi, > The code on github works (tests pass) but is not ready for production > use. What you can do to help is test and mostly, write tests to help > us get there. There will be a sprint on model-validation during pycon, > which should resolve most of the outstanding issues. > > > > Honza Král > E-Mail: honza.k...@gmail.com > ICQ#: 107471613 > Phone: +420 606 678585 > > > > On Mon, Mar 9, 2009 at 11:25 AM, David Reynolds > wrote: > > > > Hi, > > > > I need to use some sort of model validation in a project I am working > > on and I wondered what that status of > > http://code.djangoproject.com/ticket/6845 is. > > > > Is the github code useable for production? Is there anything I can do > > to help with it? Is there a recommended workaround for model > > validation in the meantime? > > > > Thanks, > > > > David > > > > -- > > David Reynolds > > da...@reynoldsfamily.org.uk > > > > > > > > > > > -- Regards, Ben Ford ben.for...@gmail.com +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: [gsoc2009-testing] Windmill Runners Kicking it up a Notch
Hi Kevin, I haven't had a chance to really read and digest this - but yu might want to check out fixture http://farmdev.com/projects/fixture/ and the django support branch for it http://bitbucket.org/kumar303/fixture/ for loading fixtures for tests. Fixture might give you more options for loading data to test. Cheers, Ben 2009/6/27 Kevin Kubasik > So, up until now, most of the work on windmill hasn't exactly been 'tester > ready' in that, if you didn't you my exact incantation of settings, luck, > module versions and love, you didn't get very far. While I still haven't > gotten real docs done (my original plan for the week) I have learned a > valuable lesson in assumptions. I foolishly assumed that splitting the WSGI > handler into its own thread, > while introducing another server (windmill), and doing my fixture loading in > the primary thread would just work. Surprisingly, it almost does, except for > the sqlite3+:memory: instance. > Obviously, as this is how 99% of django tests are run, I had hit a snag. I > believe that the problem is > solved, but it did put me a bit behind, as I have had to totally reconsider > how the loaders and databases > are handled. Namely, the following questions arose: > >1. Should windmill tests default to non-isolated/non-transactional DB >behavior? Basically, we are providing the means for functional tests, these >should be tests like 'Register and Complete a Profile', then 'Edit > Profile'. >We don't want this as one massive test, and it seems like that would be the >expected behavior most of the time, and still allowing for the option of >specific tests being run in isolation seems like the best move. However, >this could be confusing, or just bad practice, so I wanted to get some >feedback. >2. Scratch my #2 I caught one of the Windmill guys on the IRC and got >some good direction on detection stuff. >3. What is the general interest in test-only models as a public api? >The mechanics of it have been worked out for the regression suite, but the >debate falls to one of the following systems. > 1. A class used instead of db.Model (db.TestModel) > 2. A module in the app (test_models.py) > 3. Similar to fixtures (a property on tests) > 4. A settings option >4. I am assuming that code coverage of windmill tests isn't that useful >of a number, given the specialized execution paths etc. But I wanted to >double check that people wouldn't be surprised by that. > > Overall, there has been marked improvements in the runner state, and I've > added some more tests as well. However, I am holding off on the really js > intense tests until the framework is rock solid. But I wanted to get moving > on documentation sooner rather than later, so I expect a bit more cleanup > next week to make sure that the elaborate charade we play (conning windmill > to play with us) is reliable for 3rd party applications as well. > > The branch isn't really ready for testing yet, but it has been known to > work. And Eric has kindly thrown up a coverage report! > > http://media.ericholscher.com/django_coverage/ > -- > Kevin Kubasik > http://kubasik.net/blog > > > > -- Regards, Ben Ford ben.for...@gmail.com +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: django doc improvement suggestion
g groupings are > also welcome. > > >> I find http://docs.djangoproject.com/en/dev/topics/db/ page quite > >> inadequate. If I was doing that page I would include links to the > following > >> pages, along with the ones that are there: > >> > >> (Model Field Rerence, where all standard fields are listed) > >> http://docs.djangoproject.com/en/dev/ref/models/fields/ > >> (Writing Custom Model Fields) > >> http://docs.djangoproject.com/en/dev/howto/custom-model-fields/ > > This essentially reinforced my previous point. You have referenced > three different sections of documentation here: > - A topics guide (/topics) > - The reference (Everything you need to know about custom fields) > - The HowTos (Howto build a custom field) > > These _should_ contain three different aspects of detail about model > fields. They are complimentary, and should be cross linked. > > >> The content of these pages used to be clubbed with > >> http://docs.djangoproject.com/en/dev/topics/db/models/, which was very > >> handy, I have to open just one page, and all relevant information for > >> writing models is there, now I have to keep on searching. So I would at > the > >> least add the links to the above to pages prominently on the top or on > the > >> sidebar, as of now they are quite difficult to find. > > Since the days of old, we have added a lot of extra documentation. In > the interests of organization, the single monolithic page has been > split. Rather than try to manage sub-sub-sub-sub headings on a single > page, we have introduced a hierarchy of pages. > > >> Please merge the three pages together, or at least put the cross links > >> prominently. > > Merging - no. Cross linking, yes, in conjunction with relevant > additional content: > > - The HowTo should have a step-by-step walkthrough of the process of > building a specific custom field, with a link to the topic guide and > reference. > - The topic guide should have a broad description of the issues > surrounding custom fields, with a link to the reference, and the howto > - The reference should have a complete rundown of the spec required by > a custom field, with links to the topic guide and howto. > > In some cases, the content doesn't exist yet. In some cases, the > reference content may be in the topics section (for historical > reasons, or due to oversight). Part of what is needed is an audit of > what is missing, as well as proposals for what should be reorganized. > > >> Will create a patches if given a go ahead. > > Always welcome. > > Yours, > Russ Magee %-) > > > > -- Regards, Ben Ford ben.for...@gmail.com +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
multiple-db-support
Hi JP, I've been using your multiple-db-support for sometime now and i wondered if you are still actively developing it? I have a couple of suggestions/queries to do with a few of my specific needs and I would guess those of others... There are a couple of tickets about this on trac, but I might try and do some bits independently so I wanted to ask for your feedback... - I'd really like if I could define a FK relationship that spans databases, at present that will work but the backwards relationship is broken and it throws a validation error. - I'd like this 'spanning foreignkey' to not break the django ORM... so that Model.objects.filter(fk_class__column='whatever') would still work. I'm guessing that this would be quite a lot of work... Probably consisting of: - A new type (or types) of Field, perhaps OtherDBForeignKey or something - A new type of descriptor for each of these new field types - Some new logic in django.db.models.query to catch the case of a db spanning relationship and split the queries up - Finally some way to build the sql so that it looks like SELECT [columns,] from first_table where pk in (list_of_pk_values); the list will be generated from a separate query as a result of splitting the queries up in the last step. What do you think about the amount of work that would be needed to get all this working? As an aside i have a copy of multiple-db-support that I've merged with trunk as of rev 5371, it might be good to get this checked into django so other people can work on it, if there's still any interest... Thanks in advance, Ben -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: multiple-db-support
Russ, Can I suggest doing it the other way round in this case. I believe that the multiple-db-support branch has only a small user base and I haven't had time to properly test all aspects of the merge, I think it might make it easier for others to test if they can just svn up. There are some aspects of the branch that still need work and I think that most people who are using it recognize this. Also as Carole has expressed an interest in contributing and also SVK, it would be beneficial to her (and any others) to be able to grab a local copy of the branch which she can keep in source control as she adds bits to it. What do you think? Ben On 08/06/07, Russell Keith-Magee <[EMAIL PROTECTED]> wrote: > > > On 6/8/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > > It would be nice if someone might tell either myself or Ben, how we > > would go about getting the merged code committed, once it's been > > completed and tested or do we just submit it as a patch or > > something on the wiki page for this branch? > > Initially, I would suggest submitting as a patch. If there isn't > already a ticket for the multi-db feature, open one, and keep the > patch up to date. > > Once you demonstrate that you're (1) capable of doing the work, and > (2) are in this for the long haul, we can get you an SVN login for the > branch. > > Yours, > Russ Magee %-) > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: multiple-db-support
Hi Russ, Thanks for the reply. What's the prescribed way to track multiple patches in one working copy? I haven't had much experience with patches, but I built my django base code using patches from a couple of different branch into one working copy and it caused me problems in that i couldn't then "svn revert" any of those changes individually... I completely understand the rationale behind django's svn access policy and I agree that you can't let just any, unproven person in... With this in mind and given the fact that I think some kind of SCM is important if more than a couple of people are working on the branch, I wonder what you and other core devs would think of this idea: I could mirror the branch using svk onto my server. Anyone who is interested in working on it, sharing ideas (or checking on progress) could then have access to their own copy with the ability to commit changes. It would also be easy to keep trunk merged into this repo, even though it is remote. When the code is felt ready to check back into the django branch, svk has a mechanism to do that, and as a bonus it can also do it incrementally, so that for each change made on the remote repo there will be a log entry into the django repo. At any point along the way core devs would be able to check on progress/code quality etc and make any suggestions. I have no interest whatsoever in forking django or in any kind of conflict, I suggest this merely as a possibility for consideration. I think it might be a fairly elegant way to solve some problems, and would be interested in getting some feedback... Thanks, Ben On 08/06/07, Russell Keith-Magee <[EMAIL PROTECTED]> wrote: > > > On 6/8/07, Ben Ford <[EMAIL PROTECTED]> wrote: > > Russ, > > > > Can I suggest doing it the other way round in this case. I believe that > the > > multiple-db-support branch has only a small user base and I haven't had > time > > to properly test all aspects of the merge, I think it might make it > easier > > for others to test if they can just svn up. There are some aspects of > the > > branch that still need work and I think that most people who are using > it > > recognize this. Also as Carole has expressed an interest in contributing > and > > also SVK, it would be beneficial to her (and any others) to be able to > grab > > a local copy of the branch which she can keep in source control as she > adds > > bits to it. > > I appreciate your enthusiasm, and I understand the desire to have a > SVN checkout. > > However, we (the core devs) need to set a minimal set of requirements > before we hand out SVN logins. We have, in the past, given out branch > logins to anyone that wanted them. This has, almost without exception, > lead to stale branches as the developer involved loses interest, etc. > A stale branch is worse than no branch at all, because it raises false > expectations, and doesn't reflect well on Django as a whole. > > We also have the problem of making sure that the job is going to be > done properly. The ultimate goal will be to merge the branch back into > trunk. However, this isn't going to happen if the branch doesn't meet > Django's coding standard. I'm not just talking about indentation and > spacing here - I mean the technical quality of the code. > > Prove to us you can do the work, and that you're not going to lose > interest, and we'll get you a checkout. In the interim, patch -p0 < > mychanges.diff isn't that hard to do; and I would suggest that the > people that can't work out how to use patch probably won't be able to > make a very big contribution to debugging the branch, either. > > Yours, > Russ Magee %-) > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: multiple-db-support
Hi, The branch is now up to date as of 5371 (from 4188, yes _stale_ is one word you could use) in my local svk repo. Some of the tests are failing, (but they're also failing at the same point when I run the tests in trunk too, so I'm not quite sure what's going on!!). As for bug fixes I have my eye on 3642 <http://code.djangoproject.com/ticket/3642> and 3885<http://code.djangoproject.com/ticket/3885>. Both of these refer to ForeignKeys breaking down at the point where the relation crosses from one db to another. I think this is a worthy feature as i guess the whole point of integrating other db's into your django app is to interace with existing objects. I'm thinking of another kind of Field (perhaps OtherDBForeignKey) and Descriptor objects to solve this... Also off the top of my head i would imagine django.db.models.query will need some TLC to make it work too!! Ben On 08/06/07, Russell Keith-Magee <[EMAIL PROTECTED]> wrote: > > > On 6/8/07, Ben Ford <[EMAIL PROTECTED]> wrote: > > Hi Russ, > > Thanks for the reply. What's the prescribed way to track multiple > patches in > > one working copy? I haven't had much experience with patches, but I > built my > > django base code using patches from a couple of different branch into > one > > working copy and it caused me problems in that i couldn't then "svn > revert" > > any of those changes individually... > > There isn't anything prescribed; feel free to use whatever works for > you. There are any number of patch management tools out there that > could make your life easier. > > > I have no interest whatsoever in forking django or in any kind of > conflict, > > I suggest this merely as a possibility for consideration. I think it > might > > be a fairly elegant way to solve some problems, and would be interested > in > > getting some feedback... > > If you feel enthused enough to set up a SVK repository, feel free, but > the first task that is required is updating the branch to follow the > latest changes in trunk. This isn't really a task that can be > effectively shared amongst many developers so I'm not sure that an SVK > repository will help. > > However, providing a series of patches that bring the a stale branch > up to date (and multi-db is _quite_stale) is one very convincing way > to demonstrate that you are serious and capable. Combine that with one > or two non-trivial additional bugfixes, and you will have met the > criteria for getting SVN access. > > Yours, > Russ Magee %-) > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: multiple-db-support
Which tests? modeltests/test_client/ClientTest.test_session_modifying_view returns ok and after that everything stops... I have to kill the bash session. >Are you running MySQL? There are some known test failures due to >problems in the way MySQL handles row-level integrity. One of my databases is MySQL, but the tests are all running on Postgres >Hrm... my gut reaction is that the model description shouldn't know it >is separated across databases; this is a deployment issue, so it >wouldn't be good to hard code deployment into the model. That's my feeling as well. I'm trying to keep my mitts off the existing code as much as possible though, I guess one other option would be a property on ForeignKey that it could set on initialization by looking at it's rel.to._default_manager.connection and making sure it's the same as the class the FK is defined on? >The queryset is (hopefully) about to undergo a bit of a refactor; so: > >1) Don't base too much of your changes on massive reworkings of query.py >2) If you come across any refactorings that would be useful - >especially in relation to writing custom fields, etc - make sure you >make them known. The parts I have in mind that would most need to change are probably lookup_inner. I'll be looking to intercept the query at the point where a __ indicates a change of database and construct a select that looks more like this: SELECT [columns,] from first_table where pk in (list_of_pk_values) The list_of_pk_values will have to be generated by running the part of the query after the __ in a separate query on the related database. So these classes: class ThisDBModel(models.model): class OtherDBModel(models.model): lookup_field = models.CharField(..) firstdb_class = models.ForeignKey(ThisDBModel, related_name='otherfk') And this query; ThisDBModel.objects.filter(otherfk__lookup_field='whatever') Would end up as: list_of_fk_values = SELECT firstdb_class_id FROM otherdbmodel WHERE lookup_field = 'whatever'; #run on the other db and: final_query = SELECT * from thisdbmodel WHERE pk_column in (list_of_fk_values); #run on the main db What do you think? -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: multiple-db-support
>The Model should probably be keeping track of the database it is on, >rather than the field. The model base already tracks column names and >the like; adding a database name wouldn't be out of place. Then, if a >field references a model, you can compare the remote model's database >name with the local model. Essentially, that's what this bit is doing: rel.to._default_manager.connection In muti-db the model's default manager keeps a reference to the db. I'll see if i can have a play with the tests and nail it down a bit better! Ben -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Patch vs checkin
Hi all, I've now got the multiple-db-support branch fully merged up to date with trunk (as of rev 5559). As I've mentioned before it's in a mirrored repo, so I could commit all of the changesets incrementally back into the django svn repo (I've done 4 separate merges so far). I think that this approach has more value than one patch, as if I've screwed something up it might make it easier to catch without impacting on the other changes. On the other hand I understand the concerns raised earlier with letting anyone commit to django and I can supply a patch to one of the core devs instead if that is preferred :-). There have been a few people who seem to be interested in moving forward with multi-db and i'd be really keen to get this stuff into svn one way or the other so others can get 'eyes on'. The tests are currently still a bit borked... They are basically locking up the django_session table in my test database (running postgres) around about the place where the output says "Request a page that modifies the session ... ok" i'm not sure what's going on and I don't think I know enough about it to find out in a reasonable space of time. Thanks Ben -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: regression test failures
se self.model.DoesNotExist, "%s matching query does not exist." % > > self.model._meta.object_name > > DoesNotExist: Site matching query does not exist. > > -- > > Ran 135 tests in 166.710s > > FAILED (errors=1) > > Destroying test database... > > > > So, is it something missing in my environment > > or the way I run the tests or what? > > Whoops; you've found a subtle bug -- not sure if it's a code bug or a > documentation bug yet. Change your SITE_ID value to be something other > than 0 and the tests will pass, at least for the latter case. > Embarrassingly, I don't have a clean 0.96 tarball lying around, so I > haven't checked that case. > > Somewhere we're doing something like "if settings.SITE_ID" or "if val" > where val is settings.SITE_ID and failing because 0 == False. That might > or might not be easy to work around (or find). It could be that we end > up having to say SITE_ID must be nonzero,, but that's certainly the > workaround for now. > > Regards, > Malcolm > > > -- > Two wrongs are only the beginning. > http://www.pointy-stick.com/blog/ > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Ticket 4747
Hi all, I've raised a ticket 4747 <http://code.djangoproject.com/ticket/4747> to share my patch of trunk (rev 5589) back into multi-db. I can supply incremental patches if anyone wants them, but I had trouble adding them to the ticket. I'd encourage everyone who's expressed an interest to apply it to trunk and have fun with it... There are some problems with the tests on my machine, basically i can't get them to run properly, so if anyone could look into that it would be great, I'm guessing that the tests will be somewhat incomplete! Thanks Ben -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Patch vs checkin
Hi, None as yet, I'm really keen to get it back into the svn repo, but I think the people concerned are busy with other things at the moment. I can send you the patch if you like and I'll chase them up again. Ben On 11/07/07, koenb <[EMAIL PROTECTED]> wrote: > > > Is there any news on updating the svn repo for this branch ? > > k > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Problem inspecting a (django) postgresql table
Peter,
I ran into this problem when I was bringing the multi-db branch up to date
and was so busy I forgot to tell anyone! (My bad) I fixed it, but it was on
another machine. It's a pretty simple change, but the exact diff escapes me
at the moment! Is there a ticket on this, as I think it might be a genuine
problem in trunk (as well as my branch)? If you raise a ticket (if there
isn't one already) and email me the number, I will attach the diff when I
get back onto my other machine.
Cheers
Ben
On 11/07/07, Peter Nixon <[EMAIL PROTECTED]> wrote:
>
>
> When running "python manage.py inspectdb" against the following postgresql
> table:
>
> # \d auth_message
> Table "public.auth_message"
> Column | Type | Modifiers
>
> -+-+---
> id | integer | not null default
> nextval('auth_message_id_seq'::regclass)
> user_id | integer | not null
> message | text| not null
> Indexes:
> "auth_message_pkey" PRIMARY KEY, btree (id)
> "auth_message_user_id" btree (user_id)
> Foreign-key constraints:
> "user_id_refs_id_650f49a6" FOREIGN KEY (user_id) REFERENCES
> auth_user(id)
> DEFERRABLE INITIALLY DEFERRED
>
>
> I get the following error from django:
>
> class AuthMessage(models.Model):
> Traceback (most recent call last):
> File "manage.py", line 11, in
> execute_manager(settings)
> File "/usr/lib/python2.5/site-packages/django/core/management.py", line
> 1742, in execute_manager
> execute_from_command_line(action_mapping, argv)
> File "/usr/lib/python2.5/site-packages/django/core/management.py", line
> 1642, in execute_from_command_line
> for line in action_mapping[action]():
> File "/usr/lib/python2.5/site-packages/django/core/management.py", line
> 890, in inspectdb
> relations = introspection_module.get_relations(cursor, table_name)
>
> File
> "/usr/lib/python2.5/site-packages/django/db/backends/postgresql_psycopg2/introspection.py",
> line 35, in get_relations
> relations[int(row[0][1:-1]) - 1] = (int(row[1][1:-1]) - 1, row[2])
> TypeError: int() argument must be a string or a number, not 'list'
>
>
> ie. Django doesn't know how to inspect its own tables...
>
> Cheers
> --
>
> Peter Nixon
> http://peternixon.net/
>
> >
>
--
Regards,
Ben Ford
[EMAIL PROTECTED]
+628111880346
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Patch vs checkin
Forest, I'm already happily using SVK, and was originally planning to use it to remotely host the multi-db branch on my virtual server... However the only info I could find on achieving this was incorrect and I'm not sure if it is even possible to split repos (onto 2 different machine), which is what I'd originally planned... I currently have all of the changes I've made to the branch hosted in an SVK repo, but I have no way to share those changes with interested parties - except by way of patches, which seems inelegant! Do you know of anyway to achieve this using SVK..? Ben On 11/07/07, Forest Bond <[EMAIL PROTECTED]> wrote: > > On Wed, Jul 11, 2007 at 09:12:14AM -0500, Jacob Kaplan-Moss wrote: > > Probably the best way to host an "unofficial" branch would be by using > > Tom's Mercurial mirror of Django trunk: > > http://hg.korpios.com/django.trunk/. > > Since you're on svn, you might also try svk. > > -Forest > > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.6 (GNU/Linux) > > iD8DBQFGlOZHRO4fQQdv5AwRAspiAJ9D3/FKYTeCw42vQok6M5u6EXnpEQCeMKbL > 6n8oBKPFqBWhjKQxYQt09cI= > =pqb0 > -END PGP SIGNATURE- > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Problem inspecting a (django) postgresql table
Yes, word for word damnit! Wish I'd seen that before I started messing... Still it taught me a bit more about postgres I guess so it's all good! :-) Cheers, Ben On 12/07/07, Gary Wilson <[EMAIL PROTECTED]> wrote: > > > Ben Ford wrote: > > Peter, > > I ran into this problem when I was bringing the multi-db branch up to > date > > and was so busy I forgot to tell anyone! (My bad) I fixed it, but it was > on > > another machine. It's a pretty simple change, but the exact diff escapes > me > > at the moment! Is there a ticket on this, as I think it might be a > genuine > > problem in trunk (as well as my branch)? If you raise a ticket (if there > > isn't one already) and email me the number, I will attach the diff when > I > > get back onto my other machine. > > http://code.djangoproject.com/ticket/2591 > > Was your fix the same? > > Gary > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Ticket 4747
Hi Koen, Probably best if you just update the ticket with any info, I'll try and do the next round of updates tomorrow or the next day, and I'll stick another patch on the same ticket. I'm on my windows laptop at the moment so I'll have a look at the patch tomorrow and incorporate these changes into the next merge. Thanks for the feedback!! Cheers Ben On 13/07/07, BIERMANS Koen <[EMAIL PROTECTED]> wrote: > > Ben, > > > > I just tried this out, how do you want feedback ? > > Found a few little things, but you probably had these covered already (see > the patch in annexe). > > > > The patch is a bit strange revision wise, but I have no experience with > svk, so I just took your merger of 5589 and put that in a separate svn repo. > I'll try and read up on svk this weekend (I guess that should make things a > little easier). > > > > One of the changes is maybe less obvious: returning a different QuerySet > for different models in db/models/manager.py. I changed this due to the fact > that I am using my home-made DB2 backend, which uses the same trick as the > Oracle backend, that is: return a different QuerySet class. No problem if > you are using ONLY DB2, but if you are mixing DB2 and sqlite, than the > manager must be able to return QuerySet objects of different classes per > model. > > > > The typo in db/models/fields/__init__.py was preventing the auto_now in > the admin_log model to work (it bypassed the valuesetting in the DateField > class. > > > > A first try in a little app of mine seems to work now. I will try to see > if I can get some of the testsuite working. > > > > Thanks a lot for the good work! Django really needs this (and so do I > BTW). > > > > Koen > > > -- > > *Van:* Ben Ford [mailto:[EMAIL PROTECTED] > *Verzonden:* woensdag 11 juli 2007 14:36 > *Aan:* BIERMANS Koen > *Onderwerp:* Fwd: Ticket 4747 > > > > Hi Koen, > This is the ticket that I raised... I'll probably update the patch soon to > bring it up with trunk again. It might be worth sending an email to > django-dev (or any of the core devs if you know them) to try and raise the > profile on this. I know that several people are interested in getting the > branch up and running again, it seems a shame to let it stay stagnant when > there is a desire to progress! > Cheers Ben > > -- Forwarded message -- > From: *Ben Ford* <[EMAIL PROTECTED]> > Date: 03-Jul-2007 17:08 > Subject: Ticket 4747 > To: django-developers@googlegroups.com > Cc: jason pellerin <[EMAIL PROTECTED]>, [EMAIL PROTECTED], > Brian Harring <[EMAIL PROTECTED]>, Carole Zieler <[EMAIL PROTECTED] > > > > Hi all, > > I've raised a ticket 4747 <http://code.djangoproject.com/ticket/4747> to > share my patch of trunk (rev 5589) back into multi-db. I can supply > incremental patches if anyone wants them, but I had trouble adding them to > the ticket. > > I'd encourage everyone who's expressed an interest to apply it to trunk > and have fun with it... There are some problems with the tests on my > machine, basically i can't get them to run properly, so if anyone could look > into that it would be great, I'm guessing that the tests will be somewhat > incomplete! > > Thanks > > Ben > > -- > Regards, > Ben Ford > [EMAIL PROTECTED] > +628111880346 > > -- > Regards, > Ben Ford > [EMAIL PROTECTED] > +628111880346 > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: db backend refactoring
Hi Brian,
Just a quick question from me regarding your comment about supporting
multiple databases... I've had a quick look through your patch and it seems
to cover a lot of the ground that the multi-db has already changed.
- I was wondering if you're familiar at all with mulitple-db branch
yet? (You sent me an email about getting the patch for it a little while
ago, and just wondered if you'd looked at the existing code).
- If you are do you have any idea how much your patch will impact the
existing work?
- How do you see supporting multiple databases progressing?
I've been really busy lately, but if you think it might help, I can update
the branch with the latest changes from trunk and either fire you a patch or
try and get the mirrored SVK repo somewhere public...
Cheers,
Ben
On 15/08/07, Brian Harring <[EMAIL PROTECTED]> wrote:
>
> On Tue, Aug 14, 2007 at 04:36:22PM -0500, Adrian Holovaty wrote:
> > After a cursory read-through the patch, my only major complaint is the
> > unnecessary cleverness of some of the code.
>
> Alt example besides BackendOps/BackendCapabilities would be
> appreciated (your phrasing implies there is more); those two classes
> are frankly borderline frankenstein at the moment due to continual
> mangling.
>
>
> > The stuff in BackendOps,
> > for example, could be rewritten in a much clearer way. (Granted, it
> > would be longer, but that's an acceptable tradeoff, in my opinion.)
> >
> > For example, this is in the patch:
> >
> > class BackendOps(object):
> > required_methods = ["%s_sql" % x for x in
> > ('autoinc', 'date_extract', 'date_trunc', 'datetime_cast',
> > 'deferrable', 'drop_foreignkey', 'fulltext_search',
> 'limit_offset',
> > 'random_function', 'start_transaction', 'tablespace')]
> > required_methods += ['last_insert_id', 'max_name_length',
> > 'pk_default_value', 'sequence_name', 'get_trigger_name']
> > required_methods = ["get_%s" % x for x in required_methods]
> > required_methods = tuple(required_methods)
> >
> > And this is a much, much clearer way of expressing exactly the same
> thing:
> >
> > class BackendOps(object):
> > required_methods = (
> > 'get_autoinc_sql',
> > 'get_date_extract_sql',
> > 'get_date_trunc_sql',
> > 'get_datetime_cast_sql',
> > 'get_deferrable_sql',
> > 'get_drop_foreignkey_sql',
> > 'get_fulltext_search_sql',
> > 'get_limit_offset_sql',
> > 'get_random_function_sql',
> > 'get_start_transaction_sql',
> > 'get_tablespace_sql',
> > 'get_last_insert_id',
> > 'get_max_name_length',
> > 'get_pk_default_value',
> > 'get_sequence_name',
> > 'get_get_trigger_name'
> > )
> >
> > Not only is it clearer, but it saves us from having to do calculations
> > at run time.
>
> While that may skip a calculation, honestly surprised there is no
> complaint about 2-10 lines below where it's shoving a curried func
> into locals (half expected a knee jerk "yuck" on that one ;).
>
> As indicated further up, those two classes will need another cleanup;
> the currying/locals I'd advocate leaving in, but expanding
> required_methods there is definitely doable- it's form right now is
> just a left over from how I went about pinning down what ops were
> 'standard' for backends. Crap like that will be removed next time I
> do a full walk of the changes (this round is basically a "this is the
> api I'm proposing, and here is a working patch for the api").
>
> So... any spots that don't read as just idiocy? Specific complaints
> about the presented API, seperation, etc, is what I'd like to get if
> possible- still have a lot of gutting to do internally anyways, so the
> form of it ought to improve.
>
> ~harring
>
>
--
Regards,
Ben Ford
[EMAIL PROTECTED]
+628111880346
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Objet Dj'art
To clarify, are you trying to develop a script which will take a database
and from that create all of the necessary django stuff to enable you to fire
up a browser and look at the database in one step..?
One thing you might want to look at is using SQLAlchemy to introspect the
database, it should be a fairy simple to translate the structure of a Table
(SA) instance to a dynamically created Model class. Check out Marty's
blog<http://gulopine.gamemusic.org/2007/08/dynamic-models-in-real-world.html>and
this <http://code.djangoproject.com/wiki/DynamicModels> from the django wiki
for some tips.
Also if you're going to look at deploying this as a kind of standalone
thing, why not just supply an sqlite db file with it with a superuser and
all the django stuff already created..? This would be relatively easy to do
with multiple-db and you could also provide a barebones settings.py and the
requisite app/project structure to chuck your newly created models into.
Neat idea! :-)
Ben
On 24/08/07, Felix Ingram <[EMAIL PROTECTED]> wrote:
>
>
> Here's a little something that I've hacked up, which I'm posting here
> to see whether anyone can take it any further.
>
> After seeing Jacob use Django to inspect his iPhone's databases I
> wondered whether some sort of automatic database browser could be
> scripted up. Here's the simple example I got working:
>
> 1. First make a new project (db_test) and app (my_big_fat_fake_app).
> 2. Configure the settings file to point at the database you'd like to
> inspect.
> 3. Add admin and my_big_fat_fake_app to the INSTALLED_APPS.
> 4. Create an inspect.py file with the following in it:
>
> import sys, os
>
> from django.core.management.commands import inspectdb, runserver, syncdb
> from django.core.management import setup_environ
>
> import settings
> setup_environ(settings)
>
> c = inspectdb.Command()
> s = c.handle_inspection()
>
> new_lines = []
> for line in s:
> if "class Meta" in line:
> new_lines.append("class Admin:")
> new_lines.append("pass")
> new_lines.append("class Meta:")
> new_lines.append("app_label='my_big_fat_fake_app'")
> else:
> new_lines.append(line)
> lines = "\n".join(new_lines)
> a = compile(lines, "", "exec")
> exec(a)
>
> s = syncdb.Command()
> s.execute()
>
> r = runserver.Command()
> r.execute()
>
> 5. Run the file (python inspect.py)
> 6. This should create all of the Django tables in your database,
> prompt you to create a superuser and then launch the development
> server. You should then be able to log in and browse and edit the
> database.
>
> Problems:
> 1. Running it twice will add all of the Django tables to the admin index.
>
> Future features:
> These are the things I'd like to implement but I'm getting stuck on:
> 1. Prompt the user for the database string (I'm getting stuck tweaking
> the settings module).
> 2. Auto-create a superuser (use a default or take it from the command
> line).
> 3. Eliminate the need to create a project and app.
> I've tried dynamically creating urls and settings and can get the
> admin interface up, but no models are shown (this seems to be because
> the fake app I created isn't recognised by the admin). I've been using
> the 'new' module but quickly started to bump up against my lack of
> python-internals knowledge.
> 4. (This is the magic one) Use the multi-db branch to create the
> Django and admin tables in a sqlite://:memory: database.
> If this was possible then you could point the script at any database,
> browse and edit, and leave it otherwise untouched. Which I think would
> be pretty sweet.
>
> I've posted this to developers as I think it requires some in depth
> knowledge of how Django does its job. Apologies if it's a bit off
> topic.
>
> Hope this inspires someone.
>
> Regards,
>
> Felix
>
> P.S. Apologies for the pun in the subject line: it's the best Django
> joke I could come up with (thank you http://thesaurus.reference.com/)
>
> >
>
--
Regards,
Ben Ford
[EMAIL PROTECTED]
+628111880346
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re The sprint
Hi folks, I think I can be free for most of Friday (Indonesian time GMT+6). I'd like to suggest focusing my time to get the multi-db branch as up to date as possible, I would imagine that up to the backend refactoring at the very least, it's up to 5589 on my machine right now so an extra push to 5949 should be easily achievable! I think it would be a really worthwhile aim to get all of this into the repo so that the many people who have expressed an interest can get access to an up to date branch and start to contribute or just use it in their projects. I haven't examined the changes to backend (starting at r:5949) in any detail, but it would seem logical that this is going to impact multiple-db pretty heavily. Therefore a secondary aim should probably be to think about a redesign of the mechanics of multiple-db to take advantage of these changes. If we can get this stuff into the repo, I'm perfectly happy to put myself forward as maintainer for the branch and to make sure it stays up to date in the future. Thanks, Ben -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re The sprint
Thanks for the responses guys. Russ what is your feeling about getting multi-db into the repo so that people can then use it? I'm happy to do the work that I mentioned above in merging the branch up to date to the point of the backend refactor in trunk, and after that to start exploring re-factoring the multi-db to leverage some of the later changes in trunk. But I'd really like to ensure that I can send the patch to someone and thence get it into the repo. Once we're to that point maybe we can re-explore the possibilities of me having commit access to the branch and acting as maintainer. If that's still a no goer, then I will look at setting up a BZR/SVK/some other public means for people who have an interest in taking the branch forward. Cheers, Ben On 11/09/2007, identify <[EMAIL PROTECTED]> wrote: > > > Hi > > > No problems. If I was too harsh, I apologize. You're not alone in > > wanting this feature - if you search the archives, there is a > > reasonable number of people that have expressed an interest - however, > > this is one of those features that people want, but very few are > > willing to help out with. After a while, the repeated 'This is a must > > have feature' messages from people that don't offer to help start to > > get on the nerves. > > I know this scenario very well, believe me. > > > Welcome to the pool. The water is great! :-) > > I didn't got out since the last three weeks. :-) > > > In case you missed the announcement - we're holding a sprint this > > Friday. If you want to get your feet wet with the Django internals, > > why not join in and try your hand at fixing some smaller bugs? I'm > > sure someone on the day will be able to suggest some suitable > > candidates. > > I've read the announce. Well - unfortunatetely it IS a friday, what's > a normal working day for me. But maybe there's time at the evening. > I'll see... > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re The sprint
Thanks for the input and the info. I'll have a look at those issues and hopefully get them sorted before the sprint. Ben On 12/09/2007, Russell Keith-Magee <[EMAIL PROTECTED]> wrote: > > > On 9/12/07, Ben Ford <[EMAIL PROTECTED]> wrote: > > Thanks for the responses guys. > > Russ what is your feeling about getting multi-db into the repo so that > > people can then use it? I'm happy to do the work that I mentioned above > in > > merging the branch up to date to the point of the backend refactor in > trunk, > > and after that to start exploring re-factoring the multi-db to leverage > some > > of the later changes in trunk. But I'd really like to ensure that I can > send > > the patch to someone and thence get it into the repo. > > Is the diff on #4747 the latest version of the patch that you have? If > so, I have a few reservations about applying this diff. > > Just taking the first few changes in the patch (django/test/utils.py) > as an example: > - The very first change is a multi-line import, which, while legal, > isn't good form. > - The next block of changes involve introducing an > instrumented_test_iter_render, which doesn't exist in the current > trunk. > - The next change (around line 117 of the new version) hard codes the > location of a database file in your personal home directory. > > I haven't worked through the full 200k patch, but if the first few > changes are any indication, there is still work to do before this gets > merged. > > There are two sets of changes that need to be applied: merges from > trunk, and new code. These patches need to be kept separate for commit > purposes, and each new idea should be a separate patch, committed > separately. From all appearances, the patch on #4747 is a mix of both > merge changes and new features. > > I'm happy to apply any patches that are raw merges from trunk. If you > also have some fixes/enhancements that go beyond trunk merges, I'm > happy to look at them, although they will have to meet a basic quality > standard. > > > Once we're to that > > point maybe we can re-explore the possibilities of me having commit > access > > to the branch and acting as maintainer. > > That's an issue you'll have to take up with Jacob and Adrian - they're > the ones that control access to SVN. However, as I have said many > times before, access to the repository is generally restricted to > those that have an established track record. > > Regarding using some sort of distributed version control - there is a > semi-official Mercurial repository available; it's been mentioned in > the developers list a few times. > > Yours, > Russ Magee %-) > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +628111880346 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Fwd: Ticket 4747 (multi-db)
Hi folks, I'm forwarding this for all of the people who have expressed an interest in multi-db over the last few days/weeks. Koen has made the fantastic effort of updating the branch from my changes ( to r5589) up to r6110. I've attached that patch to ticket 4747 <http://code.djangoproject.com/ticket/4747> which refers to the effort to update multi-db. I haven't tested this patch myself, and I'm not in a position to do any work on the branch for the next week or so as I have a project that I need to finish first (and I'm also moving house!). I will be around on the mailing list as much as possible and will attempt to clear up any questions arising from the use of the patch if I can. If any of you folks who have expressed an interest in multi-db have any time it would be great if you could have a look at this and lend a hand to the outstanding issues (especially testing!). Some of my own changes were a little bit ropey to be honest... I did a poor job of separating merges from my own changes in support of the project I was working on. If we can get this patch up to a standard that a core dev is willing to check it into the SVN repo branch it will obviously make it easier for people to share/use/develop further, which can only be a good thing!! Thanks Ben -- Forwarded message -- From: BIERMANS Koen <[EMAIL PROTECTED]> Date: 14 Sep 2007 21:52 Subject: RE: Ticket 4747 To: Ben Ford <[EMAIL PROTECTED]> Hi Ben, In annexe a diff of my multidb folder compared to trunk r6110. Some background: * removed (most of) the schema evolution stuff (also the tests) * get_db_prep_save gets the model instance so it can determine what backend it uses (needed for mysql exception in datetimefield) * queryset gets set in manager since some backends have custom querysets (eg oracle) * (hopefully) all quote_name references have been resolved to the models connection ops via model._default_manager.db.connection.ops.quote_name * query.py: quote_only_if_word uses the backends quote_name (passed in) * management stuff: syncdb put model connection determination all around, seems to work, needs testing * validation: needs a closer look * sql: table_list ok, create ok, delete ?, flush NOT OK, others TO DO * thread_isolation test failing : TODO * test_client test failing (flush?): TODO This means Unicode, backend and management refactoring should be more or less ok. There are some things to look at, but I need a break now. I have not done to much testing yet though. I'll see when I can get round to that (maybe later this evening when the kids are asleep). I hope this helps. Koen -- Regards, Ben Ford [EMAIL PROTECTED] +6281317958862 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Oracle slicing and backend refactor
Hi there, I've just been looking at some of django's internals trying to get my head round the backend refactor... I seem to have no luck slicing oracle flavoured QuerySets. From reading the source it seems that the limit_offset_sql logic is going to be moved to a separate module (called query.py in the backend) but this file doesn't exist. There's also what appears to be a lot of limit and offset looking code in the _get_sql_clause function of the OracleQuerySet class. I'm not at all familiar with oracle I'm afraid, but am I right in thinking that this stuff should live in the limit_offset_sql function of the DatabaseOperations class and just needs to be moved? Or is there a deeper issue causing problems? Thanks Ben -- Regards, Ben Ford [EMAIL PROTECTED] +6281317958862 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Oracle slicing and backend refactor
Hi guys, Thanks for the reply. So if my only recourse at the moment to execute qs._get_data() and then work with a slice as a list..? If there's any way around that I'd like to hear it, my models are in a legacy DB, and it's HUGE!! Is anyone using some method to get round this at the moment? Cheers, Ben On 04/10/2007, Matt Boersma <[EMAIL PROTECTED]> wrote: > > > On Oct 3, 6:23 am, Malcolm Tredinnick <[EMAIL PROTECTED]> > wrote: > > Oracle changes on the queryset-refactor branch are probably going to be > > about the last thing to land on that branch because it's very time > > consuming to test and work with for me. > > It sure is--we tend to run only single tests during the day, and the > entire suite (against all backends) nightly via a cron job. If > there's any help with Oracle testing or coding we could provide on the > queryset-refactor branch, please don't hesitate to ask. > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +6281317958862 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Oracle slicing and backend refactor
Hi Ian, OracleQuerySets currently do not have any slicing ability AFAIK, apart from returning a slice of the qs._result_cache list... in the backend refactor all of the logic for building the SQL is in a function called limit_offset_sql on the DatabaseOperations class. This is overridden in oracle/base.py to return ''. it seems this is going to be addressed in the queryset refactor coming shortly. Ben On 04/10/2007, Ian Kelly <[EMAIL PROTECTED]> wrote: > > > On 10/3/07, Ben Ford <[EMAIL PROTECTED]> wrote: > > Hi guys, > > Thanks for the reply. So if my only recourse at the moment to execute > > qs._get_data() and then work with a slice as a list..? If there's any > way > > around that I'd like to hear it, my models are in a legacy DB, and it's > > HUGE!! Is anyone using some method to get round this at the moment? > > In trunk, OracleQuerySet slices should work just the same as regular > QuerySet slices. If that's not the case, then there's a bug. What > problem is it that you're having? > > Ian > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +6281317958862 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Oracle slicing and backend refactor
Hi Ian, There was some funky code in the branch I'm using and query.py wasn't getting the correct subclass of _QuerySet. Ben On 04/10/2007, Ian Kelly <[EMAIL PROTECTED]> wrote: > > > On 10/3/07, Ben Ford <[EMAIL PROTECTED]> wrote: > > Hi Ian, > > OracleQuerySets currently do not have any slicing ability AFAIK, apart > from > > returning a slice of the qs._result_cache list... in the backend > refactor > > all of the logic for building the SQL is in a function called > > limit_offset_sql on the DatabaseOperations class. This is overridden in > > oracle/base.py to return ''. it seems this is going to be addressed in > the > > queryset refactor coming shortly. > > Ben > > Ben, > > I don't know what it will look like after the refactor, but in the > current trunk, the limit_offset_sql function isn't flexible enough to > do the level of query rewriting that's necessary to support the > feature in Oracle, and so it's done by directly overriding > QuerySet._get_sql_clause instead (the comment about handling > limit/offset in oracle/query.py is erroneous). Supporting > limit/offset was the primary reason for subclassing QuerySet in the > oracle backend in the first place, so if there's a bug relating to it > in trunk, I'd like to hear about it. > > Ian > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +6281317958862 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django + SQL Alchemy
Hi Paul, I'd love to have time to play with this, but I don't at the moment. It's something I've been interested in for a while though, and also something that I want to take a long hard look at pretty soon. Have you seen: http://jeroen.concept-q.com/projects/djangoalchemy/? Maybe something you could look at in the spirit of DRY if you're both doing the same thing. How do you feel about mapping Tables directly to django Models...? I.E: db_table = ._meta.db_table sastring = (I have a function to build this from settings.py) engine = create_engine(sastring) sa_table = Table(db_table, engine, autoload=True) mapper(, sa_table) You can now use .c to get access to all of the SA stuff, and do funky things with joins and what-not that you can't with django ORM. But all of the objects that the resulting query returns are Model instances so all of your django functionality (admin, etc) still works. This is kinda off the top of my head, and I've only used it superficially with no testing at all, but on first inspection it appears to work. what do you think? Ben On 05/10/2007, Paul Davis <[EMAIL PROTECTED]> wrote: > > > Before anyone gets to excited, I'm going to state up front that I have > *not* replaced django's ORM. > > But I do have a small project that gives proof of concept for > automatically integrating SQL Alchemy ORM into the Django ORM. I've > been kicking the idea around for a bit, and after going through the > code I felt it was fairly hopeless to actually try and replace the > entire Django ORM with an sqlalchemy back end. > > Major key points: (remember this is PoC so suggestions are indeed welcome) > > SA = SQLAlchemy > > 1. SA creates its Table objects using reflection > 2. SA ORM classes are dynamically created > 3. SA ORM classes are directly importable via tranquil.models.* > 4. I haven't got the ability to fully configure the setup of SA > tables, ORM classes, and mappers yet. This is planned, but I felt like > it wasn't important for PoC > 5. Currently, SA reflection happens every time the server starts. > Obviously, caching this is on the ToDo list > 6. All of this is provided by an external app with no patches to Django > itself. > > Project Page: http://code.google.com/p/tranquil/ > Example views.py: > http://tranquil.googlecode.com/svn/trunk/proj_name/app_name/views.py > > Basic usage: > Assuming tranquil is in the installed apps, just do: > > from tranquil.models.app_name import ModelName > > ModelName is the SQLAlchemy ORM object which means its pure SQLAlchemy > from there. > > Any feedback is welcome. If there's enough interest I'll move forward > with filling out the corners. > > Thanks, > Paul Davis > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +6281317958862 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django + SQL Alchemy
Hi Paul, I don't think it would entail two db lookups, but like I said I haven't tested it. SA will be instantiating a django Model, but it will be filling in it's data from the database, so there should be no lookup required by django. It does overwrite the field as far as I know, but that shouldn't matter in because all you get from a field is that field's value from the database anyway. The django ORM is still usable... So you still get the ability to do reverse related lookups and such, even from objects in a list returned by SA: i.e. if you have the following class Alarm(models.Model): ran_obj = models.ForeignKey(RANObject, related_name='alarms') . and you get a list of RANObjects from some complicated lookup that you use SA for, then you should still be able to do: ran_ob.alarms.all() This is one case where I could envisage using SA and django ORM side by side. That way you won't have to mess with any of your views/admin/django core functionality, as the underlying objects are still just django models. What do you think? Ben On 06/10/2007, Paul Davis <[EMAIL PROTECTED]> wrote: > > > SmileyChris, > > At the moment, I'm using the class_prepared signal to gather all > loaded models. Theoretically I could use this to attach an SA > attribute. > > Are there use cases out there where you'd want to use *both* the > Django and SA ORM's side by side? To me it makes more sense to import > one or the other depending on what you plan on doing. For instance all > of my code will use just the SA models, but the admin will still use > the Django models. > > Paul > > On 10/5/07, SmileyChris <[EMAIL PROTECTED]> wrote: > > > > > > > > On Oct 6, 5:24 am, "Paul Davis" <[EMAIL PROTECTED]> wrote: > > > Ben, > > > > > > I contemplated something like this, but I assumed that field > > > properties would clash. For instance, after the call to mapper(...) > > > you would no longer have access to Django ORM fields with the > > > Model.field syntax (as they would be replaced by SA's). > > > > Could you use the post_init signal to attach a LazySAModel object as > > an SA (or something) property of all newly created Django models? > > > > > > > > > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +6281317958862 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django + SQL Alchemy
Hi Paul, Model field values are just the value of that field in the database, *not* the field itself. The fields themselves are atored in _meta.fields and _meta.many_to_many. The field does all of it's work by looking at the model, i.e. getattr(model_instance, field_name). So it makes no difference if you change the value. Therefore when a model is created, it should make no difference if the field value are set by django or SA... as long as they are the same then there's no problem. This is a bit of a superficial explanation, as of course it makes no allowance for the case where the db_column has been set in the field and is therefore different to the attribute name applied to the model. However I think the theory is sound. Ben On 07/10/2007, Paul Davis <[EMAIL PROTECTED]> wrote: > > > Ben, > > If I understand you right, you're arguing to merge the SA and Django > models so that we can use Django's reverse lookups. > > SA supports reverse lookups and I'm just getting started on > implementing these automagically by introspecting the django models. > > I'm still not convinced that overriding Django model fields with SA > model fields is a good idea. There seems to be quite a bit of code in > the Django fields for dealing with forms and what not. I'm assuming > that this is an artifact of oldfields and will be removed eventually, > but until then all code depending on oldfields would choke. Something > I'm trying to avoid. > > The only reasons that I can come up with supporting using Django > models as a base would be to try and automatically use the extra > methods defined for a model or *maybe* using the custom managers. > > Both of these 'features' seem like they would present an endless can > of worms because of all the implicit relationships between the django > models, fields, and managers. > > At the moment, I'm trying to keep things as straightforward as > possible so its easy to switch back and forth between the two model > styles. > > Anyway, thanks for your thoughts. Hopefully I can introspect all the > relationships and have back references working automatically before I > go home tonight. > > Paul > > On 10/5/07, Ben Ford <[EMAIL PROTECTED]> wrote: > > Hi Paul, > > > > I don't think it would entail two db lookups, but like I said I haven't > > tested it. SA will be instantiating a django Model, but it will be > filling > > in it's data from the database, so there should be no lookup required by > > django. > > > > It does overwrite the field as far as I know, but that shouldn't matter > in > > because all you get from a field is that field's value from the database > > anyway. The django ORM is still usable... So you still get the ability > to do > > reverse related lookups and such, even from objects in a list returned > by > > SA: i.e. if you have the following > > > > class Alarm(models.Model): > > > > ran_obj = models.ForeignKey(RANObject, related_name='alarms') > > . > > > > and you get a list of RANObjects from some complicated lookup that you > use > > SA for, then you should still be able to do: > > > > ran_ob.alarms.all() > > > > This is one case where I could envisage using SA and django ORM side by > > side. > > That way you won't have to mess with any of your views/admin/django core > > functionality, as the underlying objects are still just django models. > > > > What do you think? > > > > Ben > > > > On 06/10/2007, Paul Davis <[EMAIL PROTECTED]> wrote: > > > > > > SmileyChris, > > > > > > At the moment, I'm using the class_prepared signal to gather all > > > loaded models. Theoretically I could use this to attach an SA > > > attribute. > > > > > > Are there use cases out there where you'd want to use *both* the > > > Django and SA ORM's side by side? To me it makes more sense to import > > > one or the other depending on what you plan on doing. For instance all > > > of my code will use just the SA models, but the admin will still use > > > the Django models. > > > > > > Paul > > > > > > On 10/5/07, SmileyChris < [EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > > > On Oct 6, 5:24 am, "Paul Davis" <[EMAIL PROTECTED]> wrote: > > > > > Ben, > > > > > > > > > > I contemplated something like this, but I assumed that field > > > > > prope
Fwd: Multiple DB-Support at django
Sorry for the forward, forgot to add the dev group when I sent the mail! -- Forwarded message -- From: Ben Ford <[EMAIL PROTECTED]> Date: 11 Oct 2007 10:14 Subject: Re: Multiple DB-Support at django To: BIERMANS Koen <[EMAIL PROTECTED]> Cc: "Adam, Mario AD/LGN-IL" <[EMAIL PROTECTED]> Hi Guys, Yes it just so happens that I've been working on just this issue. Yesterday I put my work so far into a djangosnippet here<http://http://www.djangosnippets.org/snippets/442/>. It relies on a couple of changes that I've made to my copy of the multi-db trunk, ( e.g as an interim measure model_connection_name now checks for a db attribute on the model itself which is a key in django.db._init__.connections). I believe that we should ditch most of the logic currently in db.__init__.. A lot of it has now been negated by the stuff in the backend refactor from what I can see... Maybe we should just think about starting again from a vanilla trunk We should provide the ability to regester models with different databases (similar to how templatetags works), maybe in a separate module (project/connections.py perhaps) with the database defaulting to the details found in the normal setting module. I think that would work nicely for all cases, if you have a model in more than one db, just register it more than once with different connection settings. With the backend refactor it should be relatively simple to map models to DatabaseWrappers. I also agree with you Koen that the reference to the database would probably be better attached to the model somehow. Mario, The snippet mentioned above probably won't work as is with the existing copy of multi-db, I have had to make a couple of changes. Also it's undocumented and I didn't write tests for it. (yes yes i know, but I've been under a lot of time constraint and I haven't had time to learn to be productive with TDD yet - next project for sure :-)! ) But I use it quite happily with my multiple databases with the same schema and it solves my problems perfectly :-) django-dev, FYI and for any comments. Cheers Ben On 10/10/2007, BIERMANS Koen <[EMAIL PROTECTED] > wrote: > > *Ben, I put you in CC because I think you know more about this than I do. > * > > > > Hi Mario, > > > > I am glad it is working. > > > > About foreign keys to other databases: we will have to wait for queryset > refactoring to land before we can see if that can be solved I think. You are > best off doing the queries separately yourself for now. > > > > As to using different connections for one model: (I may be playing out of > my league trying to answer this, so forgive me for any inaccuracies): > > > > The connections are used in query.py via the manager of the model > (connection = model._default_manager.db.connection). > > > > The "db" var is a ConnectionInfoDescriptor object from db.__init__.py. > > > > The nice thing about this object is that you can simply "SET" this to any > named connection you want (use the "connections" proxy to get them. The only > thing that bugs me is the _default connection (I don"t know why this one was > implemented the way it is via this dummy object, but you can just import > _default and call connections[_default]). > > Beware: the connection that is linked to this manager is reset after every > request via the request_finished signal. > > > > The connections themselves are lazy: they are only really connected when > there is a query to perform (django does not do any connection pooling > beyond the request). > > > > So my guess is: yes, you can define all your databases in your settings > files (OTHER_DATABASES, no need to add models) and set the correct > connection to your models at the start of every request, eg via middleware. > ( something like modelX._default_manager.db = connections['connectionY'] > should work I think). > > > > */me takes a note: we should be passing the connection to the query > instead of using the _**default** _manager's one, that way you could > simply define different managers (eg modelx.objects.all() uses the default > connection, modelx.objectsfromY.all() uses the 'connectionY' connection > etc...(Ben, can we do this when we define the queryset class for the models > manager? I'll need to look at this.)* > > > > I have however no experience with all this, maybe Ben has more info on > this ? (I seem to remember him using it to copy data from one db to another > ?) > > > > Koen > > > -- > > *Van:* Adam, Mario AD/LGN-IL [mailto:[EMAIL PROTECTED] > *Verzonden:* woensdag 10 oktober 2007 10:21 > *Aan:* BIERMANS Koen >
Re: multi-db branch
Hi Ed, There are a few people looking at multi-db at the moment. I've had to put my own contributions on hold because I've had rather a hectic personal life lately, but hopefully I'll be able to crack on with it next week or so. There has also been some discussion about rethinking some of the design of multi-db, especially in light of recent changes to the way backends work and forthcoming changes to querying, which is something I'd like to have a look at when I get back into it. Watch this space at the moment it seems! :-) Cheers Ben On 31/10/2007, Ed Summers <[EMAIL PROTECTED]> wrote: > > > On Oct 29, 3:31 pm, "James Bennett" <[EMAIL PROTECTED]> wrote: > > You probably want to start by reading messages here: > > > > http://groups.google.com/group/django-developers/search?q=multi-db&st... > > So it sounds as if the branch is dead, and there are patches being > attached to #4747? I'd be willing to pitch in some time to get the > multi-db code up to snuff, if there is a clear road ahead for what > needs to be done. If there isn't I can live with that too :-) > > //Ed > > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +6281317958862 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: S3 file storage
Hi Marty, What's your feeling about how much work is involved? I might be in a position to help out and his is something that I'd love to see! Ben On 16/01/2008, Marty Alchin <[EMAIL PROTECTED]> wrote: > > > Hey all, > > I finally got around to creating a ticket[1] for an S3 file storage > backend. Yay! Of course, it's very, very broken at the moment. I > haven't touched it since the last sprint, where I was trying to switch > from Amazon's own library to boto. > > I don't use S3 though, so even if I did finish it myself, I couldn't > test it. So, in the spirit of open source, I'm desperately asking for > help. Now that there's a ticket, anybody with S3 know-how who's been > following #5361 can run with it. > > I've gotten a couple private emails offering assistance with it, so > I'm directing this email directly to those people as well, in case > they're not reading django-developers at the moment. I don't really > care who takes it on, just as long as people aren't waiting on me for > it anymore. > > -Gul > > [1] http://code.djangoproject.com/ticket/6390 > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447503145951 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Is it a good thing to continue using django multi-db-support ?
I second this opinion unfortunately I did bring multi-db up to HEAD previously, the results of which are in #4747 as Russ said. I'm very busy on a new project at the moment, but multi-db in django is something I definitely want to revisit in the future. On the advice of a few people (and due to work commitments) this is probably won't be something I'll be doing for a little while. It's almost certain to be easier after queryset-refactor lands anyway. My advice at present would be to use SQLAlchemy as your ORM if you absolutely need to access multiple databases in the short term. It's extremely powerful, and I believe there are some nice declarative type syntax wrappers for it floating around too. You could also use django db stuff for the core of your app and just use SA for the other db's (you wouldn't have to chuck out django's ORM completely if you don;t want). Multi-db as it is right now will probably never get back to being in sync with trunk. When I revisit it I'm really aiming at a rewrite from scratch so that effort won't be backwards compatible to multi-db in it's present form. Cheers, Ben On 18/02/2008, Russell Keith-Magee <[EMAIL PROTECTED]> wrote: > > > On Feb 18, 2008 7:10 PM, Grindizer <[EMAIL PROTECTED]> wrote: > > > > Hi everybody. > > > > We are actually trying to use Django multi-db support for internal > > need in our group. We used this branch because we have historical > > constraint (we build a Django app over an old db application which use > > several databases) > ... > > So do you thing that it is a good idea to continue using this branch > > for a real project ? > > First off, this isn't a question for django-developers. > Django-developers is for answering questions about the development of > Django itself - user queries (how do I/should I type questions) should > be directed to django-users. > > However, that said: the decision to use multi-db is entirely up to > you, your capabilities and your expectations. The short answer is > "probably not". > > Multi-db isn't part of the Django core. Nor is it one of the branches > targeted for inclusion in v1.0 (those two branches are newforms-admin > and queryset-refactor). This means that while it may be a useful > feature (for some - certainly not me personally), multi-db isn't an > immediate priority for the Django core developers. > > The branch has not been updated for some time - this is because the > original maintainer has not been updating the branch. A number of > people have offered to help maintain the branch. If you read the > mailing list that refer to multi-db, you will find a consistent > response - you don't need commit access to work on this branch, just > work on the ticket in your own SVN checkout and upload tickets that > implement the changes required by the branch. > > A number of people have gone down this path, and ticket #4747 has been > the ticket where the patches congregate. However, these patches are > currently 5 months and almost 800 revisions behind trunk at present. > Last I heard, there were still many issues to be resolved, and I > suspect that the upcoming queryset-refactor merge will cause all sorts > of headaches for this branch. > > If you are willing to take on the challenge of handling any bugs that > you find, and managing the merge process for any new features from > trunk that you want, then you will be able to use multi-db. However, > if you're just looking for a tool to use 'no questions asked', > multi-db probably isn't the way to go. > > Yours, > Russ Magee %-) > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
Hi all, I'll sort out the hg repo (it now needs to point at trunk - not qsrf) and trac project if I get time this evening and make it public readable for everyone who's interested. Is there a ticket in django we could use to track progress on this? We could use 4747, but if we do decide on a new API that might be a bit confusing... We could of course just use the mailing list and trac project, thoughts? It's great to see some interest in multiple db support again :-) Ben 2008/5/20 Nicola Larosa (tekNico) <[EMAIL PROTECTED]>: > > Daryl Spitzer wrote: > > If I don't, I see if I can at least make enough time to write up the API > > I came up with at PyCon. > > Please do, that would be great. > > -- > Nicola Larosa - http://www.teknico.net/ > > > > -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
Hi Jacob, I'd be interested in your thoughts on a declarative approach to defining the other databases...? I'll have my mercurial repo synced up to trunk tomorrow (my time) and I'll re-apply the patch I got from Daryl to it as a starting point. Hopefully people will be able to have a look through it and compare the declarative approach proposed with the existing multi-db approach. As far as I can tell, you've currently provided two hooks to use a > secondary connection: set the model's default connection in the > settings module (which is OK, I suppose, though I might want to > nitpick the syntax a bit), and assigning to ``Model.objects.db``. > > This second one is a disaster waiting to happen -- you've had to muddy > things up with threadlocals to work around some problems already. Also > consider the "bookkeeping" you'd need to do to deal with objects > across multiple database simultaneously (think sharding). You'd have > to keep juggling ``Model.objects.db`` and saving old ones... ugh. I built a non trivial application with multi-db as it is right now and found the api to be a bit hairy to be honest. I think it would be an advantage, especially in a "database rich" environment to be able to build a db on the fly much like a model, rather than be tied to what's in a dict in settings. I don't really like the objects.db way of doing it, and I found myself doing a fair bit of hacking to get it to work. > * There needs to be an official API to get a model (or perhaps a > manager) which references a different "context" -- > ``Model.objects.db`` should be read-only. So you'd call some API > method, and get back a sort of proxy object that uses the other > connection. Here's a strawman API:: > > >>> from django import db > >>> from someapp.models import Article > > >>> Article.objects.all() > [... all Articles from the default database ...] > > >>> ArticlesOnOtherDatabase = db.get_model_for_other_connection(Article, "private") > >>> ArticlesOnOtherDatabase.objects.all() > [... all Articles from the database defined with the "private" key ...] Agreed, the way I got round this was to build the model again from scratch each time I wanted to access objects in a different database and have the dynamicaly created model persist in the app cache. I took most of this from the dynamic models entry on the wiki, it's here, look in the duplicate_model function: http://www.djangosnippets.org/snippets/442/ This would really need work (especially the field copying code, which is fairly horrifying! I know that doesn't work for all field types too - yuk) before it becomes a 'good idea', and I'm not even sure it's the right way to go, however I'd be interested in weather people think it's a valid approach. * I'd like the default connection for each and every object to be the > default database forever and always. I find putting models for default > connections in settings distasteful and I'd rather just a single API > for changing the connection (see below). However, I imagine I'll be in > the minority here so I'm prepared to cede this point if necessary. The API which I think is being proposed is that there should be a central register for database connections. In my mind this would be the place to go to get hold of a connection for use in a queryset (and all the other places it's needed) and I think the correct default behaviour of the class/object would be to return the connection defined in settings.DATABASE_*. The code to build the declarative DatabaseWrapper is already there, and there a method to build one of these from what's in settings too. This should make it easy to get hold of connection in all of the places where we currently do "from django.db import connection". It's great to see this revived again :-) Cheers Ben -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
Hi all, I've now re-applied Daryls patch (which was against qsrf) to a clone of django trunk in a mercurial repo. It's available at http://hg.woe-beti.deand there's a trac set up for it at http://trac.woe-beti.de. Feel free to make use of both of these. Although I've disabled to ability to create tickets perhaps the wiki might be a good place to discuss the API? Anyone can clone from the hg repo, give me a shout if you would like push access and I'll sort it out. Cheers, Ben -- Regards, Ben Ford [EMAIL PROTECTED] +447792598685 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
