Re: Model validation incompatibility with existing Django idioms
On 8 jan, 10:03, James Bennett wrote: > Suppose I have a ModelForm and call save(commit=False) to get the > instance so I can do some more work on it. I'm basically saying to > Django "I don't think this object is ready to be saved yet, but I need > the object so I can do stuff to it". If Django goes ahead and does > implicit model validation there and raises an exception, it's just > telling me what I already knew: the object's not ready to be saved. > But now I get to do extra work to catch the exception, and that's bad > and wrong. > > Calling ModelForm.save(commit=False) should simply disable model > validation, period. If I want to validate the instance before saving > it, I'll validate the instance before saving it, but commit=False is > as clear a way as we have of saying "I'm not going to save this yet" > and model validation should respect that. > The problem is that in the idiom the is_valid() call on the modelform tries to full_validate the instance, it is not in the save (commit=False) call. I'm with Simon here: on an incomplete modelform, let's just ignore the errors on excluded fields and only validate the user input at that point. If the developer wants to be sure the model validates after he adds the necessary extra information, it is his job to call validation before saving. Koen -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Help on solving ticket #2522 and #2470
I am also new at django. In my opinion the basic problem of 2522 is that the related object is referenced by the objects name instead of the related name. As a result two foreign keys to the same object get the same name. This reference is spread throughout related.py and other modules (like get_follow in options.py and others), so I haven't had the heart yet to try and change all these references to fix it. Maybe there is an easier way ? I haven't looked into 2470 (that one did not bite me yet). k --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: django.contrib.formtools: High-level abstractions of common form tasks
Isn't it useful to allow two-step validation in general: one by the form and one by the model when saving (both optional of course). This would allow to add constraints to a form that are not needed by the model in general (interesting when using different forms on the same model eg for different levels of authenticated users). It would be easy to expand this for the wizard-style forms: each page has its form validation (only the 'new' items) AND a validation by the model when saving to the database (on everything). Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Custom permissions on proxy model no longer created in 1.4
Hi all,
this could very well be one of those "don't do that" things, but here is my
problem.
I have been using some specific permissions concerning the auth user model,
so I created a proxy model on user like this:
class User(auth_models.User):
class Meta:
proxy = True
permissions = (
("display_users", "May display users information"),
("edit_users", "May edit users information"),
)
In 1.3 these custom permissions were created during syncdb (linked to the
auth.User model).
Now I was testing my project with the 1.4 RC, and it turns out those
permissions are no longer created.
This is caused by the refactor of the create_permissions code, which now
uses get_for_models to determine the class to get the options for, but this
returns the proxied class (auth.User), not the proxy class, so my custom
permissions are not found and not created.
Maybe I am using the proxy model for something it was not intended for. If
so, it might be a good idea to add a warning in the release notes about
this, in case others are doing something similar.
BTW, thanks for yet another awesome new release.
Koen
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/django-developers/-/gdAd5dGloeoJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Custom permissions on proxy model no longer created in 1.4
Op donderdag 15 maart 2012 11:52:42 UTC+1 schreef Aymeric Augustin het volgende: > > Le 15 mars 2012 10:27, koenb a écrit : > >> this could very well be one of those "don't do that" things, but here is >> my problem. >> >> I have been using some specific permissions concerning the auth user model > > (... snip ...) > > Now I was testing my project with the 1.4 RC, and it turns out those >> permissions are no longer created. >> > > Hi Koen, > > At first sight, you code is valid, and this is a regression. > > I suppose it happened somewhere in this list: > > https://code.djangoproject.com/log/django/trunk/django/contrib/auth/management/__init__.py > > Could you open a ticket on code.djangoproject.com and set Severity to > "Release blocker"? > > Thanks, > > -- > Aymeric. > Done: #17904 Koen -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/VhZC9AxfB9sJ. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: prefetch_related - new feature suggestion
On 27 sep, 05:18, Luke Plant wrote: > On 27/09/11 03:23, Alex Gaynor wrote: > > Would you like to share your solution? I found it pretty difficult to > come up with anything that: > > 1) could be done on a per-query basis and > 2) didn't require changes to the code that would use the QuerySet > objects i.e. fully API compatible. > > The one avenue I didn't explore yet was proxying the entire model > instance, which I'm sure would work, but could have lots of annoying > corner cases with Python magic methods etc. > > Luke > I once created django-selectreverse [1] to do something alike, but that did require changing your code (no API compat.) and was also somewhat limited. It would be very nice to have something in core for this. Koen 1: http://code.google.com/p/django-selectreverse/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposal: user-friendly API for multi-database support
Just to add a little note: back in May I did some work on multidb, some thoughts and some work can be found on http://trac.woe-beti.de/ , which Ben Ford set up for this. I stopped because django was becoming too much of a moving target to keep it in sync (and i did not have the time). I would like to point out that my starting point was mainly to be able to use data from different database ENGINES (like some tables from postgres and some from mysql). As far as I can tell, this is not supported currently by the plumbing Malcolm provided (since the operations settings eg in WhereNode are taken from the default connection and not from the passed in connection; this is no problem if your databases are using the same engine, but it returns wrong SQL if you have different engines). I allready reported this once in #7258 (which was considered invalid, probably because I forgot to mention my use case). Anyway, I am very glad to see some renewed interest in this field! Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
old sqlite versions
In ticket #10113 I signalled two problems. The first was almost immediately fixed by Russel (thanks!). For the second problem I added a specific testcase that was giving me wrong results. It concerns an annotated query with ordering on a foreign key that was not yet included in the query. Russel reported no errors for this query for him though. It turns out this query was only giving wrong results on the sqlite version I was using (3.3.4 (pysqlite 2.3.2). When I tried this with a more recent version (pysqlite 2.5.1,i.e. sqlite 3.6.2), all went well. This solves my problem, but leaves a question: the version I was using was the one built-in in my windows python 2.5 install, which means it is probably not an uncommonly used version. Does this mean there is reason to try and work around the old sqlite bug in django, by preventing the kind of query that is causing the trouble ? I did not investigate to deeply and my SQLfoo is very poor, but a simple experiment suggests me that modifying the order of the joins might help. The troublesome query had some SQL like this "select ... sum(B.x) from A left outer join B on (...) inner join C on (...) group by ... order by C.y" which confuses sqlite 3.3.4 and produces wrong results. Changing it to "select ... sum(B.x) from A inner join C on (...) left outer join B on (...) group by... order by C.y" does seem to produce correct results however. Does this ring a bell to anyone, and more importantly, do we need to do something about this ? Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: More multi-database plumbing (WAS Re: Changing DatabaseWrapper._cursor() to take a dict instead of settings)
> My "nuts and bolts" document ... Hi Alex, about a year ago, I spent a few weeks working on multiple database support myself (building further on the code in ticket #4747). I used a mercurial trac setup by Ben Ford to store the work and some notes. It can be found at http://trac.woe-beti.de. (It was down for a while, but Ben was so kind to put it up again). Obviously, most of what was done then is very much out of date now (last commit is from 5/29/08), but it may help you get some extra ideas. You may want to check out the notes in the wiki to see what was done. There is also a discussion of use cases and some API ideas. Some of the items that were covered were: - declarative style connection definitions - model registration - per connection transaction management - changes to the sql generation (WhereNode) in order to produce correct sql on different database backends - modification to field definitions: there were/are a lot of backend specific things in there (quoting and stuff) - definition of a ConnectionCommand management command class so you can specify a connection for certain management commands - ... Important to note is I wanted to get this to work with different database backends (getting data out of legacy databases), hence the focus on the backend specific differences. I was discouraged by the lack of interest back then so I gave up (getting my data for my project was easy enough without it), I hope you have some more success. Good luck, Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: More multi-database plumbing (WAS Re: Changing DatabaseWrapper._cursor() to take a dict instead of settings)
> Is there any downside to checking this in immediately? It's a simple > enough change that I don't see it introducing any bugs (famous last > words, right?)... Please don't allow using connections from different database backends unless the problems of using the correct database options are resolved first. Both model fields and query creation still have a lot of uses of the 'default' connection to get to these options. This implies that there is a possibility to get hard to debug weird sql trouble when mixing database engines. Enforcing the same backend for the different connections might be a good idea for now. Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal for discussion about Serialization requirements and requesting for Review
On 26 mrt, 17:48, "Madhusudan C.S" wrote: > Hi all, > [snipped] I can't seem to find anything on the problem concerning contenttypes in your proposal (see [1] for some recent discussion). It would be nice to see that solved too, since it is one of the inconveniences that has bitten me several times. Koen [1]: http://groups.google.be/group/django-developers/browse_thread/thread/943a36552527a018/f540ef9050aa20ce --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Patch vs checkin
Is there any news on updating the svn repo for this branch ? k --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
db2 backend
I am trying to create a django backend for DB2. I am trying to do this using a DB2 Express C and the pydb2 module. It is coming around quite nicely. I have the test suite down to 7 failures and 3 errors, but now I am a bit stuck. My biggest hurdle at this point is that DB2 does not have an easy way to deferr foreign key checking in transactions. I was experimenting with SET INTEGRITY OFF, but have been unable to get it working. This is causing the errors in the serializers and fixtures tests. How important is it to have the possibility to load prereferencing data ? (don't have any use for this myself actually) I must admit that I am new myself to DB2, so I may be overlooking something. Are there more people interested in a DB2 backend, and is there anybody experienced in DB2 around to help me out to get it fixed ? Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Patch vs checkin
On 11 jul, 16:12, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote: > On 7/11/07, Ben Ford <[EMAIL PROTECTED]> wrote: > > > None as yet, I'm really keen to get it back into the svn repo, but I think > > the people concerned are busy with other things at the moment. > > I've said this a couple times, but I guess it's been missed: > > We're moving away from giving branch access since it's not really > worked so well in the past. It simply doesn't scale very well -- > there's only one of me, and I'm quite bad about remembering to get > people access -- and people assume that SVN branches are on track to > be merged, which may not be the case. Maybe I am missing something here. I understand why you would be reluctant to open up new branches, but I don't understand why this would justify keeping a branch in a 7 month old status if someone has done the (big) effort of bringing it more or less up to date. I don't care who commits the changes to the branch, but my idea is that since it is there and the work has been done, it might just as well go in. To me it seems the best way to get some new interest for the branch and to get it moving again. I think this is especially interesting for an important one as multi- db. This feature really is a life-saver for those of us who have to use existing data, but still want to have all the niceties that working with django brings us. > > The best thing to do is host your changes remotely. That way there's > no implicit assumptions about if and when it'll get merged into > Django. Isn't that what the wiki is for: to give a clear idea about the status of a branch? > > Probably the best way to host an "unofficial" branch would be by using > Tom's Mercurial mirror of Django trunk:http://hg.korpios.com/django.trunk/. > > Jacob Groetjes uit België, Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: db2 backend
> I was, however, turned off by the state of pydb2 -- it looked like the > project hasn't been touched in years. You are right. I used the patch made by Javier Villavicencio (see [1]), which makes things somewhat better. I contacted him, and it seems he has a lot more improvements ready, but needs permission from his boss to release them. Koen [1]: http://sourceforge.net/tracker/index.php?func=detail&aid=1731609 &group_id=67548&atid=518208 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: runtests.py defaults
You might also mention the existence of "junction.exe" on windows (see [1]), which gives you more or less an equivalent of symlinks. Also very practical, and maybe not known by too many people. > Could you drop this into a wiki page, please. Give it a descriptive > title and it will show up easily searches. > Koen [1]: http://www.microsoft.com/technet/sysinternals/FileAndDisk/Junction.mspx --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: db2 backend
I created a ticket with my efforts (#5052). There are still some issues to be solved though. Feedback is welcome. Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
i18n on Windows
[5722] fixed a problem from #4899 concerning make-messages.py. However, the fix caused problems for Windows users (empty po files are generated). Ramiro Morales provided a patch to the ticket that solves the problem (this has been acknowledged by several users in the ticket). Can someone take a look to put this in ? This problem may cause a lot of difficulty for windows users trying to implement i18n. Thanks, Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Failing tests for fixtures with MySQL (#5399)
I am having a similar problem with a custom db2 backend I am working on too. Maybe it can be linked in the implementation to some backend flag (in the backends DatabaseFeatures) stating that deferred constraint checking is not (entirely) implemented ? This as opposed to hardcoding "mysql" in the tests. I could use a similar thing for the regex lookups which I cannot implement for db2 at the moment. Koen On 13 sep, 13:09, "Russell Keith-Magee" <[EMAIL PROTECTED]> wrote: > On 9/13/07, Michael Radziej <[EMAIL PROTECTED]> wrote: > > > > > So, I'd appeal for skipping tests that cannot succeed, like in this > > case. > > This is ticket #4788, and it has been accepted as an idea. We just > need an implementation. If you want this for a sprint activity, it's > all yours. > > Regarding the implementation - I'm sure the easiest solution will be > to put an 'if not mysql' check around the failing tests. However, my > preferred solution would fix this at the output layer, rather than the > test layer - i.e., let the tests run and fail, but filter the output > against a list of known failures so that the failures are reported in > the final output as "X tests passed (with Y known and acceptable > failures)" rather than the current flood of stack traces. > > This approach would mean that we don't forget that the tests are > actually failing, but we don't let the failures get in the way of an > otherwise operational test suite. > > Of course, I haven't actually looked into this to see how > practical/possible it is, so I might be dreaming the impossible dream > :-) > > Yours, > Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Failing tests for fixtures with MySQL (#5399)
Hey Mike, > To use Koen's example, he has not implement regex lookups yet, and he wishes > to tell the test not to fail as it simply not implemented. In my opinion the > test SHOULD fail because well the test is to see whether or not this feature > is implement, and its not. Now i'm not familiar with the test framework at > all (I plan on making use of it in the next week or so however) however I do > feel that a test that checks for something, as Russel has said, should > always fail if it does indeed not work. I did mean this to work as Russell suggested: those failures to be marked as known and (for the moment) intended, not to stop the test from failing. Maybe NotImplementedErrors can be caught seperately ? Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
#689: using REMOTE_USER
I added a new patch to ticket 689. This one does not use settings (thanks Joseph Kocherhans). It combines the use of a middleware and an authentication backend to use the Remote_user information passed down by apache. This is usefull in intranets where you can use apache with mod_ntlm or mod_auth_sspi to authenticate the user in the intranets domain. It is a very simple version: any unknown user is considered to be valid (apache authenticated) and is simply added to the user database without extra info. It is easy to do extra stuff by subclassing the backend. I can think of eg getting the users email-adres and maybe setting custom permissions via LDAP. I am not sure whether the backend should do more by default than it does now. My guess is it would be better to extend the documentation with some more complex examples. Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: On aggregates
If I understand it correctly, the aggregate and annotate ideas are result driven, not by the SQL that might be involved (which should indeed be entirely hidden). Aggregate normally gives one result, annotate gives you a bunch of objects, but with extra information PER object (normally from an aggregate on a foreign table). On 12 mei, 16:50, David Cramer <[EMAIL PROTECTED]> wrote: > What is the difference between annotate and aggregate? They seem like > they'd do the same thing, except annotate sounds like it should be > doing GROUP BY, which, if that's the case, then this goes against the > very reasoning which a group_by or something similar should not be > used. The logic in the implied filters should be enough to determine > if you need to use a GROUP BY or a WHERE, or honestly, even a DISTINCT > (as I discussed in IRC the other day, DISTINCT is just another > approach to grouping, and should be dropped if we won't support GROUP > BY directly). > > On May 10, 6:48 am, "Russell Keith-Magee" <[EMAIL PROTECTED]> > wrote: > > > On Sat, May 10, 2008 at 2:37 PM, Nicolas Lara <[EMAIL PROTECTED]> wrote: > > > > Hello, > > > The choice of using Decimal was not mine but that of the modules that > > > connect to the backends (postgres in this case), and this is one of > > > the problems I am facing right now. Different backends return the > > > results in different formats (Postgres: Decimal, SQLite: float... etc) > > > I am looking into a way of normalizing the results to the same format. > > > This is something we're going to want to normalize, and as Alex noted, > > floats would make a lot more sense. > > > > The issue of the datastructure that represents the results is only one > > > part of the problem. Due to the inner representation of numbers and > > > other factors of the different backends, the numeric results might be > > > different, for example, postgres might return 34.7321987 while sqlite > > > returns something like 34.732211 (rounding differentely the results). > > > Keep in mind that there are already a few places in the tests where we > > have to deal with inexact floating point references. The easy solution > > is modify the doctest to only print n decimal places, where n is > > enough places to check the test case, but won't contain any floating > > point ambiguity (in your case, check for 34.73 or 34.732) > > > Yours, > > Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Multiple database support
For those interested in multiple database support, I have started working on it again, and posted my work-in-progress to ticket #4747. I started from trunk and added things from the multidb branch little by little, since so much has changed in that area since then. There is still a lot more that needs to be checked and a number of things to be redone. References to the default connection object are all over the codebase, there is still a lot of work left to get all of them straightened out. What is working (more or less): - using existing databases (though there might still be quirks when using different engines) - running tests - some of the management commands (eg sqlall, sqlflush, but loaddata, inspectdb or syncdb are not quite there yet) Important to mention is that relations across databases are not supported. Anyway, if anyone is interested in helping, please let me know! Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
Ah, missed that one. Anyway, I only did the easy parts (that is, getting data in and out of existing databases). Thanks for the pointer, I'll try to keep an eye on that. Koen On 20 mei, 16:56, "Nicola Larosa (tekNico)" <[EMAIL PROTECTED]> wrote: > koenb wrote: > > For those interested in multiple database support, I have started > > working on it again, and posted my work-in-progress to ticket #4747. > > ... > > Anyway, if anyone is interested in helping, please let me know! > > I am going to need this in a month or so. Actions speak louder than > words, so many thanks for your efforts. However, there were news two > months ago, summarized in this thread: > > Yet another SoC introduction: Getting multi-db > donehttp://groups.google.com/group/django-developers/browse_thread/thread... > > It would be nice to coordinate each one's efforts, to avoid wasting > time. Ben, Daryl, any news? > > -- > Nicola Larosa -http://www.teknico.net/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
I really like this line of thought: having the persistence layer of a model fixed is a good idea. Relations is a big issue here: unless we can support relations across databases, switching connections is always a big opportunity to shoot yourself in the foot. I would propose a function that can collect "clusters" of models, that is a collection of models that somehow are related to each other and use that function to a) check that they all use the same database during validation, and b) if we provide a API to register a model for an additional connection (that is a second one), you get copies of the models for the entire cluster, relations and all. Like that we could even have syncdb create the tables for these 'backup models' too. Koen On 21 mei, 23:16, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote: > Hi guys -- > > Sorry for coming late to the party, but I'm just now catching up on > django-dev. > > I'm really glad to see you get the ball rolling on multiple db > support, and once I'm dug out from my backlog I'll be happy to start > reviewing code and helping out if I'm needed. > > However, before we get to that point, I've got some pretty serious API > concerns with the current approach, so I think I should outline those > before y'all go much further. I don't want you to expend much effort > just to get a -1 smackdown. > > The current mechanism of defining "other" databases in the settings > module is just fine, and the underlying mechanism of having > queries/managers "know" their connection is similarly dandy. But the > wheels come off when it comes to the "public" API where users will > choose which connection they use. > > As far as I can tell, you've currently provided two hooks to use a > secondary connection: set the model's default connection in the > settings module (which is OK, I suppose, though I might want to > nitpick the syntax a bit), and assigning to ``Model.objects.db``. > > This second one is a disaster waiting to happen -- you've had to muddy > things up with threadlocals to work around some problems already. Also > consider the "bookkeeping" you'd need to do to deal with objects > across multiple database simultaneously (think sharding). You'd have > to keep juggling ``Model.objects.db`` and saving old ones... ugh. > > Here's how I think it should work: > > * I'd like the default connection for each and every object to be the > default database forever and always. I find putting models for default > connections in settings distasteful and I'd rather just a single API > for changing the connection (see below). However, I imagine I'll be in > the minority here so I'm prepared to cede this point if necessary. > > * There needs to be an official API to get a model (or perhaps a > manager) which references a different "context" -- > ``Model.objects.db`` should be read-only. So you'd call some API > method, and get back a sort of proxy object that uses the other > connection. Here's a strawman API:: > > >>> from django import db > >>> from someapp.models import Article > > >>> Article.objects.all() > [... all Articles from the default database ...] > > >>> ArticlesOnOtherDatabase = > db.get_model_for_other_connection(Article, "private") > >>> ArticlesOnOtherDatabase.objects.all() > [... all Articles from the database defined with the "private" key ...] > > This should make the threadlocal stuff unnecessary, and (to my eye) is > a lot more sane than assigning the ``Manager.db``. Oh, and please > choose a better better name than > ``db.get_model_for_other_connection()``; given that you're building > the bikeshed you might as well paint it, too. > > Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
You need to be aware that this is quite complicated if your model has foreign keys in it: if you use the ORM to do queries, the ORM would have to be so smart as to when to split up your queries instead of doing joins. eg you have model A which foreign keys to a User model. For a row of A that is in the same database as User, the ORM could simply use a join, but for a row of A that was already in the other database, it can't. I do not believe this is a trivial change. My proposal is to keep things simple in a first phase: let's make it possible to use different databases for different models with the restriction that relations should not cross databases. Once we get all that working, we may look at making the query generation deal with those. Koen On 23 mei, 13:00, "Mike Scott" <[EMAIL PROTECTED]> wrote: > On Fri, May 23, 2008 at 2:59 AM, Simon Willison <[EMAIL PROTECTED]> > wrote: > > > 1. Replication - being able to send all of my writes to one master > > machine but spread all of my reads over several slave machines. > > Thankfully Ivan Sagalaev's confusingly named mysql_cluster covers this > > problem neatly without modification to Django core - it's just an > > alternative DB backend which demonstrates that doing this isn't > > particularly hard:http://softwaremaniacs.org/soft/mysql_cluster/en/ > > Personally I think this is something that is better solved by the database > software itself. Having replication code-side is something that I don't feel > to good about. But maybe thats just me. > > > > > 2. Sharding - being able to put User entries 1-1000 on DB1, whereas > > User entries 1001-2000 live on DB2 and so on. > > This is something I would love, an example being archives. (As Eratothene > points out. > > Maybe having to state a storage location on a per-row level. (IE this could > happen by overriding the manager, and simply switching DB at selection time. > or being able to provide the DB info at selection time.) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multiple database support
On 22 mei, 18:28, "Ben Ford" <[EMAIL PROTECTED]> wrote: > Hi all, > > I've now re-applied Daryls patch (which was against qsrf) to a clone of > django trunk in a mercurial repo. It's available > athttp://hg.woe-beti.deandthere's a trac set up for it > athttp://trac.woe-beti.de. Feel free to make use of both of these. Although > I've disabled to ability to create tickets perhaps the wiki might be a good > place to discuss the API? Anyone can clone from the hg repo, give me a shout > if you would like push access and I'll sort it out. > > Cheers, > Ben > > -- > Regards, > Ben Ford > [EMAIL PROTECTED] > +447792598685 I have been adding some code to Ben's mercurial repo on [http://hg.woe- beti.de], see also [http://trac.woe-beti.de]. What has been realised (more or less): - connection and model registration - validation that related objects use the same connection - database_engine specific SQL generation (needs more checking) - some management commands accept connection parameter, others can generate output for multiple connections - syncdb can sync different connections - transaction management over connections - some tests (needs a lot more work) This means point 3 of the discussion at [http://trac.woe-beti.de/wiki/ Discuss] is somewhat working, but definitely needs a lot more testing. I do need some help with creating tests for all this though. I have not figured out yet how to create tests that check that the right SQL is being generated for the backend used. (Nor how to test the right database was touched by an action, this is quite obvious manually, but I do not know how to automate this.) I put some ideas on using multiple databases for copying (backup or migration) of objects (point 4 of the discussion note) in [http:// trac.woe-beti.de/wiki/APIModelDuplication]. Please comment, add, shoot etc. Any help will be much appreciated. Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
