On 2 kesä, 23:20, Luke Sneeringer <l...@sneeringer.com> wrote:
> On Jun 2, 2013, at 8:22 AM, Michal Petrucha <michal.petru...@ksp.sk> wrote:
>
> > GenericForeignKey and nontrivial field types
> > --------------------------------------------
>
> > As I've indicated in my proposal, just casting any value to a string
> > and then performing a reversible transformation on such strings may
> > work well enough for string and integer database columns, not so much
> > for things like dates, timestamps IP addresses or other similar types.
>
> > Any ideas on how to make this work? Should I try to extend the backend
> > API to include explicit casts for each nontrivial column type to a
> > string representation equal to the one used by Python? Or should I
> > just document this as unsupported?
>
> There's already a `db_type` method that you can override (that receives a
> `connection` object) for the actual database type. It's pretty easy to do
> something to the effect of (for instance):
>
> if 'postgres' in connection['ENGINE']:
> return 'uuid'
> if 'mysql' in connection['ENGINE']:
> return 'char(36)'
> [...]
>
> However, having done some work myself on trying to create non-trivial field
> subclasses, it's everything after that which gets difficult. Django provides
> an overridable method to cast the value, but nothing for anything else in the
> expression (the field or the operator), which are hard-coded into the
> backend. (This is a source of frustration for me personally, because it makes
> it very difficult to write classes for, say, PostgreSQL arrays, without
> either resorting to massively ugly syntax or subclassing nearly every single
> class involved in the process of creating a query (Manager, QuerySet, Query,
> WhereNode...)
>
> I ultimately went through the subclass-half-the-world technique quite
> recently (a couple of weeks ago), as I want some non-trivial custom fields
> for a new project I am about to start for my company (sadly, the project is
> private, although I'd be happy to share field code if it would help in any
> way). What I ended up doing is checking the Field subclass for a custom
> `get_db_lookup_expression` method (that's not a Django field method -- I made
> it up), and then my Field subclasses could use that to return a full
> expression in the form "{field} = {value}". If the method is present and I
> get something (other than None) back from that method, then use it, otherwise
> I pass it on to the DatabaseOperators class for its usual processing. Using
> that method prevents me from having to modify a monolithic DatabaseOperators
> subclass for each new field I add (it seems to me that fields should know how
> to perform their lookups).
>
> The other challenge was defining the QuerySet lookup expressions. Django
> essentially hard-codes the things it understands for lookups (e.g.
> Foo.objects.filter(bar__gt=5) being transformed into "select ... from app_foo
> where bar > 5"). The set of lookup suffices (exact, gt, gte, lt, lte, etc.)
> is, sadly, also essentially hard-coded. I wrote an ArrayField to use
> PostgreSQL arrays, and really wanted a way to be able to lookup based on the
> length of the array (so, something like `Foo.objects.exclude(photos__len=0)`,
> for instance, to give me all Foos with no photos). I did manage to make that
> work, but it was a struggle. Also, the set of lookup suffices is universal,
> even though some of them don't make sense on some fields ("year" on
> IntegerField, for instance).
>
> So, my ideas on getting non-trivial field subclasses to work is basically:
>
> 1. Make fields be the arbiter of what lookup types they understand.
> (IntegerFields shouldn't understand "year"; maybe someone's ArrayField
> subclass does understand "len".) This probably needs to be something that can
> be determined on the fly, as composite fields will probably need lookup types
> corresponding to their individual sub-fields.
>
> 2. Make lookup types chainable. In the Array "len" example,
> `photos__len__gt=5` makes sense.
>
> 3. Make it so fields define how they are looked up based on database engine
> and lookup type.
>
> Moving these things into the Field implementation (rather than in the
> backend) should mean that non-trivial field subclasses become much easier.
> It'll also eliminate the need for, say, django.contrib.gis to have an
> entirely different set of backends -- a large reason gis jumps through those
> hoops (as best as I can tell from reading it; disclaimer: I am not a
> contributor) is to work around the restrictions being described.
>
> I hope that helps your thinking. I have this stuff fresh in my head because
> I've just worked on an implementation for PostgreSQL arrays and composite
> fields that I need for my work. While I've thought a decent bit about
> extensibility (for my own purposes), I haven't open-sourced it largely
> because I know I haven't solved all the problems yet. Having read your
> e-mail, I now hope that I don't have to, as I expect your work to outshine
> mine. I look forward to replacing what I've done with what you do. :-)
>
> One more absolutely massive disclaimer: I am not a core developer or even an
> active developer of any kind on Django. I'm sure other voices will offer
> guidance, and listen to theirs over mine. I'm sending this because I've spent
> a lot of time very recently in the code you are looking to enhance, and
> because extensible APIs generally are a passion of mine. That said, take my
> comments for what they are -- one of a large number of voices, where I am not
> a source of substantial expertise. I hope I can help focus and clarify your
> thinking a little bit. I am sure others will reply and offer much more useful
> and much more complete guidance.
It seems you are restricted by the lookup system, and that is what
#16187 (refactor lookup system AKA custom lookups) tries to solve. The
custom lookups situation isn't directly related to composite fields.
If custom lookup system gets implemented, then composite fields should
be just an user of that system.
FWIW custom lookups is number one item on my TODO list. The
combination of PostgreSQL 9.3 + JSONField with custom lookups should
be a perfect way to store unstructured data while still using a
conventional database.
There is a stale patch in the ticket. I think I will need to rewrite
it once more. However you might be interested in trying the patch and
check what it can do. Check out
https://github.com/akaariai/django-hstore/compare/custom_lookups
and https://github.com/akaariai/django/tree/nested_lookups. All
feedback on API is welcome. There isn't much point in reviewing the
patch on code level - it will be largely rewritten anyways.
- Anssi
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
