Re: Query Refactor Update
What would the ListField Look like? On Jul 16, 8:06 pm, Alex Gaynor wrote: > Hey all, > > The last while has been spent continuing the fight to make aggregates > work correctly, Russ provided some good insight that's gotten me > farther on it. All that code is in my query-refactor-aggregates > branch on github, as it's not fully working ATM. This week I also > pushed a few updates to SVN, including support for F() expressions in > updates, and another small fix for bulk updates. > > My goals for next week are to continue the aggregates work (maybe even > finish it!), as well as to clean up the errors for trying to use > impossible features, right now they're a bunch of asserts, but ideally > they should be clean, expressive error messages as. I'm also going to > work on getting a ListField implemented, in principle the > storage/retreival of one shouldn't' be too bad, however MongoDB also > supports querying on them (as one might across a foreign key > relationship in a relational database), and I imagine some refactoring > will be necessary there. > > Since this is about the halfway point of GSOC I'll give a general > overview: we have a working MongoDB backend, with many implemented > features, and a set of changes to Django itself (that don't break > anything else of course) that enable this. Diligent readers will > recall that originally a backend was the goal for the 2nd half of > GSOC, with the first half being allocated to internal refactors. > Seeing as how the backend is nearing completion the 2nd half will be > targeted at these refactors, starting wiht the aggregation stuff I've > been working on. > > Thoughts, questions, flames, nobel prizes nominations welcome, > Alex > > -- > "I disapprove of what you say, but I will defend to the death your > right to say it." -- Voltaire > "The people's good is the highest law." -- Cicero > "Code can always be simpler than you think, but never as simple as you > want" -- Me -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
[GSoC] Customizable Serialization Proposal
First of all: my native language is not English, so I apologize for any potential natural language error (or any error at all) below. After (briefly) reading through the current implementaion as well as the "Issues to consider" section in the GSoC2011 page on wiki, my initial observation is that the current code structure/abstraction is almost good enough to cater all the "Issues" on the wiki if The need for "Serializing non-database attributes/properties" is not considered for now. It seems to me that an simple enhancement on the configuration options of the current serializer would suffice a lot of the needs here. The best approch I thought of so far is adding a configuration object to the parameter. It contains infomation about what modification needs to be done during the serialization. Does any field name needs to be mapped to a different one? How many levels to go as far as foreign key is concerned? What are the unnecessary fields (of course, the current "fields" option would fit well in here, too)? What are the relations between two fields in the result? This configuration object will be available to all the subclass-serializers for different format all the time, so that everytime a field is being serialized, the serializer will look for infomation the configuration object recarding this field. Further, when a fk is encountered, the configuration can contain reference to another configuration object, which, if desired (level of serialization not yet reached), will be used to serialize the object fk refers to. Naturally, the configuration object should be defined with classes (I almost wanted to call it Meta, as in the Models). I feel like I'm still missing a lot to consider. I'm aware of the old proposals in the past and I'm planning to digest those as soon as possible. But I really want to hear thoughts about my idea from y'all. So please, tell me what's wrong in it. PS: I have about 3 years Python experience, big Django fan, sort of a user (wrote my own blog, of course), my ID has appeared in Django commit msgs for about 3 times so far :). -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Customizable Serialization Proposal
lustration of how a simple use case looks like, taking the
models
from the tutorial as an example:
class SerializedChoiceText(DefaultSerializedCharField): key_name =
"choice_text" value_fmt = lambda x: x.lowercase() as_tag = True #
default value
belongs_to = "field"
class SerializedChoice(DefaultSerializedModel): exclude = ["votes"]
fields = {
'pk': DefaultSerialiazedPkField(), 'classname':
DefaultClassnameSeializedField(), 'choice': SerializedChoiceText(),
'poll':
DefaultSerializedFkField( embed_level=1,
value_fmt=DefaultSerializedModel()) }
data = serializer.serialize('xml', Choice.objects.all(),
SerializedChoice())
data would contain something similar to what the current XMLSerializer
yields
except:
1. No votes number.
2. The tag for poll now contains a sublevel tag that has a
serialized poll
object in it.
3. The text of the choise is in lowercase.
And that roughly covers the API design in my proposal.
In implementation, the structure of the serializers should be somewhat
similar
to the current ones, since the whole idea is to add more
configurations.
Yours,
DaNmarner
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Customizable Serialization Proposal
#x27;s an illustration of how a simple use case looks like, taking the
models from the tutorial as an example:
class SerializedChoiceText(DefaultSerializedCharField):
key_name = "choice_text"
value_fmt = lambda x: x.lowercase()
as_tag = True # default value
belongs_to = "field"
class SerializedChoice(DefaultSerializedModel):
exclude = ["votes"]
fields = {
'pk': DefaultSerialiazedPkField(),
'classname': DefaultClassnameSeializedField(),
'choice': SerializedChoiceText(),
'poll': DefaultSerializedFkField(
embed_level=1,
value_fmt=DefaultSerializedModel()
)
}
data = serializer.serialize('xml', Choice.objects.all(),
SerializedChoice())
data would contain something similar to what the current XMLSerializer
yields except:
1. No votes number.
2. The tag for poll now contains a sublevel tag that has a
serialized poll object in it.
3. The text of the choise is in lowercase.
And that roughly covers the API design in my proposal.
In implementation, the structure of the serializers should be somewhat
similar to the current ones, since the whole idea is to add more
configurations.
Yours,
DaNmarner
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Customizable Serialization Proposal
> 1) It's almost illegible. Between your mail client's munging of line > wrapping, and the completely unreadable code samples, it's very > difficult to tell if you have presented a good idea. Pardon the format. I actually auto-wrapped the text with vim and copy pasted at the first time. Realizing the result is ugly, I actually deleted it from the thread (using the google group web) and reposted without extra formatting. I was going to apologize for the duplication but didn't want to further spam everyone on this list. I believe what you read is the first copy, and the 2nd would be better. > 2) You haven't done the one thing that the Django GSoC wiki recommends > -- provide, as a proof of your concept, an expression of Django's own > serializers using your proposed serialization syntax. > > If you'd done any research on the topic, you'd find that I've raised > the second point every single time this topic is proposed -- in fact, > there's a currently active django-dev discussion with another student > proposing a serialization project. I regret that the unfortunate format stopped you from reading through my proposal, but my code example merely presented a case where the "extra" customization is provided: excluding fields, renaming key names, serializing nested objects. If none of above needs exist, then the user *don't* need to write anything. A default configuration would be used. And that, I assumed from the beginning, would be the current format the serializers use. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Customizable Serialization Proposal
As a side note: I encountered the use case for customizing serialization last summer when building my own blog project (I wanted an app that could import/export my models from/to wordpress/blogger XML. The API I proposed largely emerged from my thought back then. I'm willing to commit efforts to make Django able to do that in all seriousness, through GSoC or not. On Apr 5, 6:44 pm, DaNmarner wrote: > > 1) It's almost illegible. Between your mail client's munging of line > > wrapping, and the completely unreadable code samples, it's very > > difficult to tell if you have presented a good idea. > > Pardon the format. I actually auto-wrapped the text with vim and copy > pasted at the first time. Realizing the result is ugly, I actually > deleted it from the thread (using the google group web) and reposted > without extra formatting. I was going to apologize for the duplication > but didn't want to further spam everyone on this list. > > I believe what you read is the first copy, and the 2nd would be > better. > > > 2) You haven't done the one thing that the Django GSoC wiki recommends > > -- provide, as a proof of your concept, an expression of Django's own > > serializers using your proposed serialization syntax. > > > If you'd done any research on the topic, you'd find that I've raised > > the second point every single time this topic is proposed -- in fact, > > there's a currently active django-dev discussion with another student > > proposing a serialization project. > > I regret that the unfortunate format stopped you from reading through > my proposal, but my code example merely presented a case where the > "extra" customization is provided: excluding fields, renaming key > names, serializing nested objects. > > If none of above needs exist, then the user *don't* need to write > anything. A default configuration would be used. And that, I assumed > from the beginning, would be the current format the serializers use. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Customizable Serialization Proposal
> So - show me what it would look like. Show me how I, as a third party, > would use your proposed syntax to define output that would match > Django's existing serialization scheme. Yes, this serialization format > will exist as a built in default; but I should be able to reproduce > that format in my own code if I want. # Fair enough. if code > literacy: # here so_be_it() class DefaultSerializedField(object): belongs_to = "field" as_tag = true class DefaultSerializedSpecialField(DefaultSerializedField): " For pk and modelname " belongs_to = None as_tag = false class DefaultSerializedModel(object): exclude = [] fields = None def __init__(self,embed_level): self.embed_level = embed_level #Somewhere in the dark forest of BaseSerializer implemented if not config_is_passed_as_an_arg: config = DefaultSerializedModel(embed_level=0) for item in queryset: pk_config = getattr(config, 'pk', DefaultSerializedSpecialField()) self.process_pk(pk_config) mn_config = getattr(config, 'modelname', DefaultSerializedSpecialField()) self.process_model_name(mn_config) fields = getattr(config, 'fields',()) for name, field_config in fields.: if not name_of(field) in item.exclude: self.process_field(name, field_config) if not fields: default_config = DefaultSerializedField() for field in regular_field_of(item): if not name_of(field) in item.exclude: self.process_field(name_of(field), default_config) The idea is not to abandon all current assumption about how serialization proceeds, but to do so only to a degree such that we get the flexibility needed. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Customizable Serialization Proposal
> The purpose of this project is to define (and then implement) a DSL > for serialization. If a DSL is what you are looking for, then I withdraw my proposal. The idea of hosting yet another DSL inside Django project scares me a little. I'll just implement my idea as a separate project, I guess. Thank you for your time. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
