Hi Russ,
Thanks for the long reply and all the suggestions. My comments are
inline.
> What if you need to support both? e.g.,
>
> <field foo="the foo value">
> <bar>the bar value</bra>
> </field>
>
> It seems to me that you would be better served providing a way to
> annotate each individual metadata value as (and I'm bikeshedding a
> name here) 'major' or 'minor'. JSON would render all metadata as
> key-values, and XML can make the distinction and render minor metadata
> as attributes, and major metadata as tags.
>
I think that's a great idea, this can be implemented with decorators
on the methods like @tag or @attribute while setting one of them as
default when no decorator is applied.
> I think I see where you're going here. However, I'm not sure it
> captures the entire problem.
>
> Part of the problem with the existing serializers is that they don't
> account for the fact that there's actually two subproblems to
> serialization:
>
> 1) How do I output the value of a specific field
> 2) What is the gross structure of an object, which is a collection of
> fields plus, plus metadata about an object, plus
>
> So, for a single object the JSON serializer currently outputs:
>
> {
> "pk": 1,
> "model": "myapp.mymodel",
> "fields": {
> "foo": "foo value",
> "bar": "bar value"
> }
>
> }
>
> Implicit in this format is a bunch of assumptions:
>
> * That the primary key should be rendered in a different way to the
> rest of the fields
> * That I actually want to include model metadata like the model name
> * That the list of fields is an embedded structure rather than a list
> of top-level attributes.
> * That I want to include all the fields on the model
> * That I don't have any non-model or computed metadata that I want to include
I believe that my model of using a recursive method and storing
temporary data in 'levels' would address most of these concerns. The
method for handling a model would consist of the following steps,
roughly:
* Get the list of fields to be serialized.
* Now serialize each field , after checking for circular references
(see below), (using handle_field and apply all metadata, formatting
options etc) to a temporary python object, most probably as a key-
value in a dict.
* If an FK or M2M is encountered, check for nesting restrictions
and then recursively apply the handle_model method.
* Add model level metadata, formatting options.
* Process reverse relations, if required. (see below)
* Store the model in some container in the serializer object.
* Clear temp data in the current 'level'.
Finally, when all models are processed, dump the data from the
container into the required format.
> When you start dealing with foreign keys and m2m, you have an
> additional set of assumptions --
>
> * How far should I traverse relations?
The user can specify a limit to the levels of nesting through
variable ``max_nesting_depth``.
> * Do I traverse reverse relations?
In my opinion, traversing reverse relations can get really ugly at
times, especially when there are M2M fields, foreign keys or circular
relations involved. But there are some scenarios where the data is in
a relatively simpler format and serializing them would be useful. To
support this, I thought of something like this:
class Srz(Serializer):
...
reverse_relations = [ (from_model_type, to_model_type), ... ]
But this should be used with caution and avoided when possible.
> * How do I represent traversed objects? As FK values? As embedded objects?
As embedded objects, if the nesting depth limit is reached, then as FK
values.
> * If they're embedded objects, how do I represent *their* traversed values?
Their traversed values would be represented just as a normal model
would be, with field-value mappings. The user can choose which fields
to dump.
> * What happens with circular relations?
For all model type objects, like the base model in the query set and
all FK and M2M fields, some uniquely identifying data (like the
primary key, content type) will be stored in a list as each one of
them is processed. Before serializing a model, it would be checked if
the model is already on the list or not. If it is there, it is a
circular reference and that model would be ignored .
> * If I have two foreign keys on the same model, are they both
> serialized the same way?
Yes.
> When you start dealing with the XML serializer, you have all these
> problems and more (because you have the attribute/tag distinction for
> each of these decisions, too -- for example, I may want some fields to
> be rendered as attributes, and some as tags.
>
For XML, I thought of using an intermediary container for a node that
would store all these details.
> > ---------------------------------------------------------
> > New features in the serialize() function
> > ---------------------------------------------------------
> > Apart from the changes I’ve proposed for the ``fields`` argument of
> > serialize, I would like to add a couple of features:
>
> > • An exclude argument, which would be a list of fields to exclude from
> > the model, this would also contain the fields to exclude in related
> > models.
>
> > • An extras argument, which would allow properties and data returned
> > by some methods to be serialized.
>
> For me, the goal should be to deprecate these sorts of arguments. The
> decision to include (or exclude) a particular field is a feature of
> serialization that is intimately tied to the serialization format, not
> something that is an external argument.
>
Initially, I thought the goal was not to tie down a serializer to any
model, I can integrate these features into the serializer class then.
> > -----------------------------------
> > Permission Framework
> > -----------------------------------
>
> I'm not sure I see the value in this bit -- at least, not as a
> baked-in feature of the serialization framework. A serialization
> format encompasses "what should I output"; if you've defined a
> sufficiently flexible framework, it should be possible to introduce
> permission checks without needing to embed them into the base
> serialization framework -- they should just be a set of specific
> decisions made by a specific serializer.
>
> In fact -- this may be a good test of your proposed API: Could a third
> party write a serializer that prohibited serialization of certain
> attributes, or modified the serialization of certain attributes, based
> on a check of Django's permissions? Personally, I don't see this as a
> core requirement, but demonstrating that it is possible in principle
> would be a compelling argument for your API.
>
I need to take a deeper look at things before I can comment on this,
though, on the surface it looks possible, by binding the output to
views.
> > -----------------------------------------------------------------
> > Representing the existing serialization model
> > -----------------------------------------------------------------
> > Here is an implementation of the existing serialization format in
> > JSON, this would be the ‘fixture’ mode that I’ve mentioned above.
>
> I think these examples demonstrate what I said earlier -- your
> proposed framework allows me to customize the name given to a field in
> XML, but doesn't allow me to change the parent of that field within
> the broader XML structure.
>
I'm not sure that I follow this, It would be great if you could give
an example. As I mentioned earlier, there will be an option to
'flatten' the nested models , provide alternate names to the fields,
and wrap fields into a group. Initially I had thought of adding this
to the external ``fields`` argument in ``serialize()``, but I can add
them to the specification object too.
> One suggestion -- what isn't clear from this timeline is when we will
> start to see concrete evidence of your progress. From a broad project
> management perspective, it would be good to see some concrete
> deliverables in your timeline -- e.g., at the end of week 2, it will
> be possible to serialize a simple object with integer and string
> attributes into a configured JSON structure; by week 4, it will be
> possible to use the same structure with XML; and so on.
>
I will restructure the timeline keeping this and all the other changes
you've suggested in mind and update it soon.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
