Interested in contributing to Django

[Vivek Narayanan]

Hi,

I am Vivek Narayanan, a sophomore undergraduate student from India. I
am interested in contributing to the Django project,  through GSoC
(Google Summer
of Code) or otherwise. I have been using Django for a while and have
developed a few applications on social
networking sites . I have experience in Python . I am cloning
the git repository right now and am looking for some pointers on where
to start first.

Thanks and Regards,

Vivek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Interested in contributing to Django

[Vivek Narayanan]

On Feb 3, 3:07 am, Gabriel Hurley  wrote:
> Hi Vivek,
>
> It's great to hear you're interested in contributing! This year's GSOC is
> just getting underway and while Django will likely participate as it has in
> years past, we're not quite at that stage yet.
>
> However, if you're serious about GSOC you'll have a much better chance of
> being selected if you've gotten your feet wet with contributing to Django
> already. So definitely take on some issues you're interested in.
>
> If you'd like some ideas on how to get started, we are actually working on a
> new "how to" guide for new contributors right now! It currently lives in the
> wiki so the community can add to it, but it'd be great if you want to give
> it a read and give some feedback on how it could be made more useful to you
> as a new contributor! Check it out here:
>
> http://code.djangoproject.com/wiki/ContributingHowTo
>
> If you have further questions you can always ask here on the mailing list or
> in the #django-dev IRC channel.
>
> All the best,
>
>     - Gabriel


Thanks for the help, I'm looking over the tracker and found a couple
of items interesting. I'll work on that for the time being.

Vivek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Ticket #8809 - "Better error message when can't import url callback".

[Vivek Narayanan]

Hi,

I am new to the Django community and I would like to implement a patch
for the ticket #8809 - "Better error message when can't import url
callback".

http://code.djangoproject.com/ticket/8809

This ticket basically requires the error message, when a callback
function in urlpatterns is not found, to display the exact urls.py
file and the line where the error occurred instead of just displaying
the module name.

I was looking into the _get_callback function of the RegexUrlPattern
class. I can write a function that scans the apps directory for the
exact file and line no but I need some more background information on
the following:

1. Where are RegexUrlPattern objects created and what is the
additional information available at this stage?
2. A basic higher level understanding of how the 'urlpatterns' objects
are processed from a urls.py file


Thanks,

Vivek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

#15040: Post processing of SQL results

[Vivek Narayanan]

Hi,

I was looking at the ticket at http://code.djangoproject.com/ticket/15040
in which 0 or 1 are returned instead of boolean values in a
BooleanField when MySQL is used with select_related(). Since MySQL
handles them as 0 or 1, they need to be converted.

>From my understanding, when select_related is not used , a function
called resolve_columns defined in SQLCompiler converts the values
using the python builtin bool(). And this is called only from
RawQuerySet.

But I couldn't comprehend anything else on how these methods are
invoked. How exactly is this post processing done and what are the
chain of events leading up to it.

Thanks,
Vivek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Customizable Serialization

[Vivek Narayanan]

Hi,

I am Vivek Narayanan, an undergrad student at IIT, Varanasi in India
and am interested in participating in this year's SoC

Problem

Django provides a serialization framework that is very useful for
loading and saving fixtures, but not very flexible if one wants to
provide an API for a Django application or use a serialization format
different from what is defined by Django. Also the current handling of
foreign keys and many to many relationships is not really useful
outside the context of fixtures.

Solution

I propose a class based Serializer, extending/refactoring the current
base class which would be entirely configurable by the user. I would
also like to provide some configurations for XML and JSON/YAML, that
can serve as building blocks for writing other serializers. The class
would have the following configurable options:

• A basic structure, markup or template representing each field an
object and also representing the object.

• An external wrapping structure, a structure for an array, delimiters
and assignment symbols.

• Arbitrary level of nesting depth, Which fields of related models are
to be represented? etc. This nesting can be handled by recursion. The
data about related models can be extracted in the start_object or
end_object methods.

• Choosing which datatypes to dump as is, which ones to convert to
something else. The user can provide a mapping between types in the
form of a dict and conversion functions when needed.

• Adding metadata about each field, like data-type or content-length;
this can be represented as additional attributes in a tag in XML or
arrays in a key-value representation. This can be done by adding some
class methods.

• The level of indentation.

The idea is to store them as a python list or list of dicts till the
final 'dumping' stage. This way we can still use existing libraries
like SimpleXML, SimpleJSON etc. This 'dumping' method would be
overrideable. The user would have to choose between using a standard
library and specifying a format. I would also like to add a couple of
features:

• An exclude argument, which would be a list of fields to exclude from
the model, this would also contain the fields to exclude in related
models. I would like to extend the fields argument in the same way.

• An extras argument, which would allow properties and data returned
by some methods to be serialized.

While this is by no means a complete proposal, I was looking for some
feedback on the idea and would be happy to incorporate your
suggestions.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Customizable Serialization

[Vivek Narayanan]

ol)
...
def unicode_tuple(self, object):
# Do something with the object


Representing the existing serialization format
--
Here is an implementation of the existing serialization format in
JSON:

class JsonSerializer(Serializer):
mode = "json"
wrap_fields = "fields"
nesting_depth = 0

def meta2_pk(self, model):
'''This method is not required to be overridden as a default
method would be provided'''
def meta2_model(self, model):
...


In XML

class XMLSerializer(Serializer):
mode = "xml"
wrap_fields = "fields"
nesting_depth = 0
metadata_display_mode = ATTRIBUTES
indent = 4

field_tag_name = "field"
model_tag_name = "object"

def meta2_pk(self, model):
...
def meta2_model(self, model):
...
def meta_type(self, field):
...

Sincerely,

Vivek Narayanan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Customizable Serialization

[Vivek Narayanan]

@Sebastien: I got your point about using a dict as an intermediate
structure and the use of recursion, and I looked at your
implementation which is somewhat similar to what I have in mind.

Well, here is a list of deliverables for the project:

• Investigate existing structure of the serializer, make changes,
refactor to suit needs.  (1 week)

• Implement metadata methods, change the ``fields`` argument of
serialize(), write unit tests. (2 weeks)

• Implement structures and templates parsing for custom serialization,
configurations for XML/JSON/YAML etc. Also, write tests for this. (2
weeks)

• Handling of nested and related models. (1 week)

• Investigate the changes to be made at deserialization side and
implement them. (1 week)

• More tests and write documentation. (2 weeks)

This is a conservative estimate and am keeping 3 weeks as a cushion.



On Mar 3, 7:15 pm, sebastien piquemal  wrote:
> Ok ... I have to admit I was not very clear. Here is what I meant,
> illustrated with some code examples :
>
> http://readthedocs.org/docs/django-serializers-draft/en/latest/index
>
> These are my "dream" django-serializers !
>
> On Mar 1, 10:20 am, Vivek Narayanan  wrote:
>
> > @Sebastien, Thank you for your suggestions, that's exactly what I had
> > considered.
>
> > As I've mentioned earlier, I would like to start with providing basic
> > XML, JSON, YAML and text serializers, that would be built on the
> > existing base structure with a few modifications, as building blocks.
> > But before I start with the class structure, let me describe a feature
> > that I would be adding for the purpose of metadata.
>
> > Metadata Methods
> > ---
> > The user can define methods beginning with “meta_” to add metadata
> > about each field. And functions starting with “meta2_” can be used to
> > add metadata at the model level. Here is an example:
>
> > class ExampleSerializer(serializers.Serializer):
> >     ...
> >     def meta_foo(self, field):
> >         '''
> >         Extract some metadata from field and return it.
> >         It would be displayed with the attribute ``foo``
> >         '''
>
> > In JSON the metadata would be represented inside an object as
> >     "key": {"foo": "bar", "value": value}
> > instead of
> >     "key": value
>
> > In XML, two options would be provided, to represent the metadata as
> > individual tags or with tag attributes, through a field option in the
> > class.
> >      metadata_display_mode = TAGS #or ATTRIBUTES
>
> >      TAGS
> >      -
> >       
> >           ..
> >           ...
> >           Value
> >       
>
> >      ATTRIBUTES
> >      -
> >       Value 
>
> > To select which fields would have which metadata, the arguments should
> > be passed in the ``serialize()`` method as
> >       data = ExampleSerializer.serialize(queryset, fields = ('field1',
> > ('field2',['foo']) )
>
> > Each field can be specified in two ways:
>
> > 1. As a string:-> no metadata will be added.
>
> > 2. As a 2-element tuple, with the first element a string representing
> > field name and the second a list of strings representing the metadata
> > attributes to be applied on that field.
>
> > Instead of manually specifying the attributes for each field, the user
> > can add all metadata functions for all the fields using the
> > ``use_all_metadata`` parameter in ``serialize()``
> >       use_all_metadata = True
>
> > The existing implementation of ``model.name`` and ``model.pk`` can be
> > described using “meta2_” functions. These will be provided as
> > ``meta2_name`` and ``meta2_pk`` to facilitate loading and dumping of
> > fixtures.
>
> > Basic Structure
> > 
> > Now coming to the basic structure of the fields. This need not be
> > specified for JSON/YAML as this will be handled by the libraries.
>
> > For text based serializers a custom template would be provided:
>
> > class TextSerializer(Serializer):
> >     mode = "text"
> >     field_format = "%(key)s :: { %(value)f, %(meta_d1)s, %(meta_d2)}"
> >     # Simple string template, meta_xxx would be replaced by
> > meta_xxx(field) if meta_xxx is callable
>
> >     #The three parameters below are required for text mode
> >     field_separator = ";"
> >     wrap_begin = "[[" # For external wrapping structure
> >     wrap_e

Re: Customizable Serialization

[Vivek Narayanan]

On Mar 6, 11:54 am, Russell Keith-Magee 
wrote:
> On Sun, Mar 6, 2011 at 2:41 PM, Vivek Narayanan  wrote:
> > @Sebastien: I got your point about using a dict as an intermediate
> > structure and the use of recursion, and I looked at your
> > implementation which is somewhat similar to what I have in mind.
>
> > Well, here is a list of deliverables for the project:
>
> > • Investigate existing structure of the serializer, make changes,
> > refactor to suit needs.  (1 week)
>
> > • Implement metadata methods, change the ``fields`` argument of
> > serialize(), write unit tests. (2 weeks)
>
> > • Implement structures and templates parsing for custom serialization,
> > configurations for XML/JSON/YAML etc. Also, write tests for this. (2
> > weeks)
>
> > • Handling of nested and related models. (1 week)
>
> > • Investigate the changes to be made at deserialization side and
> > implement them. (1 week)
>
> > • More tests and write documentation. (2 weeks)
>
> > This is a conservative estimate and am keeping 3 weeks as a cushion.
>
> Here's some advice: If this is what your final plan looks like, I
> would expect that your proposal would be rejected. Here's why:
>
>  * We prefer projects to have a clear design in mind before
> implementation begins. It's ok if refinements happen along the way,
> but "investigation" periods (and you have 2 of them) are not something
> that should be required. You investigate while you develop your
> proposal.
>
>  * Testing isn't an activity that can be clearly separated. It's an
> integral part of code development. Having a "more tests" activity
> indicates you either haven't allocated enough time for testing during
> development, or you're trying to pad your timeline.
>
>  * Padding with a 3 week cushion gives the impression that you haven't
> thought about the effort required. 3 weeks of full time development is
> a long time.
>
>  * I'm sceptical of any plan that consists of "2 week" estimates.
> Again -- a week is a long time. If you can't clearly express what will
> be developed, tested and delivered in a week long timeframe, then I
> don't think you've thought about the problem hard enough -- at least,
> not hard enough for us to recommend that Google give you $4k, and
> someone from the project spend many hours mentoring you.
>
> Yours,
> Russ Magee %-)

Thank you very much for your feedback, I must admit that I've not been
very clear and gave an impression different from what I really meant.
I already have much of the structure in mind and I would be just
looking at a few minor details, that is what I meant by investigation
and those periods I mentioned earlier won't be entirely dedicated to
investigation. I would be testing along with development, I had just
meant the tests that might have been overlooked and some final bug
fixing. Here is a revised schedule:

1. Write the basic skeletal structure of the serializer. (1 week)
2. Modify/Refactor the structure of the deserializer. (1 week)
3. Implement support for the metadata methods. (1 week)
4. Modify the ``fields`` argument in the call to ``serialize()``. (2
days)
5. Implement support for the unicode conversion methods. (3 days)
6. Add the ``exclude`` and ``extras`` arguments  in ``serialize()``.
(1 week)
7. Add support for string templates, output formats and structure. (1
week)
8. Handling of nested and related models. (1 week)
9. Write documentation and provide many examples. (2 weeks)
10. Some final housekeeping. (1 week)

Sincerely,
Vivek Narayanan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Customizable Serialization

[Vivek Narayanan]

On Mar 6, 11:54 am, Russell Keith-Magee 
wrote:
> On Sun, Mar 6, 2011 at 2:41 PM, Vivek Narayanan  wrote:
> > @Sebastien: I got your point about using a dict as an intermediate
> > structure and the use of recursion, and I looked at your
> > implementation which is somewhat similar to what I have in mind.
>
> > Well, here is a list of deliverables for the project:
>
> > • Investigate existing structure of the serializer, make changes,
> > refactor to suit needs.  (1 week)
>
> > • Implement metadata methods, change the ``fields`` argument of
> > serialize(), write unit tests. (2 weeks)
>
> > • Implement structures and templates parsing for custom serialization,
> > configurations for XML/JSON/YAML etc. Also, write tests for this. (2
> > weeks)
>
> > • Handling of nested and related models. (1 week)
>
> > • Investigate the changes to be made at deserialization side and
> > implement them. (1 week)
>
> > • More tests and write documentation. (2 weeks)
>
> > This is a conservative estimate and am keeping 3 weeks as a cushion.
>
> Here's some advice: If this is what your final plan looks like, I
> would expect that your proposal would be rejected. Here's why:
>
>  * We prefer projects to have a clear design in mind before
> implementation begins. It's ok if refinements happen along the way,
> but "investigation" periods (and you have 2 of them) are not something
> that should be required. You investigate while you develop your
> proposal.
>
>  * Testing isn't an activity that can be clearly separated. It's an
> integral part of code development. Having a "more tests" activity
> indicates you either haven't allocated enough time for testing during
> development, or you're trying to pad your timeline.
>
>  * Padding with a 3 week cushion gives the impression that you haven't
> thought about the effort required. 3 weeks of full time development is
> a long time.
>
>  * I'm sceptical of any plan that consists of "2 week" estimates.
> Again -- a week is a long time. If you can't clearly express what will
> be developed, tested and delivered in a week long timeframe, then I
> don't think you've thought about the problem hard enough -- at least,
> not hard enough for us to recommend that Google give you $4k, and
> someone from the project spend many hours mentoring you.
>
> Yours,
> Russ Magee %-)

I sent a message to this thread yesterday, I'm not sure why it didn't
appear here, so I'm sending it again.

First of all, thanks a lot for taking the time to go through the
proposal and I really appreciate your feedback. I must concede that
I've not been very clear and ended up sounding different than what I
really meant. I will be testing as I code, I just referred to some bug
fixing in the final stages. Here is a revised timeline:


In the run up to May 23rd, I'll be familiarizing myself with the
codebase and community practices of Django, examining all the
integration points and looking at the best practices of serialization.

Week 1: I'll be implementing a basic skeletal structure for the
serializer, which will set the stage for the rest of the project.
Week 2: Implementation of the deserializer.
Week 3: I'll add support for the metadata methods as discussed above.
Week 4: Support for datatype conversion and the unicode conversion
class methods.
Week 5: Add support for string templates and output formats.
Week 6: Add support for 'modes' as in JSON, XML, YAML etc, complete
the deserializer.
Week 7: Implement serialization of related models, M2M fields, foreign
keys and nesting depth.
Week 8: Add support for ``extras`` and ``exclude`` parameters in the
call to serialize(). Modify ``fields`` parameter as described above.
Weeks 9-10: Check for bugs, fix them and write documentation with many
examples.
Weeks 11-12: Refine the project and its documentation.

I'll be spending 40-45 hours a week on average.

Sincerely,
Vivek Narayanan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Customizable Serialization

[Vivek Narayanan]

On Mar 8, 3:14 pm, Tom Evans  wrote:

> Splitting down your project into small chunks will also demonstrate to
> people reading your proposal that you understand the subject matter,
> and they can have a high confidence of the project being delivered.

Thanks, I didn't know this and I don't have much experience writing a
spec, I'll keep this in mind when writing the final proposal.

> Weeks 9-10 made me smile though - no bug fixes allowed in the other weeks? :)

Well, there will be bug fixes every week, its just a final dash of
fixing bugs.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

[GSoC Proposal] Customizable Serialization

[Vivek Narayanan]

self, model):

   ...

def meta2_model(self, model):

   …

def meta_type(self, field):

   ...

===
Deliverables and Timeline
===

I would be working for about 40-45 hours each week and I would be
writing tests, exceptions and error messages along with development.
This would more or less be my timeline:

Till May 23

I will familiarize myself with community best practices and the
version control systems used by Django, read the code of all the
relevant modules related to the serialization framework, look at
implementations of other serialization libraries in different
languages and go through all the model and regression tests related to
serialization.

Weeks 1 to 2

I will use this time to set up the basic foundations of the projects
by :

   1. Writing the skeletal structure of the serializer, based on the
current implementations in core.serialization.base and
core.serialization.python.
   2. Setting up basic configurations for JSON, YAML, XML, Text and
creating the fixture mode.
   3. Making changes to loaddata and dumpdata in
core.management.commands to ensure backward compatibility.
   4. Using a dict as temporary storage before the final ‘dumping’
stage.
   5. Modifying the deserializers to handle custom formats of
serialization and specifying the requirements for deserialization.

Week 3

   1. Implementation of the metadata methods at field and model level
using getattr and similar methods.
   2. Make changes to the fields argument of ``serialize()``.
   3. Representation of output formats of the metadata in JSON/YAML,
XML etc.

Week 4

   1. Implementation of the unicode and datatype conversion methods in
a way similar to the metadata methods.
   2. Providing the user the choice of ‘protected’ types for the
serialization.

Week 5

   1. Provide all the configurable options for output formatting as
discussed above.
   2. Add support for string templates and their parsing.
   3. Parsing the dict used for temporary storage to generate XML and
custom text outputs

Week 6

   1. Implement serialization of related models like foreign keys and
many-to-many fields using recursion.
   2. Integrate with ``fields`` argument of ``serialize`` and specify
the format for representing a related model.
   3. Implement nesting depth limit feature.

Week 7

   1. Implement the ``exclude`` feature in serialize() which will
allow the user to choose fields to exclude while serializing.
   2. Adding an ``extras`` argument to serialize(), allowing the user
to specify additional properties of a model, which are not field
variables but derivatives of field variables and defined as methods or
properties in the model by the user.

Week 8

   1. Implement the permissions framework , which would give varying
levels of access to data to different users based on their permission
level.
   2. Integrate with Models API and field options.
   3. Add permission_level argument to serialize().

Weeks 9 - 10

   1. Write documentation for the project and provide many examples.
   2. Write a few tutorials on how to use the framework.
   3. Write some project-level tests, do some extensive final testing
and refine the project.

=
About
=

I am Vivek Narayanan, a second year undergrad student at the Institute
of Technology, Varanasi majoring in Electronics Engineering. I’m
really passionate about computers and have been programming for the
past 5-6 years. I have some experience in Python, C/C++, Java,
Javascript, HTML, PHP, Haskell and Actionscript. Python is my favorite
language and I’ve been using it for a couple of years. While working
on a web application, I stumbled upon Django a few months back and I
really love its elegant approach to everything.

I have submitted patches to tickets #15299 [1] , #12489 [2] and #8809
[3] on the Django Trac.

Some of the projects I’ve worked on are:

   1. blitzx86: An assembler for the Intel X86 Architecture using lex
and yacc. [4]
   2. An assembler for the dlx architecture supporting pipeline
optimizations. [5]
   3. mapTheGraph! - A location based social networking application on
web and Android platforms.
   4. A social networking game based on forex trading which is under
development.
   5. pyzok - A python based LAN chat server.  [6]
   6. aeroFox - A .NET based open source browser for Windows with
transparent  windows that managed over 10 downloads. [7]

I am a fast learner and can grasp new technologies / languages in a
short period of time. Among other things, I enjoy playing tennis and
reading books on a wide variety of topics.


Links


[1] http://code.djangoproject.com/ticket/15299

[2] http://code.djangoproject.com/ticket/12489

[3] http://code.djangoproject.com/ticket/8809

[4] https://github.com/vivekn/blitz8086

[5] https://github.com/vivekn/dlx-compiler

[6] https://github.com/vivekn/pyzok

[7] http://sourceforge.net/projects/aerofox/files/aerofox/0.4.8.7/

-- 
You received this message becaus

Re: Customizable Serialization

[Vivek Narayanan]

>From a previous discussion on this list (
http://groups.google.com/group/django-developers/browse_thread/thread/2da69b9e24cf3438/17d87e3b27d4395d
) I gather that modifying the field options of a Model is not
desirable due to a loss in orthogonality. Here is a modified
permissions framework for serialization.



Permission Framework
=
While creating an API, there may arise a need to give varying levels
of access to data to different people. For this I propose a permission
framework, where the user can choose to restrict data to certain
groups while defining a model. I guess a different name should be
used, so that it is not confused with the “Permission” model used in
contrib.auth and contrib.admin. Here’s an example

class User(models.Model):
name = CharField(max_length=128)
picture_url = URLField()
security_question = CharField(max_length = 200)
security_answer = CharField(max_length = 200)

class Meta:
  serialize_permissions = {
 ‘default’:  [‘name’],
 ‘admins’: [‘name’, ‘picture_url’,  ‘security_question’,
‘security_answer’ ],
 ‘friends’: [‘name’, ‘picture_url’],
 ‘self’: [‘name’, ‘picture_url’,  ‘security_question’]
}

Here different permission groups like ‘self’, ‘friends’ and ‘admins’
are created as a field of the Meta class . If no such field is
specified by the user, all fields are included under the default
permission group.

To use this, specify the permission_level in the call to serialize

data = serializers.serialize(queryset, permission_level = ‘friends’ )

=

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Customizable Serialization

[Vivek Narayanan]

> I also haven't seen any proposals or examples of how I'd use the API as
> an end user - are people going to be able to register serialisers to
> models (since they're apparently tied to specific models anyway)?

There will be different types of serializers like JSONSerializer,
YAMLSerializer, XMLSerializer etc . The end users will have to
subclass these, just like creating a new model.
While they need not be tied to specific models and can be used as
'generic' serializers, its finally down to the user's choice.


> I'm unclear about what meta2_ accomplishes - is it for things that are
> not fields, but still serialisable? Surely there's a better way to go
> about this?

It is for meta data of whole models (those implemented in the database
as "tables" and go through the start_model -> end_model cycle in the
current implementation of the serializer), basically a collection of
other fields, I believe the word 'model' is quite ambiguous in this
context. If there is a meta2_name method defined , the output would be
like this.

{
   "name": " Output of meta2_name() here ",
   ... # fields of the model follow
}

> > An extras argument, which would allow properties and data returned
> > by some methods to be serialized.
>
> How, exactly? What do you pass in this argument?

The extras option allows the user to serialize properties of a model
that are not fields. These properties may be almost any standard
python attribute or method. Say, there is a model , Article, defined
like this:

class Article(models.Model):
headline = models.CharField(maxlength=100, default='Default
headline')
contents = models.TextField()
pub_date = models.DateTimeField()
...
def get_permalink():
  #return some absolute URL



Eg: serialize(queryset, extras = ('Article.get_permalink') )

It takes an an iterable of strings in the format
"Object_Type.method_or_attribute_name".


>
> > -
> > Representing the existing serialization model
> > -
> > Here is an implementation of the existing serialization format in
> > JSON, this would be the fixture mode that I ve mentioned above.
>
> Presumably you're planning to leave the existing fixture-loading code as
> it currently is, given that there's no mention of it here? Are the
> customisable serialisers purely for use by other, non-Django
> applications in your plan?

I would be leaving most of the existing fixture loading code intact
but add support for deserializing fixtures in an arbitrary format.
These fixtures should contain some minimum required data for
generating the model like content type, required fields etc.


> How about if I just want to customise how the serialiser outputs
> DateTimeFields, or tell it how to serialise my new, shiny, custom field
> - does your proposal have any way to override things on a field type basis?

The data in the fields can be retrieved in the form of a python object
using the to_python method of the field as described here (
http://docs.djangoproject.com/en/dev/howto/custom-model-fields/ ). In
the case of the DateTimeField, it will be a datetime.datetime object.
Now, this is where the 'unicode_' methods come into play. By defining
a method "unicode_datetime_datetime()" in the serialiser , the
serialisation process for DateTimeFields  can be overridden. The same
holds good for custom fields. If this naming convention based on the
python type is confusing, I can implement it according to the Django
field type.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC Proposal] Customizable Serialization

[Vivek Narayanan]

Hi Russ,
Thanks for the long reply and all the suggestions. My comments are
inline.

> What if you need to support both? e.g.,
>
> 
>     the bar value
> 
>
> It seems to me that you would be better served providing a way to
> annotate each individual metadata value as (and I'm bikeshedding a
> name here) 'major' or 'minor'. JSON would render all metadata as
> key-values, and XML can make the distinction and render minor metadata
> as attributes, and major metadata as tags.
>

I think that's a great idea, this can be implemented with decorators
on the methods like @tag or @attribute while setting one of them as
default when no decorator is applied.

> I think I see where you're going here. However, I'm not sure it
> captures the entire problem.
>
> Part of the problem with the existing serializers is that they don't
> account for the fact that there's actually two subproblems to
> serialization:
>
>  1) How do I output the value of a specific field
>  2) What is the gross structure of an object, which is a collection of
> fields plus, plus metadata about an object, plus
>
> So, for a single object the JSON serializer currently outputs:
>
> {
>     "pk": 1,
>     "model": "myapp.mymodel",
>     "fields": {
>         "foo": "foo value",
>         "bar": "bar value"
>     }
>
> }
>
> Implicit in this format is a bunch of assumptions:
>
>  * That the primary key should be rendered in a different way to the
> rest of the fields
>  * That I actually want to include model metadata like the model name
>  * That the list of fields is an embedded structure rather than a list
> of top-level attributes.
>  * That I want to include all the fields on the model
>  * That I don't have any non-model or computed metadata that I want to include

I believe that my model of using a recursive method and storing
temporary data in 'levels' would address most of these concerns. The
method for handling a model would consist of the following steps,
roughly:

   * Get the list of fields to be serialized.
   * Now serialize each field , after checking for circular references
(see below), (using handle_field and apply all metadata, formatting
options etc) to a temporary python object, most probably as a key-
value in a  dict.
   * If an FK or M2M is encountered, check for nesting restrictions
and then recursively apply the handle_model method.
   * Add model level metadata, formatting options.
   * Process reverse relations, if required. (see below)
   * Store the model in some container in the serializer object.
   * Clear temp data in the current 'level'.

Finally, when all models are processed, dump the data from the
container into the required format.

> When you start dealing with foreign keys and m2m, you have an
> additional set of assumptions --
>
>  * How far should I traverse relations?

The user can specify a limit to the levels of nesting through
variable ``max_nesting_depth``.

>  * Do I traverse reverse relations?

In my opinion, traversing reverse relations can get really ugly at
times, especially when there are M2M fields, foreign keys or circular
relations involved. But there are some scenarios where the data is in
a relatively simpler format and serializing them would be useful. To
support this, I thought of something like this:

class Srz(Serializer):
   ...
   reverse_relations = [ (from_model_type, to_model_type), ... ]

But this should be used with caution and avoided when possible.

>  * How do I represent traversed objects? As FK values? As embedded objects?

As embedded objects, if the nesting depth limit is reached, then as FK
values.

>  * If they're embedded objects, how do I represent *their* traversed values?

Their traversed values would be represented just as a normal model
would be, with field-value mappings. The user can choose which fields
to dump.

>  * What happens with circular relations?

For all model type objects, like the base model in the query set and
all FK and M2M fields, some uniquely identifying data (like the
primary key, content type) will be stored in a list as each one of
them is processed. Before serializing a model, it would be checked if
the model is already on the list or not. If it is there, it is a
circular reference and that model would be ignored .

>  * If I have two foreign keys on the same model, are they both
> serialized the same way?

Yes.


> When you start dealing with the XML serializer, you have all these
> problems and more (because you have the attribute/tag distinction for
> each of these decisions, too -- for example, I may want some fields to
> be rendered as attributes, and some as tags.
>

For XML, I thought of using an intermediary container for a node that
would store all these details.

> > -
> > New features in the serialize() function
> > -
> > Apart from the changes I’ve proposed for the ``fields`` argument of
> > serialize, I would like to add

Re: Customizable Serialization

[Vivek Narayanan]

Hi Russ,

Thanks for the suggestions once again, I've thought of changing the
model for handling nested fields.

Each model can have a no of serializers, and they can be plugged in to
other serializers
and in this way nested models can be handled instead of cycling
through a tree of arbitrary depth.
Each serializer object would have a ``dump_object()`` method that
would return a python object.
This method is called when a serializer is plugged into another in a
nested model.

>My point is that it should be *possible* to define a "generic"
>serialization strategy -- after all, that's what Django does right
>now. If arguments like this do exist, they should essentially be
>arguments used to instantiate a specific serialization strategy,
>rather than something baked into the serialization API.

Yes, I'll be adding arguments ``attrs`` and ``exclude`` to the
__init__ method of the serializer.

Here is how the API can be used for generating the formats in those 2
examples:

"""
For the first option.
"""

class OriginalOutput(JSONSerializer):
wrap_fields = "fields"
indent = 4

def meta2_pk(self, obj):
#get pk
def meta2_model(self, obj):
#get model name

"""
The second option.
"""

class Base(JSONSerializer):
"""
A base serializer with formatting options.
"""
indent = 4

class Editor(Base):
"""
This can be used to serialize a Person model.
Note the company field, if no such Serializer object is specified,
the foreign key is returned.

By specifying the ``attrs`` argument while initializing the
object, one can restrict the fields
being serialized. Otherwise, all available fields and metadata
will be serialized.
"""
company = Company(from_field = 'company', attrs = ['name',
'founded'])

class Author(Base):
"""
The dump_object method is called when a serializer is plugged into
another and returns a python object,
this is normally a dict but this can be overridden.
"""
def dump_object(self, obj):
return obj.first_name + ' ' + obj.last_name


class BookDetails(Base):
"""
This is the serializer that will yield the final output.

Since the 'authors' field is M2M, the Author serializer will be
applied over the list of authors.

Aliases is a dict that maps attribute names to their labels during
serialization.

"""
wrap_all = "book details"
authors = Author(from_field = 'authors')
editor = Editor(from_field = 'editor', attrs = ['firstname',
'lastname', 'coworkers', 'phone', 'company'])
aliases = {
'title': 'book_title',
'acount': 'author_count'
}

def meta2_acount(self, obj):
# return count of authors


The serialized data can be obtained by calling BookDetails.serialize()
or OriginalOutput.serialize().

Some of the options I didn't cover in the above example are:

* A ``fieldset`` option, which is a list of fields to be serialized.
If it is not set, all fields are part of the
serializer. This can be additionally restricted during initialization
as shown above.

* Reverse relations can be used in the same way as other pluggable
serializers by
specifying the ``from_field`` argument as the related name during
initialization.

I hope this will add more flexibility.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Patch for using custom managers in a reverse relation. (#3871)

[Vivek Narayanan]

Hi,

I've added a patch that provides functionality for selecting a custom
manager in a reverse relation , rather than the default manager. For
example:

 author = Author.objects.get(id=1)
 # Selects the manager 'foobar' of Post.
 mgr = author.post_set.managers("foobar")

https://code.djangoproject.com/ticket/3871

Would be great if someone could review it.

Vivek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Patch for using custom managers in a reverse relation. (#3871)

[Vivek Narayanan]

Well, this is useful in the case where you have multiple custom
managers
defined on a model and you need to a select a specific manager. Doing
so,
 common filter operations that are long expressions or are quite
complicated,
 need not be repeated every time.

On Oct 4, 9:40 am, Tai Lee  wrote:
> Your custom manager could do nothing to filter results by default and
> so behave the same as the default automatic manager, but provide
> additional methods that filter the results as you need.

If the get_query_set() method of a manager is overridden, the
filtering
behavior can actually be changed. In the tests there is an example
does this.

>
> Cheers.
> Tai.
>
> On Oct 3, 3:25 pm, Vivek Narayanan  wrote:
>
>
>
>
>
>
>
> > Hi,
>
> > I've added a patch that provides functionality for selecting a custom
> > manager in a reverse relation , rather than the default manager. For
> > example:
>
> >          author = Author.objects.get(id=1)
> >          # Selects the manager 'foobar' of Post.
> >          mgr = author.post_set.managers("foobar")
>
> >https://code.djangoproject.com/ticket/3871
>
> > Would be great if someone could review it.
>
> > Vivek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.