Customizable serialization (again and again)

2011-10-14 Thread sebastien piquemal
Hi !

Again ! on this topic many times discussed :

http://groups.google.com/group/django-developers/search?group=django-developers&q=serialization&qt_g=Search+this+group
https://code.djangoproject.com/wiki/SummerOfCode2011#Customizableserialization

I came up with a framework that feels right to use (imo), allows to
customize literally everything, fulfils all the requirements listed in
the GSoC page, but is still (imo) quite simple. I thought I would
suggest it as a candidate for the new Django serialization framework
(in which case I am of course volunteering for the integration !).

The thing would be then to handle serialization (same goes for
deserialization) as a 2 steps process :
 ---{ the framework }-> 
---{ specialized library }> 

 would be whatever { specialized library }
accepts as an input. For example, for serializing to JSON,
 is a composition of dicts, lists, ints,
strings, ... And for XML, this  is whatever the
xml library handles (e.g. xml.etree.ElementTree.Element, or whatever).

The syntax for using my framework is django-ish :

- Declaring a serializer (deserializer, or whatever operation) :
http://dpaste.com/634114/

- Specifying what to output for django serializers : http://dpaste.com/634128/

More examples there (with links to API doc) : 
http://any2any.posterous.com/introducing-any2any
Repo : https://bitbucket.org/sebpiq/any2any

Any comments, thoughts, critics are much appreciated.

Cheers,

Sébastien

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Ability to get values using HTTP GET in admin changelist_view

2011-10-14 Thread Andrew Martin
Hello,

It would be of great help to me if the ability to pass variables
through HTTP GET were possible within the admin changelist view.  For
example, I use Django admin tools to manage security advisories that
come from various sources.  I need the ability to create a "meeting
view," which essentially is a filtered view with *multiple* filters
(e.g. all advisories marked as 'new' OR 'pending review' OR ...
whatever condition).  Currently, if I attempt to use a URL like the
following

http://example.com/admin/MY_SITENAME/MY_MODELNAME/?meeting_view=1

meeting_view gets rewritten as e=1 (indicating it's an invalid flag of
some kind).

My (horrible) workaround for this was to edit the Django options.py
file, adding the following block to changelist_view definition:

if 'meeting_view' in request.GET:
new_GET = {}
for key in request.GET:
   if key != 'meeting_view':
  new_GET[key] = request.GET[key]
request.GET = new_GET
self.meeting_view=1
else:
self.meeting_view=0

That catches the variable in request.GET, saves it to an attribute
(self.meeting_view), then rewrites request.GET without meeting_view in
it.  Amazingly, this has not blown up in my face yet.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Feature request prop for django db cache extention

2011-10-14 Thread Cal Leeming [Simplicity Media Ltd]
Hey all,

Today, we had a client getting around 600k webapp requests per hour (avg
6k/min), and had to do some emergency perf hotfixes (CodeIgniter+PHP)

In the end, we monkey-patched the code so raw SQL statement was used to
generate a cache key, and we performed a lookup on that. It was absolutely
great at preventing snowballing of things like tmp tables causing connection
backlog - and resolved the problem completely.

Anyway, it got me thinking that it would be nice if Django could do this out
of the box. Sure, the Django query cache works great, but when you have a
large cluster of servers, and large queries causing tmp tables, something
like this would be perfect.

Something like a DATABASE_CACHE='memcache://127.0.0.1:11211'

Does anyone have any thoughts about doing a feature request for this? If
feedback is positive, I'll put in a feature request and put some time aside
to write the patch.

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Feature request prop for django db cache extention

2011-10-14 Thread Javier Guerra Giraldez
On Fri, Oct 14, 2011 at 9:13 PM, Cal Leeming [Simplicity Media Ltd]
 wrote:
> In the end, we monkey-patched the code so raw SQL statement was used to
> generate a cache key, and we performed a lookup on that.

why SQL? it seems a memcached INCR would be faster and easier to
distribute (if you have a different INCR for each object, it should
naturally fall on different shards)

-- 
Javier

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Feature request prop for django db cache extention

2011-10-14 Thread Russell Keith-Magee
On Saturday, October 15, 2011, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:
> Hey all,
> Today, we had a client getting around 600k webapp requests per hour (avg
6k/min), and had to do some emergency perf hotfixes (CodeIgniter+PHP)
> In the end, we monkey-patched the code so raw SQL statement was used to
generate a cache key, and we performed a lookup on that. It was absolutely
great at preventing snowballing of things like tmp tables causing connection
backlog - and resolved the problem completely.
> Anyway, it got me thinking that it would be nice if Django could do this
out of the box. Sure, the Django query cache works great, but when you have
a large cluster of servers, and large queries causing tmp tables, something
like this would be perfect.
> Something like a DATABASE_CACHE='memcache://127.0.0.1:11211'
> Does anyone have any thoughts about doing a feature request for this? If
feedback is positive, I'll put in a feature request and put some time aside
to write the patch.

It sounds to me like you've just rediscovered Django ticket #17.

We've made the decision that this sort of caching would fundamentally change
the design contract of the ORM, and there isn't a single obvious way to
solve the problem that we could provide as an optional behavior (cache
setting is fine; it's cache invalidation where problems arise).

However, the good news is that it can be handled as an external extension to
the ORM. There are a number of external model caching libraries that address
this problem; for example Johnny Cache and Cache Machine.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.