Re: Django UnicodeEncodeError in errorlist

2012-04-27 Thread Thomas Guettler

this question belongs to django-user.

You need to use u'å' not 'å' in your python code.

> Is there a good reason
> why |force_unicode| and |lazy| do not use |smart_str|?

smart_str() creates a bytestring, but in Django everything is unicode until the 
end. Only one of the last
steps is to encode the unicode result to (most of the times) utf8.

I have seem a lot of unicode errors in django. But most of them were fixed long 
ago.

The last (I know of): https://code.djangoproject.com/ticket/18063
  "repr() should return only ascii, not unicode"

  Thomas

Am 26.04.2012 20:41, schrieb Andrei:

Hi,

I am having an issue with rendering Django's ErrorList if one of my error list 
items is unicode. When Django renders my
errorlist

|{{  form.non_field_errors}}
|

it runs the following code 
:

|class  ErrorList(list,  StrAndUnicode):
"""
A collection of errors that knows how to display itself in various 
formats.
"""
def  __unicode__(self):
return  self.as_ul()

def  as_ul(self):
if  not  self:  return  u''
return  mark_safe(u'%s'
%  ''.join([u'%s'  %  
conditional_escape(force_unicode(e))  for  ein  self]))
|

then in |force_unicode| 
:

|s=  unicode(str(s),  encoding,  errors)
|

and then translation in |lazy| 
:

|def  __str_cast(self):
return  str(self.__func(*self.__args,  **self.__kw))
|

The problem is that my string contains 'å' symbol and |str(u'å')| raises 
|UnicodeEncodeError|. Is there a good reason
why |force_unicode| and |lazy| do not use |smart_str|? I have to do it myself 
and provide error messages as |str|
objects instead of unicode to make it work.

So I get TemplateSyntaxError /Caught UnicodeEncodeError while rendering: 
'ascii' codec can't encode character u'\xe5' in
position 17: ordinal not in range(128)/. This seems telling that rendering my 
error list item (which is |u'å'|) caused
the first UnicodeEncodeError having unicode message /'ascii' codec can't encode 
character u'\xe5'/ and then second
UnicodeEncodeError while rendering the message from the first one. Am I 
mistaken?

Django version: 1.3.1 (but this seems to happen in 1.4 as well)

Full traceback: 
https://raw.github.com/gist/2499077/ba60cb752acdb429dd6c2814ffb24272037a367a/UnicodeEncodeError.txt

Thanks,

Andrei



--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/pYaefZxhll4J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.


--
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



[GSoC] Customizable Serialization check-in

2012-04-27 Thread Piotr Grabowski

Hi!

I'm Piotr Grabowski, student from University of Wroclaw, Poland
In this Google Summer of Code I will  deal with problem of customizable 
serialization in Django.


You can find my proposal here https://gist.github.com/2319638

It's obviously not a finished idea, it's need to be simplified for sure. 
My mentor Russel Keith Magee told me to look at Tom Christie's 
serialization API. I found it similar to my proposal, there is a lot in 
common - declarative fields, same approach to various aspect of 
serialization , but his API is simpler and it feels better.


Since Tom already post on group about his project I can refer to it:

W dniu 27.04.2012 06:44, Tom Christie pisze:

...

Given that Piotr's GSoC proposal has now been accepted, I'm wondering 
what the
right way forward is?  I'd like to continue to push forward with this, 
but I'm
also aware that it might be a bit of an issue if there's already an 
ongoing

GSoC project along the same lines?

Having taken a good look through the GSoC proposal, it looks good, and 
there

seems to be a fair bit of overlap, so hopefully he'll find what I've done
useful, and I'm sure I'll have plenty of comments on his project as it
progresses.

I'd consider suggesting a collaborative approach, but the rules of the 
GSoC

wouldn't allow that right?

--
Like I said above, your work will be very useful for me. I must read 
GSoC regulations carefully but for sure collaboration with code writing 
is impossible. I don't know that I could use your existing code base but 
I think it's also impossible. However sharing ideas and discuss how the 
API should look and work it will be very desirable.



My plan for next few weeks is to meet Django contribution requirements, 
solve ticket to prove I now the process off doing it, and what's most 
important  have discussion about serialization API. I hope community 
will be interested in this feature.


After weekend I will post my proposal with updates from Tom's API.

--
Piotr Grabowski


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Django UnicodeEncodeError in errorlist

2012-04-27 Thread Andrei
Hi Thomas,

You seem read my email too quickly. I do use u'å'. In force_unicode() and 
lazy() Django runs str(u'å') which results in the exception(s). If it would 
use smart_str instead, the exception would not happen. Am I missing 
something?

Andrei


On Friday, April 27, 2012 9:11:19 AM UTC+2, guettli wrote:
>
>
> You need to use u'å' not 'å' in your python code. 
>  
>
smart_str() creates a bytestring, but in Django everything is unicode until 
> the end. Only one of the last 
> steps is to encode the unicode result to (most of the times) utf8. 
>
>
> Am 26.04.2012 20:41, schrieb Andrei: 
> > Hi, 
> > 
> > I am having an issue with rendering Django's ErrorList if one of my 
> error list items is unicode. When Django renders my 
> > errorlist 
> > 
> > |{{  form.non_field_errors}} 
> > | 
> > 
> > it runs the following code <
> https://github.com/django/django/blob/1.3.1/django/forms/util.py#L46>: 
> > 
> > |class  ErrorList(list,  StrAndUnicode): 
> > """ 
> > A collection of errors that knows how to display itself in 
> various formats. 
> > """ 
> > def  __unicode__(self): 
> > return  self.as_ul() 
> > 
> > def  as_ul(self): 
> > if  not  self:  return  u'' 
> > return  mark_safe(u'%s' 
> > %  ''.join([u'%s'  % 
>  conditional_escape(force_unicode(e))  for  ein  self])) 
> > | 
> > 
> > then in |force_unicode| <
> https://github.com/django/django/blob/1.3.1/django/utils/encoding.py#L74>: 
>
> > 
> > |s=  unicode(str(s),  encoding,  errors) 
> > | 
> > 
> > and then translation in |lazy| <
> https://github.com/django/django/blob/1.3.1/django/utils/functional.py#L209>: 
>
> > 
> > |def  __str_cast(self): 
> > return  str(self.__func(*self.__args,  **self.__kw)) 
> > | 
> > 
> > The problem is that my string contains 'å' symbol and |str(u'å')| raises 
> |UnicodeEncodeError|. Is there a good reason 
> > why |force_unicode| and |lazy| do not use |smart_str|? I have to do it 
> myself and provide error messages as |str| 
> > objects instead of unicode to make it work. 
> > 
> > So I get TemplateSyntaxError /Caught UnicodeEncodeError while rendering: 
> 'ascii' codec can't encode character u'\xe5' in 
> > position 17: ordinal not in range(128)/. This seems telling that 
> rendering my error list item (which is |u'å'|) caused 
> > the first UnicodeEncodeError having unicode message /'ascii' codec can't 
> encode character u'\xe5'/ and then second 
> > UnicodeEncodeError while rendering the message from the first one. Am I 
> mistaken? 
> > 
> > Django version: 1.3.1 (but this seems to happen in 1.4 as well) 
> > 
> > Full traceback: 
> https://raw.github.com/gist/2499077/ba60cb752acdb429dd6c2814ffb24272037a367a/UnicodeEncodeError.txt
>  
> > 
> > Thanks, 
> > 
> > Andrei 
> > 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "Django developers" group. 
> > To view this discussion on the web visit 
> https://groups.google.com/d/msg/django-developers/-/pYaefZxhll4J. 
> > To post to this group, send email to django-developers@googlegroups.com. 
>
> > To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com. 
> > For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en. 
>
> -- 
> Thomas Guettler, http://www.thomas-guettler.de/ 
> E-Mail : guettli (*) 
> thomas-guettler + de 
>

On Friday, April 27, 2012 9:11:19 AM UTC+2, guettli wrote:
>
> this question belongs to django-user. 
>
> You need to use u'å' not 'å' in your python code. 
>
>  > Is there a good reason 
>  > why |force_unicode| and |lazy| do not use |smart_str|? 
>
> smart_str() creates a bytestring, but in Django everything is unicode 
> until the end. Only one of the last 
> steps is to encode the unicode result to (most of the times) utf8. 
>
> I have seem a lot of unicode errors in django. But most of them were fixed 
> long ago. 
>
> The last (I know of): https://code.djangoproject.com/ticket/18063 
>"repr() should return only ascii, not unicode" 
>
>Thomas 
>
> Am 26.04.2012 20:41, schrieb Andrei: 
> > Hi, 
> > 
> > I am having an issue with rendering Django's ErrorList if one of my 
> error list items is unicode. When Django renders my 
> > errorlist 
> > 
> > |{{  form.non_field_errors}} 
> > | 
> > 
> > it runs the following code <
> https://github.com/django/django/blob/1.3.1/django/forms/util.py#L46>: 
> > 
> > |class  ErrorList(list,  StrAndUnicode): 
> > """ 
> > A collection of errors that knows how to display itself in 
> various formats. 
> > """ 
> > def  __unicode__(self): 
> > return  self.as_ul() 
> > 
> > def  as_ul(self): 
> > if  not  self:  return  u'' 
> > return  mark_safe(u'%s' 
> > %  ''.join([u'%s'  % 
>  con

Re: Customizable Serialization check-in

2012-04-27 Thread Anssi Kääriäinen
On Apr 27, 11:14 am, Piotr Grabowski  wrote:
> Hi!
>
> I'm Piotr Grabowski, student from University of Wroclaw, Poland
> In this Google Summer of Code I will  deal with problem of customizable
> serialization in Django.
>
> You can find my proposal here https://gist.github.com/2319638

I quickly skimmed the proposal and I noticed speed/performance wasn't
mentioned. I believe performance is important in serialization and
especially in deserialization. It is not the number one priority item,
but it might be worth it to write a couple of benchmarks (preferably
to djangobench [1]) and check that there are no big regressions
introduced by your work. If somebody already has good real-life
testcases available, please share them...

 - Anssi

[1] https://github.com/jacobian/djangobench/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Customizable Serialization check-in

2012-04-27 Thread Piotr Grabowski

W dniu 27.04.2012 10:36, Anssi Kääriäinen pisze:

On Apr 27, 11:14 am, Piotr Grabowski  wrote:

Hi!

I'm Piotr Grabowski, student from University of Wroclaw, Poland
In this Google Summer of Code I will  deal with problem of customizable
serialization in Django.

You can find my proposal here https://gist.github.com/2319638

I quickly skimmed the proposal and I noticed speed/performance wasn't
mentioned. I believe performance is important in serialization and
especially in deserialization. It is not the number one priority item,
but it might be worth it to write a couple of benchmarks (preferably
to djangobench [1]) and check that there are no big regressions
introduced by your work. If somebody already has good real-life
testcases available, please share them...

  - Anssi

[1] https://github.com/jacobian/djangobench/


I didn't think about performance a lot. There will be regressions.
Now serialization is very simple: Iterate over fields, transform it into 
string (or somethink serializable), serialize it with json|yaml|xml.
In my approach it is: transform (Model) object to Serializer object, 
each field from original object is  FieldSerializer object, next  (maybe 
recursively) get native python type object from each field, serialize it 
with json|yaml|xml.
I can do some optimalizations in this process but it's clear it will 
take longer to serialize (and deserialize) object then now. It can be 
problem with time taken by tests if there is a lot of fixtures.
I will try to write good, fast code but I will be very glad if someone 
give me tips about performance bottlenecks in it.


--
Piotr Grabowski

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Customizable Serialization check-in

2012-04-27 Thread Anssi Kääriäinen
On Apr 27, 12:11 pm, Piotr Grabowski  wrote:
> I didn't think about performance a lot. There will be regressions.
> Now serialization is very simple: Iterate over fields, transform it into
> string (or somethink serializable), serialize it with json|yaml|xml.
> In my approach it is: transform (Model) object to Serializer object,
> each field from original object is  FieldSerializer object, next  (maybe
> recursively) get native python type object from each field, serialize it
> with json|yaml|xml.
> I can do some optimalizations in this process but it's clear it will
> take longer to serialize (and deserialize) object then now. It can be
> problem with time taken by tests if there is a lot of fixtures.
> I will try to write good, fast code but I will be very glad if someone
> give me tips about performance bottlenecks in it.

One possibility is to have a fast-path for simple cases. But,
premature optimization is the root of all evil, so lets first see how
fast the code is, and then check if anything needs to be done.

I still think it is a good idea to actually check how fast the new
serialization code is, not just assume it is fast enough. So, please
include some simple benchmarks in your project.

I hope users who have a need for fast serialization will participate
in this discussion by telling their use cases.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Customizable Serialization check-in

2012-04-27 Thread Tom Christie
Hey Piotr,

  Thanks for the quick response.

> However sharing ideas and discuss how the API should look and work it 
will be very desirable.

That'd be great, yup.  I've got a couple of comments and questions about 
bits of the API, but I'll wait until you've had a chance to post your 
proposal to the list before starting that discussion. 

> I quickly skimmed the proposal and I noticed speed/performance wasn't 
mentioned. I believe performance is important in serialization and 
especially in deserialization.

Right.  Also worth considering is making sure the API can deal with 
streaming large querysets,
rather than loading all the data into memory at once.
(See also https://code.djangoproject.com/ticket/5423)

- Tom.

On Friday, 27 April 2012 10:11:56 UTC+1, Piotr Grabowski wrote:
>
> W dniu 27.04.2012 10:36, Anssi K��ri�inen pisze: 
> > On Apr 27, 11:14 am, Piotr Grabowski  wrote: 
> >> Hi! 
> >> 
> >> I'm Piotr Grabowski, student from University of Wroclaw, Poland 
> >> In this Google Summer of Code I will  deal with problem of customizable 
> >> serialization in Django. 
> >> 
> >> You can find my proposal here https://gist.github.com/2319638 
> > I quickly skimmed the proposal and I noticed speed/performance wasn't 
> > mentioned. I believe performance is important in serialization and 
> > especially in deserialization. It is not the number one priority item, 
> > but it might be worth it to write a couple of benchmarks (preferably 
> > to djangobench [1]) and check that there are no big regressions 
> > introduced by your work. If somebody already has good real-life 
> > testcases available, please share them... 
> > 
> >   - Anssi 
> > 
> > [1] https://github.com/jacobian/djangobench/ 
> > 
> I didn't think about performance a lot. There will be regressions. 
> Now serialization is very simple: Iterate over fields, transform it into 
> string (or somethink serializable), serialize it with json|yaml|xml. 
> In my approach it is: transform (Model) object to Serializer object, 
> each field from original object is  FieldSerializer object, next  (maybe 
> recursively) get native python type object from each field, serialize it 
> with json|yaml|xml. 
> I can do some optimalizations in this process but it's clear it will 
> take longer to serialize (and deserialize) object then now. It can be 
> problem with time taken by tests if there is a lot of fixtures. 
> I will try to write good, fast code but I will be very glad if someone 
> give me tips about performance bottlenecks in it. 
>
> -- 
> Piotr Grabowski 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/K9cslx5Fa_sJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Customizable Serialization check-in

2012-04-27 Thread Piotr Grabowski

W dniu 27.04.2012 12:39, Tom Christie pisze:

Hey Piotr,


> I quickly skimmed the proposal and I noticed speed/performance wasn't
mentioned. I believe performance is important in serialization and
especially in deserialization.

Right.  Also worth considering is making sure the API can deal with 
streaming large querysets,

rather than loading all the data into memory at once.
(See also https://code.djangoproject.com/ticket/5423)

- Tom.

Maybe it can be done with chain of two black box generators. First 
generator input are queryset (iterable sequence) and  user defined 
Serializer class contains how to transform single object and output is 
python primitive type objects. Second is feed with this objects and 
outputs serialized_string. What with nested objects - more generators? 
Generators are good because we can also reuse Serializer objects == 
better performance. But like Anssi said - optimize after the code is 
written, not before :)


--
Piotr Grabowski

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Problems with date-based generic views when USE_TZ=True

2012-04-27 Thread Mike Yumatov
Hi!

I wrote a ticket about time zone warnings in date-based generic views: 
https://code.djangoproject.com/ticket/18217

After some research, I understand that there are more problems with this views, 
than I thought at first. django.views.generic.dates module uses aware and naive 
datetime objects together, which breaks all date-based views except 
YearArchiveView.

Let's have, for example:
- America/Chicago as TIME_ZONE
- an article, published at 2012-12-31 23:00:00 in local time zone (2012-01-01 
05:00:00 UTC)
- an url /articles/{year}/ which returns list of articles for requested year
- an url /articles/{year}/{month}/ which returns list of articles for requested 
month

As I understand, year and month must be in UTC in urls, or date_list from all 
date-based generic views is useless, because it contains dates in UTC.

So, our article must be in /articles/2012/ and in /articles/2012/01/. But it 
won't be in /articles/2012/01/, because MonthArchiveView will use datetime.date 
objects, which will be treated as naive datetime.datetime object by ORM, and 
local time zone will be used.

The same thing is true for DayArchiveView, TodayArchiveView and DateDetailView.

This issue affects trunk and 1.4 versions of Django. 

-- 
Mike Yumatov


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



GSoC Check-in: Security Enhancements

2012-04-27 Thread Rohan Jain
Hi,

I am Rohan Jain, a student from Indian Institute of Technology,
Kharagpur. I'll be doing a Google Summer of Code project with django
this year under the title "Security Enhancements". As the title
suggests, it has something to do with Security Enhancements: like
improvements in CSRF protection and tokenization.

I have made some small updates to the proposal with the feedback it
got. It is under VC over here: http://gist.github.com/2203174
There isn't a direct way to diff gists, so here are the changes I did
if somebody has already read the proposal:

 - The origin check will be an additional step to ensure a valid
   request and not standalone. The conventional checks will still
   exist.

 - Add some issues Luke pointed out about signing and using sessions.

 - Add info about my github fork and branches.

What I will be doing the following week:

 - I haven't done any major contribution to django yet apart from a
   tiny ticket some time ago. So, I'll be working on an ticket next
   few weeks. It is related to filesystem backend of contrib.sessions,
   was raised some time ago:
   https://code.djangoproject.com/ticket/18194

 - Cleanup and organize the proposal a bit more (Probably start
   tracking it as the CSRF protection page -
   https://code.djangoproject.com/wiki/CsrfProtection)

(I have also appended the current proposal below in this post)

--
Rohan


Proposal


#Abstract

Django is a reasonably secure framework. It provides an API and
development patterns which transparently take care of the common web
security issues. But still there are security features which need
attention. I propose to work and improved CSRF checking without any
compromises and on integration of existing work on centralized token
system. If time permits I will also attempt on integration of
django-secure.

#Description
##CSRF Improvements

Cross-Origin Resource Sharing (CORS):  
W3C has a working draft regarding [CORS][w3c-cors-draft], which opens
up the possibility for allowing client-side request cross-origin
requests. This directly triggers in mind the capability to develop
API which can be exposed directly to the web browser. This would let
us get rid of proxies and other hacks used to achieve this.
Currently all the major browsers support this: Chrome (all versions),
Firefox (> 3.0), IE (> 7.0), Safari (> 3.2), Opera (> 12.0). Firefox
and Chrome send the origin header for both AJAX and standard from POST
requests. Introduced it here as some further parts of the post refer
to this.

###Origin checking

With CORS around need for using CSRF token can be dropped, at least in
some browsers. [Ticket #16859][orig-check-ticket], is an attempt for
that. But this was rejected because of neglecting the case for
presence of `CSRF_COOKE_DOMAIN` (Refer to the closing comment on the
ticket for details). So to handle this we need to simulate checking of
CSRF cookie domain as web browsers do it. Maybe:

```python

reqest.META.get('HTTP_ORIGIN').endswith(settings.CSRF_COOKIE_DOMAIN)

```

In case the server receives an origin header in the request, it will
be used for an initial checking and then all the conventional checks
will be done. The general security will automatically be improved with
the increased market share of newer browsers which support Origin
Header.

As the closing comment points it out, we can't do this with secure
requests. They need to be essentially checked against the referrer or
origin, at least for now. We can not be sure that some untrusted or
insecure subdomain has not already set the cookie or cookie domain.
To deal with this, we have to consider https separately as it is
being done now. So it will be something like:

```python
def process_view(self, request, ):

# Same initial setup

if request.method not in ('GET', 'HEAD', 'OPTIONS', 'TRACE'):

host = request.get_host()
origin = reqest.META.get('HTTP_ORIGIN', "")
cookie_domain = settings.CSRF_COOKIE_DOMAIN

if request.is_secure():
good_referer = 'https://%s/' % host
referer = origin or request.META.get('HTTP_REFERER')
# Do the same origin checks here

# We are insecure, so care less
# A better way for this check can be used if needed
elif origin.endswith(cookie_domain):
# Safe, continue conventional checking

# Do the conventional checks here
```

If the above were to be implemented, the setting `CSRF_COOKIE_DOMAIN`
should be deprecated for something like `CSRF_ALLOWED_DOMAIN` which
makes more sense.

###Multiple Allowed Domains (was Better CORS Support)
Since, already introducing Origin checking, we can go one step further
and try to provide better support for CORS for browsers supporting it.
A tuple/list setting, which specifies allowed domains will be
provided. Using this the various access control allowance response
headers will be set when the request origin is from amongst the
allowed domains. For CSRF check, just se

Re: Implementation of Object Permissions

2012-04-27 Thread Daniel Sokolowski
Fantastic job on the performance benchmarks, the number 4-5 seconds per check 
is scary. I’ve posted a comment on django-guradian in hopes the author can 
chime in onto this conversation by posting 
https://groups.google.com/forum/#!topic/django-developers/6UNjPu1mcgc/discussion.
  

Are there any hopes of speeding up or caching GenericForgeinKey queries ? I 
found -  
http://zerokspot.com/weblog/2008/08/13/genericforeignkeys-with-less-queries/ – 
can your solution be backward compatible?

And yes you are correct there is no GenericForeginKey on the Permission model, 
my bad; at the moment my understanding of this limits me really to just the 
opinion level. 

From: Moritz S. 
Sent: Monday, April 23, 2012 8:39 PM
To: django-developers@googlegroups.com 
Cc: tguett...@tbz-pariv.de 
Subject: Re: Implementation of Object Permissions

  With an appropriate index, SQL databases should be able to find rows matching 
a condition on two columns about as quickly as if the condition was only on one 
column, even in a table containing million of objects.

Databases can of course find the rows with appropriate indexes, but the problem 
is that GenericForeignKeys are precisely not such appropriate ones. 
  I'd like to see a benchmark before rejecting this idea on performance 
grounds. Could you point us to some data supporting your assertion that this 
isn't scalable?

To prove the increased performance I actually did some benchmark. I used the 
script in the attachment. It needs a model that has only one field called text. 
It randomly generates 10,000 users and 1,000,000 objects. Then it assigns to 
each user respectively 1,000 object permissions of randomly selected previously 
created objects. At least it checks for each user respectively 1,000 randomly 
selected objects again. That's what I got:

For the first version of my contrib.auth patch:
Summary:
granted permissions in 1 day, 10:41:07.523402
checked permissions in 22:48:44.662974

For a module using GenericForeignKeys (I used django-guardian because it seemed 
to be the most used one):
granted permissions in 11:31:12.216808
checked permissions 

I don't have the results for checking permissions with django-guardian yet 
because it took ages to check them. Each single check for a object permission 
took about 4 to 5 seconds! Now think about how long it would take to check for 
all users (10,000*1,000*4secs)! I think that is out of all proportion to my 
suggested solution.
This happens because of the lack of a appropriate index as you mentioned 
before. So the database has to do a full table scan and that's definitely not 
scalable at all.
By the way, the reason for the GenericForeignKeys being faster in creating them 
is that the database does not have to create the indexes (i.e. the foreign 
keys). 
  I'm -1 on a contrib app (django.contrib.auth) injecting a dynamic field into 
every model.

  (Yes, Django does that for the primary key, to avoid declaring it in every 
model. But it's core, not a contrib app.)

I totally agree with this but I don't see another way at the moment. 
  What's the definition of "when the auth app is initialized"? Wouldn't your 
solution required that django.contrib.auth be loaded before all other apps?

Unfortunately it does but I have a open mind about better implementations. 
  With the solution that I prefer, this doesn't seem absolutely necessary, the 
following statements are simple enough:


  ObjectPermission.objects.create(user=user, object=instance)
  ObjectPermission.objects.get(user=user, object=instance).delete()

I think that's matter of taste ;-) 

  Your need to add ‘object_permissions’ to a model is an incorrect approach in 
my opinion as it would require any old apps to undergo a schema migration. Your 
generic relations concerns might be valid however I would like to mention 
Django guardian does use generic FK but so does built in django auth – so I 
think this approach is just fine.
You don't have to add this field manually, auth app does that. And django auth 
does not use generic relationships, it only makes use of the ContentType field 
to comfortably store the type assigned to a permission. 

Best regards,
Moritz
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/fp-Bu8sgUq0J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.


 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.g

GitHub migration

2012-04-27 Thread Adrian Holovaty
Hey guys, here's an important heads-up!

We're going to do the migration to GitHub today. This means we'll no
longer be committing code to our Subversion repository. Committers,
please hold off on making commits until the migration is done.

I expect it'll be done by late afternoon Chicago time. I'm going to do
a dry run first to make sure all goes well, then I'll do the real
thing.

Here are some random notes about the process --

* Any fork of the existing GitHub repository at
https://github.com/django/django (which was a mirror of Subversion)
will be broken. That is, it will no longer be able to get changes from
upstream. Unfortunately, there's no way around this. But we'll provide
some instructions here on django-developers on how to change your
forks to use the new upstream repository in such a way that any of
your local changes will be preserved. Worst case, you'll just have to
generate a patch of your local branch's changes and apply it to a new
fork.

* One interesting part of this is coming up with a definitive mapping
between our old Subversion usernames and names/emails for Git. Brian
Rosner and some other folks have done a great job of doing that
research -- 
https://github.com/brosner/django-git-authors/blob/master/authors.txt
-- and we'll be using that file to convert committer data during the
migration. Basically this means that if you've ever committed to
Django, your commits will be associated with your current GitHub
account -- as long as your GitHub account is associated with the same
email address in that authors file. Due to the way Git works, there's
no way of changing this data after the import is done. But we've
accounted for everybody except two mysterious people "dcf" and "cell,"
both of whom were given temporary commit bits during a sprint six
years ago.

* If you're a Git/GitHub expert and are interested in helping, feel
free to join us in #django-dev on Freenode.

* Thanks in advance for bearing with us during this process. There
will be some pain, but it'll be worth it in the long run. I'm excited!

Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: GitHub migration

2012-04-27 Thread Brendan Smith
woohoo!  this is great,   good luck Adrian



Brendan Smith, IT Specialist
National Priorities Project
http://www.nationalpriorities.org
http://www.facebook.com/nationalpriorities
413 584 9556




On Apr 27, 2012, at 12:50 PM, Adrian Holovaty wrote:

> Hey guys, here's an important heads-up!
> 
> We're going to do the migration to GitHub today. This means we'll no
> longer be committing code to our Subversion repository. Committers,
> please hold off on making commits until the migration is done.
> 
> I expect it'll be done by late afternoon Chicago time. I'm going to do
> a dry run first to make sure all goes well, then I'll do the real
> thing.
> 
> Here are some random notes about the process --
> 
> * Any fork of the existing GitHub repository at
> https://github.com/django/django (which was a mirror of Subversion)
> will be broken. That is, it will no longer be able to get changes from
> upstream. Unfortunately, there's no way around this. But we'll provide
> some instructions here on django-developers on how to change your
> forks to use the new upstream repository in such a way that any of
> your local changes will be preserved. Worst case, you'll just have to
> generate a patch of your local branch's changes and apply it to a new
> fork.
> 
> * One interesting part of this is coming up with a definitive mapping
> between our old Subversion usernames and names/emails for Git. Brian
> Rosner and some other folks have done a great job of doing that
> research -- 
> https://github.com/brosner/django-git-authors/blob/master/authors.txt
> -- and we'll be using that file to convert committer data during the
> migration. Basically this means that if you've ever committed to
> Django, your commits will be associated with your current GitHub
> account -- as long as your GitHub account is associated with the same
> email address in that authors file. Due to the way Git works, there's
> no way of changing this data after the import is done. But we've
> accounted for everybody except two mysterious people "dcf" and "cell,"
> both of whom were given temporary commit bits during a sprint six
> years ago.
> 
> * If you're a Git/GitHub expert and are interested in helping, feel
> free to join us in #django-dev on Freenode.
> 
> * Thanks in advance for bearing with us during this process. There
> will be some pain, but it'll be worth it in the long run. I'm excited!
> 
> Adrian
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Problems with date-based generic views when USE_TZ=True

2012-04-27 Thread Aymeric Augustin
Hi Mike,

On 27 avr. 2012, at 14:51, Mike Yumatov wrote:
> I wrote a ticket about time zone warnings in date-based generic views: 
> https://code.djangoproject.com/ticket/18217

It isn't necessary to notify django-developers when you create a ticket; that's 
the job of django-updates.

> After some research, I understand that there are more problems with this 
> views, than I thought at first. django.views.generic.dates module uses aware 
> and naive datetime objects together, which breaks all date-based views except 
> YearArchiveView.
> 
> Let's have, for example:
> - America/Chicago as TIME_ZONE
> - an article, published at 2012-12-31 23:00:00 in local time zone (2012-01-01 
> 05:00:00 UTC)
> - an url /articles/{year}/ which returns list of articles for requested year
> - an url /articles/{year}/{month}/ which returns list of articles for 
> requested month
> 
> As I understand, year and month must be in UTC in urls

They are in local time in the default time zone.

> or date_list from all date-based generic views is useless, because it 
> contains dates in UTC.

I'm not sure I understand your point there.

> So, our article must be in /articles/2012/ and in /articles/2012/01/. But it 
> won't be in /articles/2012/01/, because MonthArchiveView will use 
> datetime.date objects, which will be treated as naive datetime.datetime 
> object by ORM, and local time zone will be used.

This is an ill-defined problem, and also a hard one.

The behavior you're describing is the expected behavior, it's mostly backwards 
compatible and in my opinion it's the least unreasonable option.

See also :
https://docs.djangoproject.com/en/dev/topics/i18n/timezones/#faq
http://groups.google.com/group/django-developers/browse_thread/thread/cf0423bbb85b1bbf

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Django Admin Revamp - Any updates?

2012-04-27 Thread h3
I don't know the status of this project, but my guess is that you
shouldn't hold your breath for it.

Fortunately, there is Grappelli: https://github.com/sehmaschine/django-grappelli

We are currently working on making it compatible with django 1.4 (see
the "grappelli_2_4" branch)

Alternatively the branch on my fork works pretty well with 1.4:
https://github.com/h3/django-grappelli/tree/grappelli_2_4

Cheers

On Apr 26, 7:06 pm, Victor Hooi  wrote:
> Hi,
>
> Is there any news on the Django Admin rewrite front?
>
> I remember around a year ago, there was quite a bit of talk on revamping
> the Django admin UI - I think Idan Gazit was heading that, right?
>
> Is that still on the Django roadmap? Any idea of whether it'll be in 1.5,
> 1.6, 1.7 etc?
>
> Cheers,
> Victor

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Django Admin Revamp - Any updates?

2012-04-27 Thread Idan Gazit
Hey all,

Yeah, you aren't missing anything — originally I wanted to wait for the 
formrendering stuff, but that never panned out, and then I got busy.

Grappelli is lovely. Without knocking it at all, I think that the next version 
of the admin should be as forward-looking as the last version, which means 
rethinking what the admin does, not just how it looks. Unfortunately, I haven't 
stepped up to champion this beyond idle thoughts.

As noted, "don't hold your breath" would be wise. Even if we did jump in on an 
admin rewrite tomorrow, the fruits of that labour won't be one release in the 
making. The transition would probably resemble newforms, with "newadmin" 
appearing alongside "admin" in one release, then admin deprecated to "oldadmin" 
for one release, and finally disappearing after that. We wouldn't even release 
"newadmin" as a thing until we had something functional and mostly stable. I 
don't see much point in speculating when that might happen.

-I  

On Friday, April 27, 2012 at 11:01 PM, h3 wrote:

> I don't know the status of this project, but my guess is that you
> shouldn't hold your breath for it.
>  
> Fortunately, there is Grappelli: 
> https://github.com/sehmaschine/django-grappelli
>  
> We are currently working on making it compatible with django 1.4 (see
> the "grappelli_2_4" branch)
>  
> Alternatively the branch on my fork works pretty well with 1.4:
> https://github.com/h3/django-grappelli/tree/grappelli_2_4
>  
> Cheers
>  
> On Apr 26, 7:06 pm, Victor Hooi http://gmail.com)> 
> wrote:
> > Hi,
> >  
> > Is there any news on the Django Admin rewrite front?
> >  
> > I remember around a year ago, there was quite a bit of talk on revamping
> > the Django admin UI - I think Idan Gazit was heading that, right?
> >  
> > Is that still on the Django roadmap? Any idea of whether it'll be in 1.5,
> > 1.6, 1.7 etc?
> >  
> > Cheers,
> > Victor
> >  
>  
>  
>  
> --  
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com 
> (mailto:django-developers@googlegroups.com).
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com 
> (mailto:django-developers+unsubscr...@googlegroups.com).
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>  



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Django Admin Revamp - Any updates?

2012-04-27 Thread Daniel Sokolowski

On that note, why can't the fruits of Grappelli be used as a starting point?

-Original Message- 
From: h3

Sent: Friday, April 27, 2012 4:01 PM
To: Django developers
Subject: Re: Django Admin Revamp - Any updates?

I don't know the status of this project, but my guess is that you
shouldn't hold your breath for it.

Fortunately, there is Grappelli: 
https://github.com/sehmaschine/django-grappelli


We are currently working on making it compatible with django 1.4 (see
the "grappelli_2_4" branch)

Alternatively the branch on my fork works pretty well with 1.4:
https://github.com/h3/django-grappelli/tree/grappelli_2_4

Cheers

On Apr 26, 7:06 pm, Victor Hooi  wrote:

Hi,

Is there any news on the Django Admin rewrite front?

I remember around a year ago, there was quite a bit of talk on revamping
the Django admin UI - I think Idan Gazit was heading that, right?

Is that still on the Django roadmap? Any idea of whether it'll be in 1.5,
1.6, 1.7 etc?

Cheers,
Victor


--
You received this message because you are subscribed to the Google Groups 
"Django developers" group.

To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Daniel Sokolowski
Web Engineer
Danols Web Engineering
http://webdesign.danols.com/ 


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Django use of stdlib HTMLParser "internals"

2012-04-27 Thread Vinay Sajip
Although the Django tests all pass when run for the Python 3 port
under Python 3.2, I hit some problems when testing against recent
changes in the Python trunk (default branch, which will become 3.3).
Looking into these problems, I found that Django subclasses HTMLParser
and overrides parse_starttag, and in the overridden method uses an
HTMLParser attribute, tagfind, which has recently changed. The change
leads to the breakage.

As tagfind isn't marked as private, it seemed possible that it was OK
for Django to use it, so in the first instance I posted about it to
python-dev:

http://mail.python.org/pipermail/python-dev/2012-April/119074.html

and raised a Python issue:

http://bugs.python.org/issue14679

>From the discussion on python-dev, it seems possible that an optimal
fix might require changes both in Python and in Django. Before
creating a ticket about this for Django, I would like to get an
opinion about this from the Django committers - so can someone please
look at this python-dev thread and/or Python issue and comment?

Thanks and regards,

Vinay Sajip

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Will django escaping ever consider context of javascript and CSS?

2012-04-27 Thread Voulnet
Hello provides great protection from XSS by escaping output to
webpages, but it only does it in HTML context. XSS can be executed
when user input is inserted into javascript or CSS, which have
different context and rules than HTML, so HTML context escaping
doesn't help/protect.

Are there any remote chance of django escaping extending to other
contexts beside HTML?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: GitHub migration

2012-04-27 Thread Adrian Holovaty
On Fri, Apr 27, 2012 at 11:50 AM, Adrian Holovaty  wrote:
> We're going to do the migration to GitHub today. This means we'll no
> longer be committing code to our Subversion repository. Committers,
> please hold off on making commits until the migration is done.
>
> I expect it'll be done by late afternoon Chicago time. I'm going to do
> a dry run first to make sure all goes well, then I'll do the real
> thing.

Quick update -- I did a couple of dry runs and successfully posted the
dry run repository to my personal GitHub account. I'll post here again
when the migration is done for the official Django GitHub account.

Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Implementation of Object Permissions

2012-04-27 Thread Moritz S.

>
> Are there any hopes of speeding up or caching GenericForgeinKey queries ? 
> I found -  
> http://zerokspot.com/weblog/2008/08/13/genericforeignkeys-with-less-queries/
>  
>
Adding 'db_index=True' to the GenericForeignKey's object_id field should be 
a significant performance enhancement. It won't get as good results as the 
other solution, though, but that should prevent the database from doing 
full table scans.
But I'm not sure whether GenericRelations allow this, because no django 
object permission app does this. 

> can your solution be backward compatible?
>
My solution should be backward compatible as-is. As default, the patch 
assumes every model not having the ability to grant/check for object 
permissions. So it won't get in conflict with those (outdated) models. 
Furthermore it does not modify the existing contrib.auth models in a way 
that would require a new database table schema but only adds a new model 
called ObjectPermission.

Best regards,
Moritz

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/P1sCKnIJR38J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: GitHub migration

2012-04-27 Thread Dana Woodman
Wow this is awesome! Congrats on the change! Very excited to be able to 
contribute a Pull Request or two!

On Friday, April 27, 2012 9:50:01 AM UTC-7, Adrian Holovaty wrote:
>
> Hey guys, here's an important heads-up! 
>
> We're going to do the migration to GitHub today. This means we'll no 
> longer be committing code to our Subversion repository. Committers, 
> please hold off on making commits until the migration is done. 
>
> I expect it'll be done by late afternoon Chicago time. I'm going to do 
> a dry run first to make sure all goes well, then I'll do the real 
> thing. 
>
> Here are some random notes about the process -- 
>
> * Any fork of the existing GitHub repository at 
> https://github.com/django/django (which was a mirror of Subversion) 
> will be broken. That is, it will no longer be able to get changes from 
> upstream. Unfortunately, there's no way around this. But we'll provide 
> some instructions here on django-developers on how to change your 
> forks to use the new upstream repository in such a way that any of 
> your local changes will be preserved. Worst case, you'll just have to 
> generate a patch of your local branch's changes and apply it to a new 
> fork. 
>
> * One interesting part of this is coming up with a definitive mapping 
> between our old Subversion usernames and names/emails for Git. Brian 
> Rosner and some other folks have done a great job of doing that 
> research -- 
> https://github.com/brosner/django-git-authors/blob/master/authors.txt 
> -- and we'll be using that file to convert committer data during the 
> migration. Basically this means that if you've ever committed to 
> Django, your commits will be associated with your current GitHub 
> account -- as long as your GitHub account is associated with the same 
> email address in that authors file. Due to the way Git works, there's 
> no way of changing this data after the import is done. But we've 
> accounted for everybody except two mysterious people "dcf" and "cell," 
> both of whom were given temporary commit bits during a sprint six 
> years ago. 
>
> * If you're a Git/GitHub expert and are interested in helping, feel 
> free to join us in #django-dev on Freenode. 
>
> * Thanks in advance for bearing with us during this process. There 
> will be some pain, but it'll be worth it in the long run. I'm excited! 
>
> Adrian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/rT43KZMo9WUJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: More on Customizable Serialization

2012-04-27 Thread Russell Keith-Magee
Hi Tom,

On Friday, 27 April 2012 at 12:44 PM, Tom Christie wrote: 
> Seeing another proposal for Customizable Serialization for the GSoC this year
> prompted me to dust off the bits of work I've done along similar lines.
> I'd really like to see this get properly addressed in core and I thought it
> was about time I helped to make it happen.
> 
> I've made a fairly comprehensive start, and pushed the results of what I have
> to date as the `django-serializers` project. It's available on PyPI, and the
> source is here:
> http://github.com/tomchristie/django-serializers
> 
> There are some docs up on the GitHub site, but in brief it gives you:
> 
> * A Declarative Serializer/Field API that mirrors the Form/Field API.
> * A Serializer class which can deal with arbitrary python objects.
> * A ModelSerializer class which can handle model instances and query sets.
> * A DumpDataSerializer class which mimics the existing dumpdata behaviour.
> * Supports flat or nested ouput, and deals with recursive structures.
> * Handles FK, M2M, OneToOne, and reverse relationships, model fields, and 
> non-database attributes/properties.
> * Serialization structure decoupled from output encoding concerns.
> * Currently supports JSON, YAML, and XML.
> * Decent test suite of the above.
> 
> It's not yet at the point where it could be a proposal for core - Right now 
> the
> API isn't backwards compatible with the existing serializers, the dump
> data serializer is still missing some tweaks such as dealing with natural 
> keys,
> and there's some work to do on the XML output. It is a pretty comprehensive
> start though, and I'm really happy with the shape of the API.

Thanks for letting us know about the prior art -- I know we discussed 
serialisation at DjangoCon last year, and I'm kicking myself that I didn't 
provide more feedback at the time. Hopefully having this as a GSoC project will 
be enough to kick me into action and bring this project to completion.
> 
> Given that Piotr's GSoC proposal has now been accepted, I'm wondering what the
> right way forward is? I'd like to continue to push forward with this, but I'm
> also aware that it might be a bit of an issue if there's already an ongoing
> GSoC project along the same lines?
> 
> Having taken a good look through the GSoC proposal, it looks good, and there
> seems to be a fair bit of overlap, so hopefully he'll find what I've done
> useful, and I'm sure I'll have plenty of comments on his project as it
> progresses.
> 
> I'd consider suggesting a collaborative approach, but the rules of the GSoC
> wouldn't allow that right?

Unfortunately, the GSoC rules don't allow for collaboration - the work that is 
submitted needs to be that of the student alone. However, they do allow for 
others to contribute by providing code reviews, design feedback, and so on. 
Given that we're building a user-facing API, there's also plenty of room to 
provide assistance by testing -- i.e., hunting down obscure serialisation use 
cases, and making sure that the API we've got can cover them.

If you've got the time and enthusiasm, I'd certainly appreciate you hanging 
around in a "co-mentor"-like capacity. You've clearly spent a lot of time 
thinking about on this problem, and I'm sure your input would be extremely 
valuable. You're also in a slightly more helpful timezone for Piotr, so if he 
needs some feedback when I'm not available, it would be nice to have someone he 
can call on that is familiar with the problem and his progress.

Yours,
Russ Magee %-)

 


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



GitHub migration done!

2012-04-27 Thread Adrian Holovaty
On Fri, Apr 27, 2012 at 11:50 AM, Adrian Holovaty  wrote:
> We're going to do the migration to GitHub today. This means we'll no
> longer be committing code to our Subversion repository. Committers,
> please hold off on making commits until the migration is done.

OK, it's live!

https://github.com/django/django

A few quick points (though this definitely deserves a more in-depth writeup):

* I renamed the old (mirror) repository to
https://github.com/django/django-old. We had talked about deleting it
outright to avoid confusion, but I realized at the last minute that
doing so would have deleted all the pull requests. I didn't want to
throw all of that out, and I think we can figure out a way to use
those pull requests in the new repository.

* I only migrated trunk (now called "master"), rather than including
all of the branches. This was the result of a bunch of discussion on
IRC with Brian R., et al. The thinking is that it kept the migration a
lot cleaner/simpler, and we can always add branches later. Of course,
we'll need to create the latest release branches. Otherwise, we can
consider the SVN branches on an individual basis.

* As expected, all forks of the old repository are now broken. Can
somebody volunteer to write some documentation on how to upgrade your
old fork to use the new upstream repo (rebase? simple patch?)?

* We're going to keep the Subversion repository around indefinitely,
but it'll no longer be updated.

* We're going to keep using Trac for tickets, but pull requests on
GitHub are also welcome.

* Clearly there are lots of bits of process that need to be updated
now, from the django-updates mailing list to our "contributing"
documentation, etc. We'll take care of all of that in the coming days,
and we should all expect some degree of confusion and unsettlement in
the community. That's totally fine, and we'll get through it. :-)

* Finally, big thanks to the folks on IRC today who helped me through
the process and contributed good ideas.

I'm planning to write up a blog post on how the process went, for the
benefit of the five open-source projects still using Subversion.

Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Problems with date-based generic views when USE_TZ=True

2012-04-27 Thread Mike Yumatov
Hi Aymeric,

On Friday, April 27, 2012 at 9:21 PM, Aymeric Augustin wrote:

> > After some research, I understand that there are more problems with this 
> > views, than I thought at first. django.views.generic.dates module uses 
> > aware and naive datetime objects together, which breaks all date-based 
> > views except YearArchiveView.
> > 
> > Let's have, for example:
> > - America/Chicago as TIME_ZONE
> > - an article, published at 2012-12-31 23:00:00 in local time zone 
> > (2012-01-01 05:00:00 UTC)
> > - an url /articles/{year}/ which returns list of articles for requested year
> > - an url /articles/{year}/{month}/ which returns list of articles for 
> > requested month
> > 
> > As I understand, year and month must be in UTC in urls
> 
> They are in local time in the default time zone.
Are you sure? Because if so, YearArchiveView is broken: it uses year lookup 
(https://docs.djangoproject.com/en/1.4/ref/models/querysets/#year), which is 
performed in UTC.

Also, date_list context variable from all this views becomes useless, because 
in contains datetime.datetime objects in UTC. You can't use date_list to build 
urls in templates in this case. E.g. we have two articles: the first is 
published at 2012-04-27 in local time zone and at 2012-04-28 in UTC, and the 
second is published at 2012-04-28 in local time zone and at 2012-04-28 in UTC. 
date_list from MonthArchiveView will contain only one date - 2012-04-28 in UTC. 
Depending on what time zone we will use in template to build urls to 
DayArchiveView (local time zone or UTC), first or second article will not be 
present in output of this view.

And also, URLs to date-based views become dependent on current time zone (which 
can be changed for request on some projects), so you can't share this URLs with 
others and be sure, that they will work for others.

--
Mike Yumatov


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Problems with date-based generic views when USE_TZ=True

2012-04-27 Thread Aymeric Augustin
On 28 avr. 2012, at 07:56, Mike Yumatov wrote:

>> They are in local time in the default time zone.

> Are you sure? Because if so, YearArchiveView is broken: it uses year lookup 
> (https://docs.djangoproject.com/en/1.4/ref/models/querysets/#year), which is 
> performed in UTC.

Indeed, you're right, sorry.

In fact a very similar problem was already fixed in the admin following #17830 
[1].

I've reopened #18217 [2] for filtering ranges properly. In the current state 
the list of years, months, days in date_list may still be slightly off. That's 
in the realm of #17260 [3].

Best regards,

-- 
Aymeric.

[1] https://code.djangoproject.com/ticket/17830
[2] https://code.djangoproject.com/ticket/18217
[3] https://code.djangoproject.com/ticket/17260

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: GitHub migration done!

2012-04-27 Thread Justin Holmes
Adrian,

Thank you.  Well done.  Awesome.  :-)

On Fri, Apr 27, 2012 at 11:08 PM, Adrian Holovaty  wrote:
> On Fri, Apr 27, 2012 at 11:50 AM, Adrian Holovaty  wrote:
>> We're going to do the migration to GitHub today. This means we'll no
>> longer be committing code to our Subversion repository. Committers,
>> please hold off on making commits until the migration is done.
>
> OK, it's live!
>
> https://github.com/django/django
>
> A few quick points (though this definitely deserves a more in-depth writeup):
>
> * I renamed the old (mirror) repository to
> https://github.com/django/django-old. We had talked about deleting it
> outright to avoid confusion, but I realized at the last minute that
> doing so would have deleted all the pull requests. I didn't want to
> throw all of that out, and I think we can figure out a way to use
> those pull requests in the new repository.
>
> * I only migrated trunk (now called "master"), rather than including
> all of the branches. This was the result of a bunch of discussion on
> IRC with Brian R., et al. The thinking is that it kept the migration a
> lot cleaner/simpler, and we can always add branches later. Of course,
> we'll need to create the latest release branches. Otherwise, we can
> consider the SVN branches on an individual basis.
>
> * As expected, all forks of the old repository are now broken. Can
> somebody volunteer to write some documentation on how to upgrade your
> old fork to use the new upstream repo (rebase? simple patch?)?
>
> * We're going to keep the Subversion repository around indefinitely,
> but it'll no longer be updated.
>
> * We're going to keep using Trac for tickets, but pull requests on
> GitHub are also welcome.
>
> * Clearly there are lots of bits of process that need to be updated
> now, from the django-updates mailing list to our "contributing"
> documentation, etc. We'll take care of all of that in the coming days,
> and we should all expect some degree of confusion and unsettlement in
> the community. That's totally fine, and we'll get through it. :-)
>
> * Finally, big thanks to the folks on IRC today who helped me through
> the process and contributed good ideas.
>
> I'm planning to write up a blog post on how the process went, for the
> benefit of the five open-source projects still using Subversion.
>
> Adrian
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>



-- 
Justin Holmes

Head Instructor, SlashRoot Collective
SlashRoot: Coffee House and Tech Dojo
60 Main Street
New Paltz, NY 12561
845.633.8330

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Django use of stdlib HTMLParser "internals"

2012-04-27 Thread Aymeric Augustin
On 28 avr. 2012, at 00:49, Vinay Sajip wrote:
> From the discussion on python-dev, it seems possible that an optimal
> fix might require changes both in Python and in Django. Before
> creating a ticket about this for Django, I would like to get an
> opinion about this from the Django committers - so can someone please
> look at this python-dev thread and/or Python issue and comment?

Hi Vinay,

As far as I can tell, Django's patching HTMLParser [1] to work around a bug [2] 
that was fixed in Python 2.7 and 3.2. If that comment is accurate, we could 
apply the patch only when running on Python < 2.7.

So it seems possible to conditionally fix Django, whatever solution the Python 
devs choose.

Best regards,

-- 
Aymeric.


[1] 
https://code.djangoproject.com/browser/django/trunk/django/utils/html_parser.py#L5
[2] http://bugs.python.org/issue670664

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.