Model icons

[Marty]

Hi all,

Recently, it's trend to use icons or emoji before menu items and I like 
this idea because  IMHO people orient better and more quickly when they see 
picture.

What about to add this feature to native django? I thought the easiest way 
would be to add new Meta option to Model. The default Meta icon would be 
empty string so everything would work like now. If I want to make menu more 
readable, I just add emoji (🔨) or html (Awesome font - ** , Bootstrap icon - **, etc.) 
to Meta icon.

Code example:

*Model:*
class Hammer(models.Model):
...

Meta:
icon = ''

*app_list.html template:*
...
{{model.icon}} {{model.name}}
...

Final result:
[image: admin navbar.png]

Maybe own icon could have even the parent app (AppConfig). And the model 
icon could be in breadcrumbs.

What do you think about this idea? 🙂

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/09b292b3-25e1-42b6-b3fa-b92948e35b29n%40googlegroups.com.

Re: Model icons

[Marty]

Yeah, I've tried to create templatetag like this:

@register.simple_tag
def get_module_icon(model, app):
if app == 'auth':
if model == 'Groups':
return ''
elif model == 'Users':
return ''
elif app == 'core':
elif model == 'Generations':
return ''
elif model == 'Product models':
return ''
elif app == 'bto':
if model == 'Components':
return ''
elif model == 'Layouts':
return ''
elif model == 'SKU configurations':
return ''
elif model == 'XML component definitions':
return ''
return ''

I don't like this way so much and I thought it would be easier even for 
other potentional developers who would like to use icons so it seemed 
putting it to Meta would be better. However I get mixing HTML to model is 
not good. 🙂
What would be the most clean way to use icons like that?

On Saturday, February 18, 2023 at 8:23:28 PM UTC+1 Arthur Pemberton wrote:

> I too find the idea of hard coded HTML in a Python file to be inelegant, 
> for what it's worth.
>
> Arthur Pemberton
>
> On Sat, 18 Feb 2023 at 08:12, 'Adam Johnson' via Django developers 
> (Contributions to Django itself)  wrote:
>
>> Putting HTML for the admin in model definitions is a bit too coupled.
>>
>> I think you may be able to add icons already by overriding the admin 
>> template - can you try playing around with that?
>>
>> On Mon, Feb 6, 2023 at 12:07 AM Yeonggwang Yang  
>> wrote:
>>
>>> that sounds good with me
>>>
>>> 2023년 2월 5일 일요일 오전 9시 36분 17초 UTC+9에 Marty님이 작성:
>>>
 Hi all,

 Recently, it's trend to use icons or emoji before menu items and I like 
 this idea because  IMHO people orient better and more quickly when they 
 see 
 picture.

 What about to add this feature to native django? I thought the easiest 
 way would be to add new Meta option to Model. The default Meta icon would 
 be empty string so everything would work like now. If I want to make menu 
 more readable, I just add emoji (🔨) or html (Awesome font - *>>> class="fa-solid fa-hammer">* , Bootstrap icon - **, etc.) to Meta icon.

 Code example:

 *Model:*
 class Hammer(models.Model):
 ...

 Meta:
 icon = ''

 *app_list.html template:*
 ...
 {{model.icon}} {{model.name}}
 ...

 Final result:
 [image: admin navbar.png]

 Maybe own icon could have even the parent app (AppConfig). And the 
 model icon could be in breadcrumbs.

 What do you think about this idea? 🙂

 -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/6c859cc5-ea41-4695-9fcf-5cd435310364n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CAMyDDM1L4rmQ3qomxOqD%2BmugXcsmx7NrOPAh8a53wR9G5q%3D3Gw%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1f461742-7fe6-4798-97e9-a8133bd2364bn%40googlegroups.com.

Re: Model icons

[Marty]

I really like the idea to add just a simple 'icon' attribute. That's quite 
elegant solution. Thanks, Mark! 🙂

On Monday, February 20, 2023 at 2:33:43 PM UTC+1 Jacob Rief wrote:

> I agree with Adam Johnson that adding HTML to the model class is a bit too 
> coupled.
> But I like the idea of Mark Niehues to add them to the ModelAdmin class, 
> although I would allow a HTML/SVG snippet rather than a CSS class.
>
>- How would we then handle 3rd party apps providing their own SVG file 
>to be used as icon?
>- Shouldn't that icon definition be part of the ModelAdmin's Media 
>subclass?  
>
> – Jacob
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/45c143cc-2716-430a-93cf-7c2a938640ccn%40googlegroups.com.

Filtering chosen options

[Marty]

Hi,

Is there a reason why there is a filter only at available options but not 
at chosen options in this multiple select widget 
?

I would appreciate filtering even chosen options. 🙂

Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1ad9ced0-a7f9-4d4b-a29b-8f246f76cc36n%40googlegroups.com.

Re: Filtering chosen options

[Marty]

Great! Thanks for info. 🙂

On Friday, March 24, 2023 at 8:32:27 AM UTC+1 Carlton Gibson wrote:

> This was added in 
> https://github.com/django/django/commit/be63c78760924e1335603c36babd0ad6cfaea3c4
>  
> and will be part of Django 4.2, which is currently at release candidate 
> stage, for final release around 2023-04-03. 
>
> On Friday, 24 March 2023 at 03:20:41 UTC+1 Marty wrote:
>
>> Hi,
>>
>> Is there a reason why there is a filter only at available options but not 
>> at chosen options in this multiple select widget 
>> <https://i.stack.imgur.com/VtpXJ.png>?
>>
>> I would appreciate filtering even chosen options. 🙂
>>
>> Thanks.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/797423f0-757b-4a13-9428-5b2a74e24a46n%40googlegroups.com.

Password reset link is not displayed by default

[Marty]

Hi,

this is native django 4.2 code:

*admin/login.html*

...

{% csrf_token %}

  ...

  {% url '*admin_password_reset*' as password_reset_url %}

  {% if password_reset_url %}

  

{% translate 'Forgotten your 
password or username?' %}

  

  {% endif %}

  ...



...

 

*auth/urls.py:*

urlpatterns = [

...

path("password_reset/", views.PasswordResetView.as_view 
(), name="*password_reset*"),

...

]

In template is expected *admin_password_reset *but auth module generates 
*password_reset*.
This is my project urls.py:
urlpatterns = []

if settings.DEBUG:
urlpatterns += [
path('__debug__/', include(debug_toolbar.urls)),
*static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT),
]

urlpatterns += [
# API
# path('api/ ', include('apps.management.api_urls')),

# extra pages
path('changelog/', management_views.changelog, name='changelog'),

# (due to password reset)
path('accounts/', include('django.contrib.auth.urls')),
*path('accounts/password_reset/', PasswordResetView.as_view(), 
name='admin_password_reset'),*

# modules
path('authentication/', include('apps.authentication.urls')),
path('management/', include('apps.management.urls')),

# native admin URLs
path('', admin.site.urls),  # must be last if admin URL is empty!
]

Notice that I had to add the extra
 
*path('accounts/password_reset/', PasswordResetView.as_view(), 
name='admin_password_reset'),*to display the reset password link. Is this 
on purpose in django or is this bug that should be fixed?


Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/24afc193-c22a-4be8-9f3b-14315b2b7a8bn%40googlegroups.com.

Re: Password reset link is not displayed by default

[Marty]

Hi,

ah, I see. Okay, thanks for info and link.

Marty

On Monday, July 24, 2023 at 2:00:44 PM UTC+2 Adam Johnson wrote:

> Hi!
>
> You've found the wrong mailing list for this post. This mailing list is 
> for discussing the development of Django itself, not for support using 
> Django. This means the discussions of bugs and features in Django itself, 
> rather than in your code using it. People on this list are unlikely to 
> answer your support query with their limited time and energy.
>
> For support, please follow the "Getting Help" page: 
> https://docs.djangoproject.com/en/stable/faq/help/ . This will help you 
> find people who are willing to support you, and to ask your question in a 
> way that makes it easy for them to answer.
>
> That said, in response to your question: Django's default admin setup does 
> not come with a password reset view. It only has a "change password" view 
> for logged in users. If you want to support not-logged-in users to reset 
> their passwords, you need to add a URL as you have done.
>
> Thanks,
>
> Adam
>
> On Sun, Jul 23, 2023 at 2:40 PM Marty  wrote:
>
>> Hi,
>>
>> this is native django 4.2 code:
>>
>> *admin/login.html*
>>
>> ...
>>
>> {% csrf_token 
>> %}
>>
>>   ...
>>
>>   {% url '*admin_password_reset*' as password_reset_url %}
>>
>>   {% if password_reset_url %}
>>
>>   
>>
>> {% translate 'Forgotten your 
>> password or username?' %}
>>
>>   
>>
>>   {% endif %}
>>
>>   ...
>>
>> 
>>
>> ...
>>
>>  
>>
>> *auth/urls.py:*
>>
>> urlpatterns = [
>>
>> ...
>>
>> path("password_reset/", views.PasswordResetView.as_view 
>> <http://views.passwordresetview.as_view/>(), name="*password_reset*"),
>>
>> ...
>>
>> ]
>>
>> In template is expected *admin_password_reset *but auth module generates 
>> *password_reset*.
>> This is my project urls.py:
>> urlpatterns = []
>>
>> if settings.DEBUG:
>> urlpatterns += [
>> path('__debug__/', include(debug_toolbar.urls)),
>> *static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT),
>> ]
>>
>> urlpatterns += [
>> # API
>> # path('api/ ', include('apps.management.api_urls')),
>>
>> # extra pages
>> path('changelog/', management_views.changelog, name='changelog'),
>>
>> # (due to password reset)
>> path('accounts/', include('django.contrib.auth.urls')),
>> *path('accounts/password_reset/', PasswordResetView.as_view(), 
>> name='admin_password_reset'),*
>>
>> # modules
>> path('authentication/', include('apps.authentication.urls')),
>> path('management/', include('apps.management.urls')),
>>
>> # native admin URLs
>> path('', admin.site.urls),  # must be last if admin URL is empty!
>> ]
>>
>> Notice that I had to add the extra
>>  
>> *path('accounts/password_reset/', PasswordResetView.as_view(), 
>> name='admin_password_reset'),*to display the reset password link. Is 
>> this on purpose in django or is this bug that should be fixed?
>>
>>
>> Thanks.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/24afc193-c22a-4be8-9f3b-14315b2b7a8bn%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/24afc193-c22a-4be8-9f3b-14315b2b7a8bn%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dfca9835-6a6e-45a2-a349-6aa96d6a2b8en%40googlegroups.com.

Re: FileStorage and folders

[Marty Alchin]

Alexey wrote:
> I wonder, why is there no folder creation functions in Storage and
> FileStorage classes?

Because none of Django's features really required it.

> I ask this not because I don't know how to create it mysef. I'm just
> curios, why there are only one function for working with folders --
> listdir()?

For what it's worth, that actually wasn't intended to be there. I put 
that in when I thought that the initial file storage update would also 
support FilePathField, since that would need to be able to list a 
directory's contents. It accidentally made it into the final patch, and 
people immediately started using it, so it stayed.

> As for me,  function mkdir() should be there.

In the long run, I agree. I'm not sure it's much of a priority at the 
moment, but it's probably worth discussing for 1.2. Is there already a 
ticket out there somewhere for discussion? If not, would you mind 
creating one? Even if it doesn't make it in, it's easier to keep track 
of if there's a ticket for the request.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding signing (and signed cookies) to Django core

[Marty Alchin]

I'm obviously going to weigh in here, having authored the signed
cookies app you mentioned below, but be aware I'm not a cryptographer,
nor do I have any personal use for signed cookies at all. I can
appreciate their value, but I'm not even using my own app in anything,
so if there are problems with it, I haven't experienced them
first-hand. I won't comment on everything, because I certainly trust
your thoughts on the matter. (Your comment on the Django book is what
prompted me to write the app in the first place!) I do have a couple
thoughts, though. So, anything I don't comment on, I agree with.

On Thu, Sep 24, 2009 at 1:18 PM, Simon Willison  wrote:
> It's also something that's hard to do correctly. At the sprints Armin
> pointed out that I should be using hmac, not straight sha1, for
> generating signatures (something Django itself gets wrong in the few
> places that implement signing already). Having a cryptographer-
> approved implementation will save a lot of people from making the same
> mistakes.

I have no idea how hmac differs from straight sha1, but this was
raised in a django-signedcookies issue as well, and i've since
integrated it. I'm with you on this; if somebody who knows better
recommends something, I'm inclined to listen.

> I think signed cookies should either be a separate API from
> response.set_cookie or should be provided as an additional argument to
> that method. I'm not a fan of signing using middleware (as seen in
> http://code.google.com/p/django-signedcookies/ ) since that approach
> signs everything - some cookies, such as those used by Google
> Analytics, need to remain unsigned.

I admit, I hadn't considered third-party cookies when I put it
together as a decorator. Client-side cookie access is likely
problematic as well, but that'll always be questionable anyway. You
can't validate a cookie in the client without divulging your secret
key and you can't just ignore the signature, because that defeats the
whole purpose. My app also provides a decorator, which might help in
some rare situations, but most of the time things like analytics
cookies will be in a base template and will always be included in
whatever view happens to have the decorator. I'm very surprised that
in all this time, nobody submitted a bug about the analytics problem.

> So the API could either be:
>
>    response.set_signed_cookie(key, value)
>
> Or...
>
>    response.set_cookie(key, value, signed=True)
>
> (I prefer the latter option)

I prefer the latter as well, for an added reason. I'd personally like
to invest some time in seeing if there are any other interesting
pitfalls in set_cookie() based on it deferring to Python's
SimpleCookie implementation. When writing Pro Django, I realized that
SimpleCookie expects everything to be strings, so #6657 came up with
secure=False resulting in a secure cookie after all. I don't know if
there are other such issues, but it might be worth looking at in
detail if we already have to add in signed cookie support.

And before anyone asks, no I don't think tying the signing behavior
into secure=True would be a good idea. Secure is meant to tell the
browser it should only send the cookie back over a secure channel,
such as HTTPS. While people who need secure cookies may also want
signed cookies, they're two separate ideas that I don't think would do
well mixed together. Of course, that leaves us with a potential
response.set_cookie(key, value, secure=True, signed=True), but I think
it's worth it to be explicit.

But I do have one other concern with either of these APIs that is at
least worth discussing: the disparity between setting a signed cookie
and retrieving one.

response.set_cookie(key, value, signed=True)
value = signed.unsign(request.COOKIES[key])

Mainly, unsigning a cookie requires importing and directly using the
signing module anyway, so the benefit of having set_cookie() handle it
transparently is fairly weak (unless, of course, I'm missing something
obvious). I'd rather just see the signing code made available and
well-documented, so that at least the change in code when adding
signed cookies is equivalent on both sides.

Another option would be to have request.COOKIES be a custom
dictionary, with an extra .get_unsigned(key) method that would work
like .get(), but validates the signature along the way. That behavior
can't be added straight to __getitem__() though, because that has no
way of knowing whether the cookie was signed to begin with.

These are the types of issues that led me to just implement it as a
middleware, so the API could be as dead simple as possible. With these
new issues in mind, I don't think dead simplicity is really an option,
so I'd rather fall back to being explicit.

 signed.unsign('hello.9asVJn9dfv6qLJ_BYObzF7mmH8c')
> 'hello'
 signed.unsign('hello.badsignature')
> Traceback (most recent call last):
> ...
> BadSignature: Signature failed: badsignature
>
> BadSignature is a subclass of ValueError, meaning lazy develop

Re: Adding signing (and signed cookies) to Django core

[Marty Alchin]

On Thu, Sep 24, 2009 at 2:54 PM, Simon Willison  wrote:
> Hmm... I hadn't considered that. I was thinking that the unsigning
> could be transparent, so by the time you access request.COOKIES['key']
> the value had already been unsigned (and if the signature failed the
> cookie key just wouldn't be set at all, as if the cookie never
> existed). But as you point out, this doesn't work because you can't
> tell if the cookie was signed or not in the first place.

The behavior you mention here is exactly what django-signedcookies
does, but it can only do so because it can blindly assume that all
cookies are signed, which as you pointed out, causes other problems.

> We could fix this with a naming convention of some sort:
>
> response.set_cookie('key', 'value', sign=True)
> - results in a Set-Cookie: key__Xsigned=value header

That seems pretty ugly on the surface, but it does confine the
ugliness to somewhere most people won't bother to look. One potential
problem is if someone wants to use __Xsigned in the name of an
unsigned cookie, but a namespace clash like that should be extremely
rare in practice.

Also, does the name of a cookie factor into the cookie length limits?
My reading of RFC 2109 says yes, but it'd be worth verifying, since it
would cut down on the usable value space. With your compressed base64
stuff, that's not as big of a problem, but still something to look
into.

> request.unsign_cookie('key') might be an option, as at least that
> reflects the set_cookie / sign / unsign API a bit. Not ideal by a long
> shot though.

>> try:
>>     value = signed.unsign(signed_value).decode('utf-8')
>> except ValueError:
>>     # Whoops! UnicodeDecodeError winds up here as well!
>
> That's a great argument against subclassing ValueError - I hadn't
> considered the unicode case. You're right, if anything it should
> subclass SuspiciousOperation instead.

I don't know if it's completely anti-ValueError, because a ValueError
subclass does still make some sense semantically, and since you can
catch more than one exception type in a try block, it's perfectly
functional. It's just that when lazy people blindly catch ValueError
without checking for something more specific as well, things can
break.

So it really just comes down to whether we expect people to be
thorough or lazy. Hrm. Yeah, I guess that answers it. :)

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding signing (and signed cookies) to Django core

[Marty Alchin]

> Also, does the name of a cookie factor into the cookie length limits?
> My reading of RFC 2109 says yes, but it'd be worth verifying, since it
> would cut down on the usable value space. With your compressed base64
> stuff, that's not as big of a problem, but still something to look
> into.

Also, just to throw this out there for the sake of compleness: could
the signature be stored under a separate name, rather than being
bundled with the original cookie itself?

Set-Cookie: key=value
Set-Cookie: key__Xsignature=signature_string

It seems like this could address a couple issues at once.

* There's a clear distinction between signed and unsigned cookies, so
request.COOKIES can be populated with only valid cookies

* The key/value pair remains unchanged, so things like Google
Analytics can happily ignore the signature if it was applied
unnecessarily (middleware is back on the table!)

Since there may be an upper limit on the number of allowed cookies,
though, doubling the number of cookies could present some very real
problems. RFC 2109 recommends allowing at least 20 cookies per domain
name, and it looks like at least Microsoft takes that to be a
maximum,[1] so it could present very real problems (middleware is back
off the table!).

I'm not sure how many cookies people use on a regular basis, and this
would only be for explicitly signed cookies, so maybe it'd be okay,
but it's flirting dangerously close to being completely unworkable.
Worse yet, it doesn't look like there's any predictable way to know
which cookies would get lost in the event of having too many, so this
may end up causing some very weird application errors if things go
wrong.

At least now it's been recorded for future reference. (Hello, future
me, looking up information on why we did things the way we did! Do we
have flying cars yet?)

-Gul

[1] http://support.microsoft.com/kb/306070

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding signing (and signed cookies) to Django core

[Marty Alchin]

On Thu, Sep 24, 2009 at 3:22 PM, Benjamin Slavin
 wrote:
>> response.set_cookie('key', 'value', sign=True)
>> - results in a Set-Cookie: key__Xsigned=value header
>
> Unfortunately, this approach won't work.
>
> A malicious client can just send "key" rather than "key__Xsigned" and
> you'll never know that the cookie hasn't gone through validation.
> This also means that you can't look for cookie values that match a
> specific format (ex: body.signature) because a malicious client could
> just drop the signature portion.
>
> As always, we can't trust the client. :-(

And you've just added another reason my followup email is invalid,
though I sent that before reading your response. (Take that, future
me!)

> 3) COOKIES as an intelligent object -- We can overload .get so we're
> doing something like request.COOKIES.get('foo', signed=True) -- I
> think this has the best chance at an interface that keeps a consistent
> feel. It's completely backward compatible, though it breaks the
> traditional expectation of what you can do via the `get` method on a
> dictionary.

I was wondering about this option as well, after I mentioned adding a
request.COOKIES.get_unsigned() method. I actually like this idea a
lot, personally. As you mention, it's backward compatible, and I'm not
sure it completely breaks the traditional expectation. After all,
aren't subclasses expected to customize the behavior of their parents?
It doesn't change any existing behavior, but rather just adds
something extra.

The one downside to using get() directly, as opposed to an altogether
new method, is that get() doesn't raise a KeyError when a value
doesn't exist. That means if anyone's wrapping request.COOKIES[key] in
a try block and catching KeyError, changing to the new code is more
than just a one-liner. I'm personally okay with this, but it's
definitely worth noting.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding signing (and signed cookies) to Django core

[Marty Alchin]

On Fri, Sep 25, 2009 at 7:02 AM, Luke Plant  wrote:
> Suppose one part of an app signs an e-mail address for the purpose of
> an account confirmation link sent in an e-mail.  The user won't be
> able to forge the link unless they know HMAC(SECRET_KEY, email).
>
> However, suppose another part of the website allows a user to set
> their e-mail address (merely for convenience), and stores it in a
> signed cookie.  That means an attacker can now easily get hold of
> HMAC(SECRET_KEY, email), and forge the link.

This was something I was concerned with when I put together my app, so
I just added the name of the cookie to the signature as well, rather
than requiring some other explicit prefix. Since the two parts of the
application would need to use different names anyway to avoid other
problems, I figured the cookie name alone would be sufficient. If we
end up with something like response.set_signed_cookie() or
response.set_cookie(..., signed=True), the cookie name would be
available to the signing code internally, without any need to add in
some other key.

Of course, it'd still be worth documenting for the case of using the
signing stuff outside of cookies, because similar problems could creep
in elsewhere. I just think if we already have a name available, we
should be able to use it without any trouble at all. I wish there was
a way to sign the expiration as well, so people couldn't artificially
extend the life of the cookie, but since that doesn't come back in the
request, there'd be no way to validate it.

> So I propose:
>
>  - we add unique prefixes to the SECRET_KEY for every different
>   place it is used.  So for the e-mail confirmation link, we use
>   HMAC("email-confirmation" + SECRET_KEY, message)

I think this is good for everywhere the raw signing API is accessed,
perhaps to the point where that API can't even be used without a key
(a namespace, really - honking great idea!). Helpers on top of that
API could do without asking for that separately, if they can retrieve
a reasonable key from other forms of input, such as a cookie name or a
query-string argument name.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding signing (and signed cookies) to Django core

[Marty Alchin]

On Fri, Sep 25, 2009 at 9:26 AM, Simon Willison  wrote:
> The API would look something like this:
>
 s = signed.sign('value', timestamp=True)
 v = signed.unsign(s, expire_after=24 * 60 * 60)
>
> A SignatureExpired exception would be raised if the signature was
> older than the expire_after argument (SignatureExpired would subclass
> BadSignature)

Very true. I had considered baking the timestamp right into the value
portion as well, but I was concerned about the extra space it would
take. It looks like it would only be 8 characters max, if encoded as
base64, which could be shortened to 6 if we strip out the == at the
end. I think I remember reading somewhere that those can be reattached
programmatically on the other end if necessary. All in all, the space
usage isn't that bad, and since it would be an optional component
anyway, it wouldn't add any overhead to the common case.

Would expire_after on the unsign just automatically imply
timestamp=True? There's been a lot of concern raised about parity in
the API, and it reads a little weird with the two different arguments.
I'm not sure it's a problem, but it's just a little funny.

Even though Ben rightly pointed out that we can't autodetect whether a
value is signed or not, I wonder if we could at least autodetect the
presence of a timestamp within a signature, once we already know that
the value is supposed to be signed. Essentially the unsigning code
could look for two different types of signatures, one with a timestamp
and one without. The timestamp would then be the actual expiration
time, rather than the time it was signed, so the API can look like
this instead (with a key added per prior discussion).

>>> s = signed.sign('key', 'value')
>>> v = signed.unsign('key', s)

>>> s = signed.sign('key', 'value', expire_after=24 * 60 * 60)
>>> v = signed.unsign('key', s)

This does make some assumptions about the format of the signed value,
but once we explicitly establish that the value is signed (by way of
passing it into the unsigning code), it's safe to make certain
assumptions about the format of the value. Or am I missing something
obvious here?

In the cookie case, it might be appropriate to use the combination of
an explicit expiration and signed=True to imply that an expiration
timestamp should be added to the cookie as well. I think that would be
the desired behavior, but I'm not quite sure.

Aside from some of the specifics of the cookie implementation, I think
we're getting close to an API here. I just hope we can get some input
from a cryptographer to make sure we get a solid implementation before
we go too far with this.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding signing (and signed cookies) to Django core

[Marty Alchin]

On Fri, Sep 25, 2009 at 10:42 AM, Simon Willison
 wrote:
> On Sep 25, 3:39 pm, Simon Willison  wrote:
>> While that makes sense for caching, I couldn't say if it makes sense
>> for signatures or not - when we sign something, will we always know
>> the point at which we want that signature to expire? I don't know.
>
> Here's a good argument for signing things with the creation-timestamp
> rather than the expiration-timestamp - it leaves the door open for a
> mechanism whereby historic SECRET_KEYs are stored. When we see a
> signed string, we can use its timestamp to decide which of our
> historic keys should be used to validate it.

But that only works for signatures that do in fact use a timestamp. If
the API makes timestamps optional, then there's still the question of
what to do for signatures that aren't stamped anyawy. In those cases,
I assumed the protocol would be to simply try to most recent
SECRET_KEY and work backward if the signature failed. Once all current
and deprecated keys have been tried (which should only be a total of
two, I would think), it would raise BadSignature at that point.

If we already have that behavior for non-timestamped cookies (and
correct me if I'm wrong on that point), I don't know that it's much of
an argument in favor of which timestamp to use. As long as there's not
a huge list of valid SECRET_KEYs to choose from, the overhead of
trying them each individually should be negligible, so I'm not sure
how much of an issue it would be.

I do agree though that you do have a point about there being possible
reasons for expiring the key at a different point after the fact, but
I'd argue that in those cases, you'd want to set an explicit
expiration time. In your example, you provided expire_after=24 * 60 *
60, but that wouldn't let you expire a cookie because of an event that
happened 2 hours ago. I would think you'd want to pass in a specific
time that cookies should be considered expired instead.

Of course, that then goes back to stamping cookies with their creation
time anyway, because otherwise you couldn't really do it right. If I
say "expire as of time X" I want to expire all cookies issued prior to
that point in time, while leaving any issued after that point intact.
The only way to make that decision is to know the time it was issued,
rather than when it was originally expected to expire.

I think we're getting a bit ahead of ourselves, though. There's
nothing stopping an application timestamping its own signatures and
validating them however they like, so we're really just discussing a
reasonable default here. Heck, since the timestamp behavior is driven
entirely within an already-signed value, we don't need to provide any
behavior at all if it's not a common requirement. I think it's a good
idea, but I'd like to hear from other people before I really stand
behind including it.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Best place for code that processes stuff from settings.py once (for logging implementation)

[Marty Alchin]

On Mon, Oct 12, 2009 at 12:09 PM, Jeremy Dunck  wrote:
> Also, I bet Marty knows this area well from his book work.

Actually, I didn't research much on the initialization process as a
whole, if there indeed is such a beast. I started with what happens
when Python actually encounters a model definition, which occurs after
settings and INSTALLED_APPS have been taken into account, which is
pretty much where the DevModelCreation article starts as well. Like
most people, I've generally deferred to James on the startup issue
here.

http://www.b-list.org/weblog/2007/nov/05/server-startup/

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Call for feedback: django.utils.signed and signed cookies

[Marty Alchin]

On Mon, Dec 21, 2009 at 8:00 AM, Luke Plant  wrote:
> Rather than use 'settings.SECRET_KEY' as the default HMAC key,
> shouldn't we add a prefix so that any usage of SECRET_KEY can't be
> (potentially) used to attack other usages?  We discussed this a while
> back.  The new messages system uses:
>
>  'django.contrib.messages' + settings.SECRET_KEY
>
> for its HMAC key, which seems like a good convention to adopt. I don't
> know how strictly necessary it is, but I don't think it can hurt.

This was my concern as well, after the previous talks. The behavior
seems to be there, in the form of the `salt` argument, but I doubt
many people will use it. It might not be strictly necessary, but I
agree that it's probably a good practice that I would personally like
to see mandated by making that argument required, rather than
optional. It may seem like overkill, but if we're offering a way for
people to secure their applications, I think we should offer the most
secure option as the default, rather than simply the easiest. Of
course, the real-world importance of salting the hash would be best
addressed in the cryptographic review, but I agree that it's something
that should be noted prominently here, to make sure it gets a
once-over in said review.

> I also think the "Cryptographic signing" documentation should clearly
> note that developers should be very careful not to expose
> functionality that would allow users to retrieve signatures for
> arbitrary strings.  If they do, it will allow any other system which
> uses the same key for signing to be subverted.

This is particularly important in the case of cookies, because it's
all to easy to simply change the name of a cookie and have it send an
authenticated value. That's why my original implementation also used
the cookie's name as an additional salt in the value. The effect is
essentially the same as using the `salt` argument explicitly, and I
think it's an acceptable shortcut for the cookie case. We're already
asking users to supply the key, which is basically a namespace anyway,
so why shouldn't we just automatically use that as salt for the hash?
That way, if `salt` does become required for the generic signing case,
it can still be left out from the cookie case, since the key can play
double-duty. If there's a concern with collision with other keys, we
could always prefix it with 'django_cookie_' or something before
applying it, but I don't think that'd be a necessary step.

-Marty

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Loading Backends

[Marty Alchin]

Looking over Simon's patch for signed cookies, I noticed that yet
another feature needs to write its own "load a backend from a setting"
function. A quick search of the codebase shows at least four other
instances of this behavior (shown below), and it seems it's only going
to get more common going forward. Is there a way we can factor some of
this out into a utility? As it stands now, the author of each feature
needs to figure out which one to copy from, which means figure out how
they're different, even though many of those differences are just
stylistic.

django.contrib.auth.load_backend()
django.core.files.storage.get_storage_class()
django.template.loader.find_template_loader()
django.db.load_backend()

Database loaders are a bit of an oddball, since they load an entire
module's worth of classes based on the setting, but there's repetition
even there. Unfortunately, I can't dedicate much of my own time at the
moment to look into this, but I think it'd be worth doing before we
start adding even more features with configurable backends in the
future.

-Marty

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Loading Backends

[Marty Alchin]

On Mon, Dec 21, 2009 at 10:46 AM, Simon Willison
 wrote:
> On Dec 21, 2:22 pm, Marty Alchin  wrote:
>> Looking over Simon's patch for signed cookies, I noticed that yet
>> another feature needs to write its own "load a backend from a setting"
>> function.
>
> Yup - and as I copied and pasted it from somewhere else I thought
> exactly the same thing. Is this kind of refactoring something we can
> do after the 1.2 feature freeze? If so I think it would be worth
> cleaning up at least some of this stuff.

Absolutely, this is completely separate from anything that should
affect the feature freeze. As Russ pointed out, there are a lot of
different of this backend loading stuff, so it'll be tricky to figure
out just how much can/should be factored out, how to deal with some of
the variations and how to recommend people do it going forward. In
fact, it's not even something that needs to happen by the 1.2 release
at all, if more time is necessary to get it right. It's less than
ideal at the moment, but I'd say practicality beats purity in this
case, to make sure we can get the goods out the door.

-Marty

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Porting Django to Python 3

[Marty Alchin]

2010/1/14 Łukasz Rekucki :
> It is possible to write 3.x code that is backwards-compatible with
> python 2.6+. There are some rough edges like, names of stdlib modules,
> instance checks for strings and some introspection details. In my
> opinion, it's pretty much the same as supporting old 2.x pythons.

In many cases, this is true, but there are other scenarios (certain
forms of exception handling, for example) where there is no syntax
that's valid in both versions. That's syntax, not just libraries and
functions. There's no way to even get a file to parse in both Python 2
and Python 3 in these situations. There are certainly places in Django
that will run into these, so we really can't have a single codebase
that's completely compatible with both branches.

-Gul
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Porting Django to Python 3

[Marty Alchin]

On Thu, Jan 14, 2010 at 2:32 PM, Karen Tracey  wrote:
> Martin's approach was single codebase where the 3.x version for execution is
> generated by 2to3, not single source for execution across 2.x and 3.x.  Thus
> I'm wondering if this difference is accounted for by 2to3?  If yes, then it
> is not necessarily a problem that would stand in the way of maintaining
> single Django source and supporting Python 2.x and 3.x simultaneously.

Yes, I was a bit less clear than I should've been. I was responding on
an assumption that the author was expecting a single codebase to work
with 2 and 3 without going through 2to3 in between. To my knowledge,
2to3 does handle all the syntactic issues between the two, but I just
wanted to make it clear that it's definitely not "pretty much the same
as supporting old 2.x pythons."

-Gul
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

DurationField update

[Marty Alchin]

I know I've been talking a lot about my DurationField submission
lately, and I finally have some better code to back it up. I just
submitted a new patch for #2443 that addresses the areas Malcolm
recommended to me recently, including newforms-admin.

It's still missing tests and documentation, but I'd appreciate any
feedback or additions.

It's also a good example of the ideas I've been having for type
coercion, and it works in concert with my recent patch for #3982. I'd
especially like feedback on this aspect of it, to see if this is a
reasonable way to approach issues like this.

-Marty


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Field.contribute_to_class and contrib fields

[Marty Alchin]

I like that approach to this rather tricky situation. With all my
recent work on DurationField, I took a chance at trying to make your
code into something generic enough to work for both needs, and
possibly others. I'm imagining a FileField that can act as a file-type
object, lazily opening the file the first time something calls
obj.file_field.read() or obj.file_field.write(). Just an example of
what's possible.

Anyway, I noticed that I'm having some trouble with it. First, it
looks like it's not truly lazy, since the object is instantiated
during __set__, rather than __get__. Since Model.__init__ uses
setattr(self, field.attname, val) to assign values to the instance,
the descriptor's __set__ method gets triggered as soon as the data is
retrieved from the database. At least, that's one it's doing for me.

The other issue I'm having is with that of GEOSGeometrys srid
attribute. I don't have GeoDjango installed, but I can see what you're
trying to do with srid there, and that makes it difficult to make into
a generic descriptor. It's easy to pass in an initialization function,
but the trick is knowing whether an object has already been
initialized within the context of that field. Currently I'm having to
call it every time __get__ is called, which can't be a good thing.
That's not a problem with the patch as is, of course, just a
difficulty I'm having in getting it to work with other Field
subclasses.

-Marty

On May 17, 1:41 am, Robert Coup <[EMAIL PROTECTED]>
wrote:
> Robert Coup wrote:
> > Jeremy Dunck wrote:
>
> >> In the gis work, we'd like to contribute a fair bit of additional
> >> functionality to GIS fields and models with GIS fields.
>
> >> So far, field contributions seem to be largely done by adding
> >> attributes to the model, such as FileField's get_%s_url.
>
> >> I think this approach may just be legacy from the old codegen days,
> >> and I'm not sure this is a good approach when the number of
> >> contributions or additional attributes is large.
>
> >> As an example, I'd like to add GEOS attributes to GIS fields, which
> >> includes things like dimensions, area, intersection, boundary, convex
> >> hull, etc.
>
> > If we store the geometry /as/ a GEOSGeometry object, then it already has
> > all those attributes/methods on it :)
>
> > I have done a first stab at lazy-instantiated geometries, which is
> > nearly working how I expect it to. I want to look at Gulopine's coercion
> > work first, but I'll attach it to a ticket by the end of the weekend so
> > you can look at it and see if thats the way we want to head.
>
> Ok, my lazy-instantiated geometries patch is in there now (#4322)
>
> It uses a python descriptor to store an internal GEOSGeometry object,
> and creates a GEOS object when you assign a text/hex string to it. The
> GEOS object is only created on first access, before that it lives as the
> string retrieved from the db.
>
> Because model.my_geometry_field now returns a GEOS object, doing area,
> centroid, etc is as simple (and clear) as: model.my_geometry_field.area
>
> Not sure if it will inspire other creative solutions, but have a look.
>
> Rob :)
>
> --
> One Track Mind Ltd.
> PO Box 1604, Shortland St, Auckland, New Zealand
> Phone +64-9-966 0433 Mobile +64-21-572 632
> Webhttp://www.onetrackmind.co.nz


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Field.contribute_to_class and contrib fields

[Marty Alchin]

On 5/17/07, Robert Coup <[EMAIL PROTECTED]> wrote:
> Yeah, I just wonder whether for files/images its worth having a python
> object that provides the methods, since there is other metadata besides
> the python file object (like its name!)
>
> eg:
> my_model.attachment.file <- python file object
> my_model.attachment.filename <- filename as stored in db
> my_model.attachment.url <- url to file

Well yeah, I didn't mean have it be just a file object with no other
details. I was meaning it'd be a custom object that would have those
attributes, but also had standard file methods that lazily called file
system operations as necessary. Something like this:

my_model.attachment.filename
my_model.attachment.url
my_model.attachment.read()
my_model.attachment.write()

Such that when .read() or .write() (or whatever else) are called, it
open()s the file if it's not already open. That way, you could even do
things like write out PDFs to disk, using ReportLab.

p = canvas.Canvas(my_model.attachment)

I suppose adding .file wouldn't be the end of the world, but it'd be
more impressive without it, anyway.

> Side effect is now that it's always lazy - if i pass in the text
> "BANANA" it doesn't figure out its invalid before it's used later.
>
> Ideal case:
> - it's lazy when loaded from the DB
> - it's not lazy the rest of the time, so if i set an invalid value i
> find out straight away.

Well, I don't know what all formats can be used to instantiate
GEOSGeometry, but you might be able to at least put together a regex
to validate that it fits the format(s). But again, I don't know all
you're doing, so I can't say whether that would suit your needs well
enough.

> I think in a few cases one might need to check whether things are valid,
> or do some extra magic.
>
> I changed it so that if __set__ gets a valid object (eg. just
> instantiated from __get__) then it does the srid-init. Maybe __get__
> could call out when its instantiating to a method on the field, which
> can do extra initialisation.

Hrm, very interesting. So the srid magic happens every time a new
object is instantiated with __get__, as well as every time a new
object is set directly, but never when just retrieving a previously
cached object. I'll work with that a bit and see if that will do the
trick. Looks very slick though.

-Marty

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: django-values update

[Marty Alchin]

On 5/17/07, Jacob Kaplan-Moss <[EMAIL PROTECTED]> wrote:
> The only thing holding me back from full support is the name; "values"
> doesn't explain enough about what the app does to me. Yes, it's
> bike-shedding, but Django's always had a nice color scheme.  That is,
> names are important (to a point).

I completely agree. So far, I think I've heard one vote each for
"options", "settings" and "presets". Like many open source projects,
this started as an in-house thing, and I just called it "my values
stuff" and the team knew what I was talking about. When it came time
to release it to the public, I couldn't manage to describe it in
anything short of a full paragraph.

The real trick here is to find a short phrase that would work as a
title for its documentation, and a one- or two-word name for the
package itself, that would (ideally) reflect the name of the
documentation. I think it would be best if it could somehow reflect
the fact that this is intended to separate the role of programmer from
that of manager, but I really don't know how to go about that. I'll
keep thinking on it.

> * Minor clean-ups (I've got some nits to pick about a few things, but
> I can pick 'em myself and send Gulopine a patch)
> * Unit tests (as part of the Django test suite)
> * Sign-off from the other core devs.

I'm glad for any clean-ups you can offer. This was very much a one-man
job, and I know I can tend to focus on just "getting it done" when
there are more elegant and readable techniques available.

As for unit tests, I had started on some a while back, but those got
put aside when the feature change requests came in. I'll get back to
work on them and hope to have them ready sometime next week.

I'd also add documentation to this list. I know I wrote up a good bit
on the Google Code wiki, but I don't think it would fit very well with
Django's existing docs if it were taken verbatim. I don't know if you
guys have somebody who writes docs or if it'd be best if I do it, but
I have a soft spot for documentation, so I'll at least give it a go.

-Marty

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: django-values update

[Marty Alchin]

On 5/18/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> Nice that it already exists as a standalone app, though, so that people
> can play with it a bit without having to apply patches. That's good
> development practice.

That's been a big part of shaping the current version of the app. I
would definitely recommend that process to anyone else looking to
write a contrib app for Django, as it makes it much easier for the
eventual audience to try it out in real-world projects. I got
suggestions and requests for things I never would've considered if I
hadn't had that feedback.

-Marty

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Changing django.contrib.auth to make passwords optional

[Marty Alchin]

On 5/18/07, Jacob Kaplan-Moss <[EMAIL PROTECTED]> wrote:
> I'm +1 on these changes, including using "!" as the "look somewhere
> else for the password" designator.

Would it really be "look somewhere else for the password" or would it
be more like "if you got this far (all other authentication methods
failed), and you got this value (no password is stored), you're out of
luck"? It may seem like a strange distinction to make, but it goes a
bit to the philosophy of it, which could affect its implementation in
general.

The way you worded it, I would expect the implementation to check try
a password first, and if it doesn't find one, it would then look at
other sources (like OpenID). This would mean that users would be
required to not have a password in order for other schemes to work.

I doubt that's the way you meant it, but it might be worth being clear
on the subject, especially if the process gets documented for the sake
of future authentication authors. I'm probably just splitting hairs
though, so disregard as necessary.

-Marty

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: django-values update

[Marty Alchin]

On 5/18/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> No third-party app should expect to be installed in django.contrib,
> because that would require app users to modify their pristine (and
> possibly unwritable) Django source. Presumably django-values was written
> that way as some kind of subtle hint that he wants to include it in
> contrib/ in this case. It would have been better if it could be
> installed standalone, though, I agree.

Well, I did expect it to be considered for inclusion into the official
distribution, and I won't pretend that it certainly wasn't subtle.
Admittedly, I hadn't considered people using it in "real"
installations prior to a judgement regarding its inclusion, so I wrote
it for core developers to test it out, and I thought django.contrib
would be acceptable for that.

In addition to your comment (which Jacob should put in the
contributing document when it happens), I've gotten enough feedback to
let me know that people are indeed using it beyond what I originally
expected. I'll definitely do better about that in the future, and my
next update will include those changes, in the code as well as the
documentation.

> Generally, a third-party app writer should design their code so that it
> can be installed anywhere on the user's Python path. That's pretty
> normal Python practice. The app writer (not the app installer) gets to
> choose the directory name for the application, though, so that they can
> safely write "from my_chosen_app_name import ..."

And if I had done that from the start, I would've been forced to come
up with a better (or at least, more original) package name than
"values". Anybody got a time machine?

-Marty

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposed Field API additions

[Marty Alchin]

On Fri, Jun 8, 2012 at 8:45 AM, Jacob Kaplan-Moss wrote:

> Can't this be done by auto-discovering subclasses of Field
> (Field.__subclasses__)?


Unfortunately, __subclasses__() doesn't work down through the whole
hierarchy, just one level deep. So unless we plan to walk the tree to find
them all, some other form of registration would probably be better.

-Marty

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: finding the source of "interesting" SQL queries

[Marty Woodlee]

So the straw man solution we've implemented for now looks like this. 
(Background: we have a settings.THREADLOCALS construct which serves in a 
capacity similar to Ned's global requests; also, our monkeypatch_method 
decorator does what you probably think it does.) First, we've a tiny bit of 
middleware:

from django.conf import settings

class RequestURIMiddleware:
"""
Set the request URI in THREADLOCALS so that it can be used to comment
SQL queries with what triggered them.
"""
def process_request(self, request):
settings.THREADLOCALS.request_uri = request.build_absolute_uri()

And then we've monkeypatched the cursor method on BaseDatabaseWrapper like 
so:

from django.conf import settings
from django.db.backends import BaseDatabaseWrapper, util
from patchers import monkeypatch_method
import traceback
 
class CommentingCursorWrapper(util.CursorDebugWrapper):
def execute(self, sql, *args, **kwargs):
"""
Before sending to the DB, this adds a comment to the SQL with notes 
about the query's origin
"""
try:
if getattr(settings, 'SQL_COMMENTS_ALWAYS_TRACEBACK', False):
raise AttributeError
# THREADLOCALS.request_uri is (usually) populated by 
RequestURIMiddleware
origin_comment = ' /* Originated from request to 
{0}'.format(settings.THREADLOCALS.request_uri)
except AttributeError:
# If no URI available (e.g., Celery task), report the first 
non-Django point in the call stack:
tb = reversed(traceback.format_stack()[:-1])  # walk it bottom-up, 
excluding this frame
for frame in tb:
if 'django' not in frame:
origin_comment = ' /* Originated 
at{0}'.format(frame.split('\n')[0])
break
origin_comment = origin_comment.replace('%', '%%')
origin_comment = origin_comment.replace('*/', '\*\/')
sql += origin_comment + ' */'
return self.cursor.execute(sql, *args, **kwargs)

 
@monkeypatch_method(django.db.backends.BaseDatabaseWrapper)
def cursor(self, *args, **kwargs):
return CommentingCursorWrapper(cursor._original_cursor_function(self, 
*args, **kwargs), self)


So in short, we comment the SQL with the URI when it's available, and fall 
back to the (presumably more expensive to gather) traceback information 
when it's not.

This is just the monkeypatch we're throwing in to solve our immediate 
problem, but it'd be easy enough to convert into a patch. Before we do, any 
thoughts/criticisms of this approach?

Best,
Marty
http://www.mapmyfitness.com/profile/woodlee/

On Monday, October 22, 2012 11:23:47 PM UTC-5, Matt McClure wrote:
>
> Thanks, Russell. That's similar to the approach we were thinking of 
> implementing. Hopefully we'll have a straw man to share shortly. 
>
> Matt
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/-X0AFXuzDv8J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: finding the source of "interesting" SQL queries

[Marty Woodlee]

Yeah, I don't think we'd try to do anything like that in our actual Django 
patch... the THREADLOCALS thing has worked well for us but I realize that's 
a pretty big shift to impose on others. If anything I think the patch might 
simply modify the existing CursorDebugWrapper to add the traceback-based 
comment based on DEBUG being True (and/or some other setting).

Also, we discovered a small bug in the above for cases where manual queries 
were being sent with semicolons already on them. MySQL was seeing it as two 
queries in the same cursor execution and raising a ProgrammingError. Just 
in case someone stumbles across this while Googling and wants to use this 
solution, here's the diff vs. the above code that fixes it:

-sql += origin_comment + ' */'
+sql = sql.rstrip(' ;')
+sql = ''.join([sql, origin_comment, ' */'])


On Wednesday, October 24, 2012 12:14:49 AM UTC-5, Àlex Pérez wrote:
>
> You are modifying the settings on runtime , that practise is not 
> recommended by django and personally i have had bad experience with that...
> El 24/10/2012 00:26, "Marty Woodlee" > 
> escribió:
>
>> So the straw man solution we've implemented for now looks like this. 
>> (Background: we have a settings.THREADLOCALS construct which serves in a 
>> capacity similar to Ned's global requests; also, our monkeypatch_method 
>> decorator does what you probably think it does.) First, we've a tiny bit of 
>> middleware:
>>
>> from django.conf import settings
>>
>> class RequestURIMiddleware:
>> """
>> Set the request URI in THREADLOCALS so that it can be used to comment
>> SQL queries with what triggered them.
>> """
>> def process_request(self, request):
>> settings.THREADLOCALS.request_uri = request.build_absolute_uri()
>>
>> And then we've monkeypatched the cursor method on BaseDatabaseWrapper 
>> like so:
>>
>> from django.conf import settings
>> from django.db.backends import BaseDatabaseWrapper, util
>> from patchers import monkeypatch_method
>> import traceback
>>  
>> class CommentingCursorWrapper(util.CursorDebugWrapper):
>> def execute(self, sql, *args, **kwargs):
>> """
>>
>> Before sending to the DB, this adds a comment to the SQL with notes 
>> about the query's origin
>> """
>> try:
>> if getattr(settings, 'SQL_COMMENTS_ALWAYS_TRACEBACK', False):
>> raise AttributeError
>>
>> # THREADLOCALS.request_uri is (usually) populated by 
>> RequestURIMiddleware
>>
>> origin_comment = ' /* Originated from request to 
>> {0}'.format(settings.THREADLOCALS.request_uri)
>> except AttributeError:
>>
>> # If no URI available (e.g., Celery task), report the first 
>> non-Django point in the call stack:
>>
>> tb = reversed(traceback.format_stack()[:-1])  # walk it 
>> bottom-up, excluding this frame
>> for frame in tb:
>> if 'django' not in frame:
>>
>> origin_comment = ' /* Originated 
>> at{0}'.format(frame.split('\n')[0])
>> break
>> origin_comment = origin_comment.replace('%', '%%')
>> origin_comment = origin_comment.replace('*/', '\*\/')
>> sql += origin_comment + ' */'
>> return self.cursor.execute(sql, *args, **kwargs)
>>
>>  
>> @monkeypatch_method(django.db.backends.BaseDatabaseWrapper)
>> def cursor(self, *args, **kwargs):
>>
>> return CommentingCursorWrapper(cursor._original_cursor_function(self, 
>> *args, **kwargs), self)
>>
>>
>> So in short, we comment the SQL with the URI when it's available, and 
>> fall back to the (presumably more expensive to gather) traceback 
>> information when it's not.
>>
>> This is just the monkeypatch we're throwing in to solve our immediate 
>> problem, but it'd be easy enough to convert into a patch. Before we do, any 
>> thoughts/criticisms of this approach?
>>
>> Best,
>> Marty
>> http://www.mapmyfitness.com/profile/woodlee/
>>
>> On Monday, October 22, 2012 11:23:47 PM UTC-5, Matt McClure wrote:
>>>
>>> Thanks, Russell. That's similar to the approach we were thinking of 
>>> implementing. Hopefully we'll have a s

Re: finding the source of "interesting" SQL queries

[Marty Woodlee]

Good point, and I agree, allowing the user to use their own cursor wrappers 
is probably the best solution. At least, it would allow us to avoid the 
monkeypatching we're having to use here. From there, this particular 
wrapping could just turn into a django-snippet. I'll try to work on 
something oriented towards that instead.  It may take a 
while--priorities--but I'll post back here when I come up with something. 
Thanks!

On Thursday, October 25, 2012 10:06:51 AM UTC-5, Anssi Kääriäinen wrote:
>
> On 25 loka, 17:52, Marty Woodlee  wrote: 
> > Yeah, I don't think we'd try to do anything like that in our actual 
> Django 
> > patch... the THREADLOCALS thing has worked well for us but I realize 
> that's 
> > a pretty big shift to impose on others. If anything I think the patch 
> might 
> > simply modify the existing CursorDebugWrapper to add the traceback-based 
> > comment based on DEBUG being True (and/or some other setting). 
> > 
> > Also, we discovered a small bug in the above for cases where manual 
> queries 
> > were being sent with semicolons already on them. MySQL was seeing it as 
> two 
> > queries in the same cursor execution and raising a ProgrammingError. 
> Just 
> > in case someone stumbles across this while Googling and wants to use 
> this 
> > solution, here's the diff vs. the above code that fixes it: 
> > 
> > -sql += origin_comment + ' */' 
> > +sql = sql.rstrip(' ;') 
> > +sql = ''.join([sql, origin_comment, ' */']) 
>
> Some other possible problems: 
>   - In PostgreSQL comments nest (this is the SQL standard behaviour, I 
> don't know if other DBs implement this or the non-nesting comments 
> approach). 
>   - If the query ends in line comment (one starting with --) and an 
> attacker manages to supply in a newline then there is potential for 
> SQL injection. This should not be possible if using the uri or single 
> line of the stack trace but worth checking still in the code. 
>
> We have currently two different CursorWrappers, debug and default one. 
> Having a way to supply different CursorWrappers would allow users to 
> do anything they want, maybe rewriting some queries to use stored 
> procedures or whatever... We could also have some often needed 
> cursorwrappers in core. 
>
> I don't think it is a good idea to implement something that will allow 
> annotating the source of the query in SQL comment but nothing else. 
>
>  - Anssi 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/jjQ6CIrA0nEJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Problem with ``Model.objects.create``

[Marty Alchin]

On Tue, Jan 4, 2011 at 3:10 PM, Yo-Yo Ma  wrote:
> Simple question (A or B): Would you rather A) have a
> SomeDjangoException come up where you don't expect it and a visitor
> see a 500 page, or B) have your data ruined by a developer mistake?

Developer mistakes can ruin your data. Full stop.

That has nothing to do with Django, its ORM or the create() method
being discussed here. A developer mistake *always* has the potential
to severely cripple a website or much, much worse. Sure, we could
argue all day about whether this one particular method could add some
extra protections, but please don't try to kid yourself (or any of us)
into thinking that "fixing" create() will somehow make websites safe
from mistakes.

The fact of the matter is this: create() has a particular behavior,
and that behavior is documented. Sure, the documentation *could* be
more clear, but if we wanted to call out every potentially disastrous
mistake in bold with flashing lights and sirens, the vast majority of
the documentation would look like that, and nobody would ever read any
of it. Worse yet, it still couldn't possibly cover the full range of
possible mistakes, so anybody who did bother to read it would still be
wide open to a whole range of problems. It's our responsibility as
developers to understand the way our tools work and use them
accordingly. No amount of framework design will ever substitute for a
well-informed developer using it.

> Your data is ruined when your code's logic (the view) can no
> longer assume that the data's interface (the ORM model) is still valid.

That's just it: the data's interface (the ORM model) *is* still valid.
It's as valid as it's documented to be. An ORM is a device that
communicates between a database and an object. As long as there's
enough information for those two sides to talk to each other, a
conversation can take place. Anything that needs to happen beyond that
is, as you mentioned, your code's logic. You can encapsulate some of
that logic in a full_clean() method and call it separately to make
sure your objects are valid according your code's logic, but all the
ORM itself cares about is whether objects are valid according to the
database requirements.

Unfortunately, this is one of the many issues where "X doesn't work
the way I think it should" gets confused with "X doesn't work the way
it should" or even "X doesn't work." If you have a constructive
argument for how you think it should work that can be implemented in a
way that doesn't break compatibility with other people's code, feel
free to propose that and discuss it. But spinning around about how
"wrong" the behavior is currently doesn't help anybody.

-Marty

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Extending templates dynamically

[Marty Alchin]

On Fri, May 6, 2011 at 12:25 PM, Christophe Pettus  wrote:
> Steal from the best! :)  One additional feature that they added was a dynamic 
> way of doing {{ extends }}.  Rather than specifying the tag in the template 
> source, the inheritance path can be specified directly in the 
> render-equivalent call.  This has proven to be quite useful for those times 
> that an inner template is used in multiple wrapper contexts.  Is this 
> something that might be worth investigating in Django?  Looking at the Django 
> source, the implementation seems quite straight-forward.

Does the current behavior of {% extends %} not do what you want?

http://docs.djangoproject.com/en/1.2/ref/templates/builtins/#extends

In particular, note that you can supply any template variable as the
argument to {% extends %} in order to dynamically inherit from
whatever template your view decides is appropriate. That said, if you
have questions about using {% extends %} in this way, it's probably
best to take it to django-users instead.

-Marty

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Improving the truncatewords filter (and a +1 vote for filters with multiple arguments)

[Marty Alchin]

On Wed, Jun 4, 2008 at 1:21 PM, Aral Balkan <[EMAIL PROTECTED]> wrote:
> Unfortunately, I don't think your ticketing system likes me. I'm
> getting: 500 Internal Server Error (Submission rejected as potential
> spam).

Right on the new ticket screen, under the big heading labeled "Read
this first" you'll find the following line:

"If you're getting rejected by the spam filter, we apologize! The best
way to avoid that is to register for an account and make sure you're
logged in."

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

One more issue with file storage

[Marty Alchin]

Hey all,

I've been fairly quiet on the file storage front for a while, since
it's basically done now, and is just waiting on the streaming upload
ticket to hit trunk first. Since I got to that point, however, I've
had two different people, working on two different storage backends,
both approach me with the same question. It doesn't seem like a
showstopper to me, but I'd definitely like to get it addressed. I've
been mulling it over for a while now and can't come up with a
universally "good" solution, so I'd like to bring it to the group for
discussion.

For a little background, #5361 splits Django's file handling code is
split into two separate groups:

* Models deal with just the file's name and its content. For the most
part, the content just passes through untouched, but certain field
types may inspect the file's contents to get information about it,
such as ImageField extracting width and height information.
* Storage systems take the name that the model knows about and maps it
to an actual storage mechanism. This way, each site is free to decide
how and where its files are stored and retrieved, independent of any
specific model.

Of course, there's always been some overlap here, but it's been kept
to a minimum: models need to know what storage systems to use when
committing the file, but they shouldn't know (or care) about *how* the
storage system deals with the file. It should just be able to pass a
name and a string of content, and blissfully allow the storage system
to do its thing. Normally, a default storage system (specified with a
new setting) is used for all FileFields, though individual fields can
also accept a "storage" argument in order to use a different one. This
helps keep models generic enough that different storage systems can be
used in development vs. production, just by altering a setting, same
as it is for many other features.

Unfortunately, we don't live in that ideal utopia where a single
setting can control everything. Different model fields may want to
specify different locations for their files, different ways of naming
incoming files, or possibly different backends entirely. It's possible
to work around some of these issues by importing
django.core.filestorage.storage, which is automatically an instance of
the default storage system, and extend its class to create a new
storage system, without relying on which one is used under the covers.
However, that only works when the options being changed don't deal
with storing the file directly (for example, file naming works just
fine this way, and is probably the most common use case for it).

Sometimes, a backend may take other options when storing a file, which
map nicely to Django's field-level description of files. For example,
MogileFS taks a "class" option that controls how the file is
distributed among the various machines in the system, while S3 can
accept HTTP headers that should be sent with the file, such as
mime-types or cache configurations. These are clearly specific to each
backend, and not anything Django cares about, so the natural first
response is to simply handle them in backend methods, supplying the
options as arguments to __init__().

The big problem now is that this breaks the separation between
FileField and file storage. Now you're specifying backend-specific
options to individual fields, which doesn't work when you want to use
FileSystemStorage in development, but S3Storage in production. One
proposal to allow this was to accept additional arguments to
FileField, which just get passed to the backend, but that seems even
worse to me, since they'd likely get passed to backends that don't
support them. Fixing that might invovle having each backend explicitly
report which arguments it accepts, and that's not a path I'd like to
take.

So far, I've only found one way to reasonably solve this. I'm not 100%
happy with it, and I'm sure others will have their issues with it as
well, but it works with the current patch, and seems (to me) to solve
the problems at hand. It's just not as clean as I'd like. Say you're
looking to switch between filesystem in development and MogileFS in
production, while including a class of "avatar" for all files using a
specific FileField. It might look like http://dpaste.com/hold/55681/,
which works, but has a few bits of wackiness.

* It relies on the fact that if a backend doesn't know about an
option, it simply won't look for that option. This is a departure from
how options normally works, but it does allow for two different
"default" backends to share a subclass in this way. FileSystemStorage
doesn't care if it has a file_class attribute, it never looks for it
anyway.
* It assumes that the backend is wired to work this way. If someone
else comes along with a backend that doesn't look for options as
attributes (since it's not really the normal way of doing things),
there may be features that can't be used without sacrificing
flexibility. This could be just a documentation issue,

Re: RFC: Django 1.0 roadmap and timeline

[Marty Alchin]

I like it, but mainly that's because I'm not the "maybe" list, and I'm
sure I can get it done in time. I do have one suggestion, though.

On Wed, Jun 11, 2008 at 10:03 PM, Jacob Kaplan-Moss
<[EMAIL PROTECTED]> wrote:
> * "Maybe" features: things that *should* be in 1.0 and should be worked on
>  in the run up to the release. If, however, features on this list aren't
>  completed, they will be dropped.

I think I know what you mean here, but "dropped" sounds an awful lot
like an all or nothing deal: either it makes it into 1.0 or it never
makes it at all. It's probably best to clarify that "dropped" just
means "dropped from 1.0, to be added in the next release after it's
completed." Yeah, it might be obvious to some of us, but a lot of
people could read it the wrong way.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: RFC: Django 1.0 roadmap and timeline

[Marty Alchin]

On Thu, Jun 12, 2008 at 7:06 AM, Forest Bond <[EMAIL PROTECTED]> wrote:
> I think that this is a must-have:
>
>  #285 -- WSGI SCRIPT_NAME and PATH_INFO stuff

Then you'll be glad to know that it's #3 the list of "Must-have
features" in Jacob's email, just a bit below the portion you quoted.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: One more issue with file storage

[Marty Alchin]

On Fri, Jun 13, 2008 at 8:48 AM, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
> I've been implementing a flexible file storage system for Django for
> my own use (hacked in on top of the existing file field) and would be
> happy to make this code available if you have any interest in it. I
> honestly didn't know someone was working in this area or I would have
> gotten involved sooner.

Yeah, I haven't exactly tried to hide it, but I haven't been terribly
loud about it over the past several months either.

> The code allows you specify file storage "strategies" (which are just
> classes that conform to an "interface") and specify them in the field
> constructor. A feature of my implementation that I think works out
> well is that a special descriptor is added to the model objects to
> provide access to the file.

Hrm, that sounds incredibly familiar. If you haven't looked at it
already, check out the patch at #5361. It does exactly what you're
describing, except that it also allows for a default storage class to
be used, without having to specify it on the FileField directly
(though you can if you like). That way, you can switch from one
storage mechanism in development to another in production, by just
changing the settings file.

http://code.djangoproject.com/ticket/5361

> For example, with the following code:
>
> http://dpaste.com/hold/56439/

For what it's worth, I'm not thrilled with having to specify the name
of the descriptor. In #5361, the name of the attribute is the name of
the descriptor that'll be used, just like how ForeignKey works. Beyond
that, your example looks quite a bit like what I'm doing as well.

> Instances of this model will be augmented with an additional field
> "file", as specified in the field constructor, that returns a
> "FileWithMetaData" wrapper containing the filename, file data and a
> dict of addition meta data while The file_id field on the model stores
> a "key" or UUID used to retrieve the file from whatever strategy is
> being used. This descriptor also allows setting (and deleting).

Wouldn't this introduce backwards-incompatibility with any data that
already exists in FileField instances? I suppose that's not a concern,
since you're introducing a new field type, though.

> # returns a FileWithMetaData object
> file = my_object.file
>
> # get the original filename
> orig_name  = file.filename
>
> # replaces the original file and updates "file_id"
> my_object.file = new FileWithMetaData("document.pdf", fp.read(),
> metadata={"owner": "John Q. Public"})
>
> # deletes the file from storage
> my_object.file = None

I kinda like the metadata idea, but to me, that seems better suited
for separate model fields, so it's more easier searchable and
everything. Plus, that helps keep the code changes to a minimum, since
then the file-handling code doesn't have to do anything with it at
all.

> As I said, the current field implementation itself is a little hackish
> because it was designed to integrate with the existing oldforms admin
> without modification and overrides "_save_FIELD_file", etc.

Yeah, it's the "without modification" bit that'll get you. I didn't
have to modify the admin at all (oldforms or newforms) to get things
working, but it required significant changes to Django itself. I
suppose I could've done most of it without any changes, but the idea
from the start was to enable this stuff for everybody, so some changes
to the codebase are acceptable. I've fixed several bugs in the process
too, and also deprecated save_FOO_file(), get_FOO_filename(), and the
whole gang.

Check out the documentation in the patch for #5361 when you have a
chance, and let me know what you think.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django community aggregator and non-English posts

[Marty Alchin]

This whole thing reminds me of something I've had rattling around in
my head for a while, and maybe now's the time to bring it to the
group. In addition to the language of blog posts, I've often had a
hard time tracking down information from past conversations that have
happened in the community.

There's an aggregator, IRC log, various mailing list archives, wiki
articles, ticket comments, localized community sites, the list goes on
and on. In the spirit of community-oriented sites (like djangosites,
djangosnippets, etc), I'd like to propose djangochatter, a site
dedicated to aggregated, managing and distributing communication
taking place in the Django community.

Essentially, it could pull together information from *all* those
various sources, index it for searching, provide feeds for various
tags and keywords (users would have to tag entries manually). In light
of this discussion, I expect it could also reasonably auto-detect the
text's language, so users could sign up for a feed for whatever
language(s) they can understand.

Ideally it would also have a complete (or as near as possible) archive
of *past* chatter as well. Pull together all the existing feed
history, mailing list discussions, IRC logs, wiki history, existing
ticket comments, and make them all available in a single place all at
once.

Basically, while I feel the existing aggregator is quite useful, and
has served the community well, I think there's more that can be done,
and there's no need to require the "official" site to handle it. In
fact, I would think that once djangochatter.(com|net|org) is up and
running, the existing aggregator could just redirect over to the
appropriate page and feed URLs and be done with it.

I don't have time to head this up, but I think it would be extremely
useful for everybody. I know I've spoken with a few people about this
in the past, but is anybody interested in heading this up? I'd love to
see it happen, and I'm more than willing to take part in discussions,
but I just can't dedicate myself too much to it at the moment.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django community aggregator and non-English posts

[Marty Alchin]

On Fri, Jun 13, 2008 at 1:23 PM, Alex Koshelev <[EMAIL PROTECTED]> wrote:
> There is existing multi-lingual aggregator http://djangosearch.com/

I didn't realize djangosearch broke articles down by language,
complete with individual feeds! Looks like that's one problem down.
Combine that with Jacob's quick-and-dirty custom search, and maybe
we've already got all the bases covered.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Djangochatter [WAS: Django community aggregator and non-English posts]

[Marty Alchin]

On Fri, Jun 13, 2008 at 1:23 PM, Tom Tobin <[EMAIL PROTECTED]> wrote:
> Ugh, time.  I have a bad habit of assuming I have more of that
> resource available than I ultimately do.  :)  I'd definitely like to
> help, but I don't think I could assume a driving role at this point.
> (Chicago Djangonauts, languishing taghelpers patch, proper multiple
> time zone support ... meh!)  Maybe a mailing list for now?

So ... a new mailing list to discuss how to address the
ever-increasing number of communication outlets? Sounds like a great
idea. :( I don't know what it would take to manage something like that
though, so whatever works.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Djangochatter [WAS: Django community aggregator and non-English posts]

[Marty Alchin]

On Fri, Jun 13, 2008 at 1:38 PM, Tom Tobin <[EMAIL PROTECTED]> wrote:
> Hey, it helps me segment my incoming mail.  ;)

It's always about *you* isn't it? ;)

> Google Groups makes new mailing lists trivial.  Although this may all
> be moot if that djangosearch site pans out as useful.

Well, I'd still like to be able to get feeds for keywords, too. For
instance, with all the file stuff I've been doing, I'm curious to know
what kinds of FileField problems people are dealing with. If I could
just set up a feed for FileField, knowing it would give me a daily
digest of *everything* the community is saying about it, I'd be in
heaven. Other people would be interested to hear chatter about
"newforms-admin" (and "nfa"), "1.0", "streaming uploads", etc.

I've got an email out to Alex Aster, the guy behind djangosearch. If
he's interested in adding some features, this may well be moot. If
not, I think it might still be worth pursuing.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django 1.0 roadmap

[Marty Alchin]

On Mon, Jun 16, 2008 at 11:15 AM, Jacob Kaplan-Moss
<[EMAIL PROTECTED]> wrote:
> As requested, I've also added milestones for 1.0 alpha, beta, and
> final (as well as a "post-1.0" catch-all). Triagers, feel free to use
> these milestones thusly:
>
> * Any *feature* tickets related to the maybe list get put in the beta 
> milestone.

One question about this, since I'm probably best positioned to manage
the tickets related to file storage. #5361 is already added to the
beta milestone, but what should be done about the 15 other tickets
that are related to it[1]? Given how much code #5361 already has to
move, modify and replace, I've been wrapping some of the fixes up in
that one patch (keyword fs-rf-fixed), and the others will be addressed
by the documentation (keyword fs-rf-docs).

This might be considered hijacking the thread, but I can see three ways to go:

* Leave them as they are, and just tell whoever commits #5361 to
reference them in the commit message.

* Move all of the to the beta milestone, since they are indeed being
addressed, and also reference them in the comment message

* Close the fixes as duplicates now, and close the doc tickets as
wontfix once it lands (which will have to be done regardless).

Personally, I'd like to avoid the third option, since I think it's
valid to keep them going, as there's not a proper branch to checkout,
like newforms-admin has. The difference between the first two is
purely bookkeeping though, and I have no preference either way.

-Gul

[1] http://tinyurl.com/6rxpyp

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django 1.0 roadmap

[Marty Alchin]

On Mon, Jun 16, 2008 at 2:17 PM, Jacob Kaplan-Moss
<[EMAIL PROTECTED]> wrote:
>
> On Mon, Jun 16, 2008 at 12:43 PM, Marty Alchin <[EMAIL PROTECTED]> wrote:
>> * Leave them as they are, and just tell whoever commits #5361 to
>> reference them in the commit message.
>>
>> * Move all of the to the beta milestone, since they are indeed being
>> addressed, and also reference them in the comment message
>
> Do both of these, and also tag with with a common tag (see, e.g., the
> qsrf-fixed or nfa-fixed tag). I'd suggest "fstorage-fixed" or
> something. Actually, if you tag them first it's easy to move them all
> to the milestone -- I'd like the milestones to reflect the actually
> ticket count (even for dups of this nature).

Tags were already done, but I've now moved them all into the beta
milestone as well. I hope I didn't overstep my boundaries here, but I
also added a note on the VersionOneRoadmap to explain the related
tickets and how the tags should be used.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Django book seeking Tech Reviewer

[Marty Alchin]

Hey all,

First, I apologize for posting this on django-dev, since I know it's
not development *of* Django, but I didn't want to add to the noise on
django-users, and most of the people I'm hoping to reach are frequent
readers of this list. No need to flame, I got it.

As most of you are no doubt aware, I'm writing Pro Django for Apress,
and I'm desperately in need of a technical reviewer. I had one, now I
don't. I don't want to get into it.

It's a paying gig (though I don't personally know how much; Apress
writes the checks), and I need someone who knows Django pretty well,
or at least isn't afraid of diving in. A big part of the job is making
sure I'm writing what's actually happening in Django, and to verify
that all my code samples run correctly.

I can't speak for exactly what kind of time commitment it will
require, but I can confidently say you'll need plenty of time. I know
it's tough this summer, with the lead-up to 1.0, and I don't want to
detract from that, since it's so important to all of us, but if
anybody can spare the time to help out, it may well mean the
difference between the book getting published and ... well, I'd rather
not consider the alternative.

If you're not available, but you know someone who is or who might be,
I'm accepting recommendations as well.

Please contact me off-list, as I don't want to clutter this up any
more than I already have.

http://prodjango.com/

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Call for testing: streaming uploads (#2070)

[Marty Alchin]

On Thu, Jun 26, 2008 at 3:14 PM, Jacob Kaplan-Moss
<[EMAIL PROTECTED]> wrote:
>
> Hi folks --
>
> As far as I'm concerned, #2070, adding large streaming uploads, is
> done. I'd like to get some public kicking-of-the-tires before I push
> the change to trunk (which won't happen before Tuesday: I'm taking a
> long weekend off).

Yay!

I can't help too much with most of your needs, except possibly coming
up with a custom upload filter just to figure out how it works.
Instead, I'll work on getting the file storage patch to play nicely
with it, so that once it's merged, we're already a step ahead on
another item in the list.

I don't expect it to take much work, but I've been holding off until
2070 either merged or hit a stage like this, so I wasn't working with
too much of a moving target. I'll stay in the loop on any changes that
are made as a result of this testing, and get us in better shape once
it lands.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: More secure user password reset

[Marty Alchin]

On Fri, Jun 27, 2008 at 8:31 PM, Simon Willison <[EMAIL PROTECTED]> wrote:
> I've got code for this lying round which I'd be happy to donate if
> people agree this is the right approach.

I personally much prefer this approach. I've worked in a couple
communities where personal attacks were quite frequent, and a common
tactic was to claim a password was lost on someone else's account. It
didn't give them access to the account in question, but it would
adequately lock the person out if they happened to visit the site
prior to checking their email.

Of course, sites like that also tended to have password change forms
that accepted GET requests, and didn't have sufficient XSS protection.
As you can imagine, wackiness ensued.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Community representation, or, #django user "Magus-" needs to go far away

[Marty Alchin]

On Wed, Jul 2, 2008 at 12:55 AM, Kenneth Gonsalves
<[EMAIL PROTECTED]> wrote:
> actually this debate belongs on django-users (which is why I put in
> my 2 paise) - I think if taken there we can get a real measure of the
> pros of his help against the cons of his curtness at times (I
> wouldn't call it being rude)

Or we could all just stop trying to deal publicly with what should be
a private issue.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Ticket #7591: Authenticate By Email Support

[Marty Alchin]

On Wed, Jul 2, 2008 at 1:24 PM, Paul Kenjora <[EMAIL PROTECTED]> wrote:
> Reasons why an alternate backend or view is not good:
> 3. Maintaining my own back end, means others are doing the same, again poor
> re-use.

Not if you:

1. Write it well
2. Distribute it freely
3. Make its presence known

You'll find people in the Django world are quite happy to use
third-party applications.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Ticket #7591: Authenticate By Email Support

[Marty Alchin]

On Wed, Jul 2, 2008 at 2:56 PM, George Vilches <[EMAIL PROTECTED]> wrote:
> On Jul 2, 2008, at 1:24 PM, Paul Kenjora wrote:
>
>> I understand the resistance but you've got demand, you've got a
>> willing developer, and you've got a clean fix that significantly
>> improves the adaptability of the framework.  What better reason
>> would you need?
>
> Someone who has a proven history of contributions and maintenance of
> the core framework for a significant period of time.
>
> 
>
> Providing the addition as a 3rd party framework allows the community
> to vet your work and decide whether 1) it's worth inclusion, and 2)
> that you have the stamina to maintain it in Django for a long while to
> come.

I'll also add that appealing to the community is often a much better
way to get feedback on improvements that can be made. My first
contribution, signed cookies,[1] started as a patch in Trac, that
ended up moving to its own application, at which point I got a much
better response of people using it, finding bugs, suggesting
enhancements, etc.

My second, dbsettings,[2] started as its own project, then had the
possibility of being included in trunk, so I immediately started
making changes to satisfy the immediate onslaught of suggestions. As
time went on, I came to regret that, as sitting back and riding the
normal community support wave would've made for a much better, more
usable application.

I think the main issue is that when something's sitting in trunk, or
even in Trac, there's something of a bureaucracy, whether real or
perceived, where the core developers are the gatekeepers of all
decisions. People are quick to identify bugs, but given how long some
decisions can take in trunk, people are less likely to speak up about
suggestions or improvements.

On the other hand, third-party applications in general have a better
track record for acknowledging and responding to feedback much more
quickly, and by more people. This helps get everything refined and
polished fairly quickly, and with a high level of quality.

And, as has been said before, if you write a good application, support
it well, make it consistent with how Django itself works, and help a
lot of people with it, there's a good chance it could be considered
for inclusion into Django itself later on. Certainly a much better
chance than if you just throw it in a ticket and ask why it hasn't
been committed yet, anyway.

Jonathan Buchanan's django-tagging[3] is a prime example of this, and
will probably be one of the first considered for django.contrib after
Django 1.0 hits store shelves. It's already on the shortlist of
applications that many developers install in every new environment
they work with, and once Django itself settles down, it'll probably
work its way in fairly quickly. Mind, though, that I'm not one of the
people who makes those decisions, I've just been observing
conversations for quite a while.

-Gul

[1] http://code.google.com/p/django-signedcookies/
[2] http://code.google.com/p/django-values/
[3] http://code.google.com/p/django-tagging/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: UploadedFile API changes (was: Is it UploadedFile.chunks() or .chunk() ?)

[Marty Alchin]

On Thu, Jul 3, 2008 at 12:22 PM, Jacob Kaplan-Moss
<[EMAIL PROTECTED]> wrote:
> I just talked about this with Mike on IRC; I think chunk() is
> non-obvious and we're gonna change it to chunks(). Hopefully nobody's
> written so much code in the last week to be pissed.
>
> Also, we're going to change UploadedFile.file_name to
> UploadedFile.name to more closely match the built-in file API.
>
> Sorry about not thinking this through fully before committing, but
> it's gonna be better this way.

Well, as long as the floor's open, I'd like to make sure file storage
code is consistent. Currently, it uses filename instead of file_name
or just name, and it also uses size instead of file_size. I assume I
should go with name instead of filename, but should I use file_size,
or should the upload stuff use size?

On a related note, when merging #5361 with the new trunk, should the
file-related code, such as moving files and whatnot, be moved into
django.core.filestorage.filesystem? Also, should the code in
django.core.filestorage be moved over to the new django.core.files
package instead?

So many bikesheds, so little time.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: UploadedFile in cleaned_data

[Marty Alchin]

On Thu, Jul 10, 2008 at 4:46 AM, Ivan Sagalaev
<[EMAIL PROTECTED]> wrote:
> I've seen already two people highly confused with this thing. After
> successful validation of a form with file upload
> form.cleaned_data['fieldname'] contains an instance of a wrapper --
> newforms.fields.UploadedFile. People confuse it with the real
> core.files.uploadedfile.UploadedFile which is accessible through
> form.cleaned_data['fieldname'].data
>
> If I understand right the only point of a wrapper is holding an original
> name of a file that was uploaded. I think it would be better to get rid
> of the wrapper, put a read UploadedFile in cleaned_data and add
> "original_filename" as an optional parameter to it. Thoughts?

This was reported in #7614, and fixed in [7859], so if you get a new
copy of trunk, it should be all set.

-Gul

http://code.djangoproject.com/ticket/7614
http://code.djangoproject.com/changeset/7859

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Will oldforms completely go after 1.0?

[Marty Alchin]

On Thu, Jul 10, 2008 at 12:35 PM, Adrian Holovaty <[EMAIL PROTECTED]> wrote:
> It'll be a badge of honor/experience. "I used Django back when forms
> was called newforms!"

Yeah, and everybody at DjangoCon can have a little icon on their name
badges that indicates how many backwards-incompatible changes they've
gone through ... and for those of contributing, how many of those they
caused.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: One more issue with file storage

[Marty Alchin]

On Thu, Jul 10, 2008 at 2:52 PM, [EMAIL PROTECTED]
<[EMAIL PROTECTED]> wrote:
> Sorry for the late reply. I meant to look at this awhile ago but it
> slipped off my plate for a bit. Looking at the latest patch
> (filestorage.16.diff) it does look like a similar approach the main
> differences coming from my need to monkey patch this functionality
> over a stock Django install. It looks to fit the bill though and
> anything I don't have to support myself is a bonus :). Can you tell me
> if your implementation is on schedule to be completed by the 1.0
> release?

At the moment, it looks good. Now that file uploads have made there
way in, I've been working on updating the patch to take those changes
into account, and just last night I got a working test suite. There
are still a few things I need to do to make it more consistent with
how uploaded files work, so it's not really ready for a new patch just
yet, but I hope to have one ready this weekend or early next week.

The issue mentioned in this ticket is the only outstanding question,
but like I've said before, I'm not considering this a blocker to
inclusion in 1.0. Adding support for per-field storage configurations
would be just adding an option, so it'd be backwards-compatible and
can be held off until the big push is over and things settle down a
bit. I'll see what I can come up with before then, but it's not
critical.

> The one main difference with my approach, and a reason for the
> metadata dict, was the use of UUIDs for filenames. After playing
> around automatically renaming files (which truthfully leaves a bad
> taste in my mouth) and allowing for overwrites, etc. I decided to
> identify files with a globally unique "key", or UUID (which would
> serve as the filename when saved to disk) and store the original
> filename along with additional meta data separate from the raw "data".

I guess I can understand this, allowing duplicate filenames without
adding numbers or underscores on the end of them. I'll admit that it'd
be difficult to pull this off as a default behavior, and bolting it on
aftermarket would probably look pretty ugly, but I expect it'd still
be possible. Once I get the new patch ready, I encourage you to take
it for a spin and see if what you're doing gets any easier.

> This was also influenced by the Amazon S3 API which was going to be
> implemented as an alternate strategy for this project.

There's also work being done on an S3 storage system that'll be dead
simple to drop in once my file work lands. I don't have a link for it
offhand; David, if you're listening, can you point the way to your
repo? It's gonna be some good stuff.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Test Suite on a mac

[Marty Alchin]

On Thu, Jul 10, 2008 at 3:23 PM, Jeff Anderson <[EMAIL PROTECTED]> wrote:
> So I'm a linux user, but I have a MBP. I was playing with the django test
> suite a while back, and some test failed on my mac but would succeed on any
> of my linux machines. After discovering the django buildbot, I thought it
> might be useful to have a buildbot that runs on a mac.

I've wondered if it would be possible to set up a buildbot on a
Windows machine as well, since I recently found a bug[1] in our tests
that would've been caught earlier if we had one. I don't have access
to a box that could be dedicated to it, though. And of course, the
real trick is whether separate buildbots can all report their results
to the main instance, so they can all show up in the same place.

-Gul

[1] http://code.djangoproject.com/ticket/7696

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Add a helper function in UploadedFile's API for writing files on disk?

[Marty Alchin]

On Fri, Jul 11, 2008 at 4:14 AM, Julien Phalip <[EMAIL PROTECTED]> wrote:
> Not sure if we're talking about the same thing here. What I ask is if
> it'd be a good idea to have a generic method for UploadedFile which
> lets copy/move the file to where ever you want. These method would
> have to be overriden by InMemoryUploadedFile or TemporaryUploadedFile,
> but would allow to have something as simple as:
>
> uploaded_file.move_to("some/path/")
>
> I agree with Ivan that it is a common operation and that having it
> core in the API would avoid some boilerplate code, and as result that
> would prevent potential mistakes by the developer.

Okay, two things:

First, Django does now contain the code you're asking for, at
django.core.files.move, which performs memory-friendly file moves in a
platform-independent way.

Second, file uploads are really just half of the equation. I'm
currently working on a patch for file storage, which will handle the
other side of it. Right now, yes, the focus has been on getting the
file uploaded in a sane manner, but once the other half lands (I'm
nearly done with a new patch for it), it'll be dead simple to save
uploaded files without wasting memory *and* without caring what
backend you're using.

Here's an example of what the API would probably look like:

from django.core.files.storage import default_storage

default_storage.save('path/to/files/%s' % uploaded_file.name, uploaded_file)

That API may change slightly before it gets committed, but behind the
scenes, that would use shutils or os.rename, or chunk directly to S3
or whatever is most efficient for what you're trying to do.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: API change in set_cookie for 1.0?

[Marty Alchin]

On Tue, Jul 15, 2008 at 7:49 PM, Jeremy Dunck <[EMAIL PROTECTED]> wrote:
> I was using HttpResponse.set_cookie for the first time and annoyed
> that expires has to be a properly-formatted string.
>
> Why not take a datetime and do the formatting in the function?
>
> Obviously this is a breaking change, unless we have an isinstance for
> backwards incompatibility.

When I looked into set_cookie a while back, I was surprised how much
was just being passed verbatim to SimpleCookie, which ended up causing
some API confusion with secure cookies as well.[1] I don't know if
it's worth looking at other attributes, but there are probably some
other areas the API can be improved as well.

-Gul

[1] http://code.djangoproject.com/ticket/6657

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Last call for 1.0 alpha

[Marty Alchin]

On Mon, Jul 21, 2008 at 11:37 AM, Tom Tobin <[EMAIL PROTECTED]> wrote:
> Gah, I really haven't had time to do the
> move-code-out-of-__init__-modules dance.  I'm fairly certain that
> it'll be completely backwards compatible (via re-importing back into
> __init__), though, so I'm not too worried; I got a tad anxious about
> it a couple of weeks back, but I can't come up with a scenario that
> would break existing code.

I certainly hope 1.0 alpha isn't about hitting the
backwards-compatible limit yet, because if it is, #5361 has failed to
make the cut in time. The way I understand it, we still have some time
to introduce mild incompatibilities in 1.0 beta, when it'll hit a hard
freeze. jacob, is this correct?

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Model method decorators

[Marty Alchin]

On Thu, Jul 24, 2008 at 9:54 AM, zvoase <[EMAIL PROTECTED]> wrote:
> Developers,
> I've just looked at the latest revision (r8068) for Django and there
> are a couple of things I'd like to suggest.

I've already replied to part of this email, but I'd like to take a
minute to also reply to the other portion, now that I've taken the
time to read through it.

> I propose the addition of 4 decorator functions, to
> ``django.contrib.admin`` (or a relevant submodule), which could be
> used to decorate functions so that these attributes could be set
> automatically, like so::
>
>class MyModel(models.Model):
>...
>
>@short_description('Custom method...')
>def custom_method(self):
>return something()
>
> I feel that this way is far more Pythonic, and makes code look a lot
> nicer. I've put some Python source at the end of this message which
> demonstrates the idea, and defines a generic attribute-setting
> decorator-making function which allows more decorators to be created
> if the admin changes in the future.

This sounds awfully familiar, and I don't know why. Oh wait, yeah I
do! I proposed this very thing about two years ago. At the time I was
more focused with just the alters_data attribute, but the idea is the
same. Please take a quick read through the comments on the ticket[1]
to see why this has been rejected in the past.

-Gul

[1] http://code.djangoproject.com/ticket/3009

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation, magic, and is it really that useful?

[Marty Alchin]

On Mon, Sep 22, 2008 at 4:03 PM, Andrew Godwin <[EMAIL PROTECTED]> wrote:
> So, I'd love people's opinions about where I should take this - to a
> dark corner and hide it, to a separate app like django-extensions (which
> involves keeping the horrid hacks in there), into a separate app but
> with patches to core to make it less hackish (i.e. more signals), or add
> it as my 1.1 pony with the proviso that I'm happy to write all the code.

Personally, I think it's best suited for a django-d13n (that's d13n
for denormali(s|z)ation) app that stands on its own. That way, you can
provide a host of other options as you see fit, have other people get
onboard and provide whatever they use, and so on.

I think there's definite merit to what you're doing, but I just don't
see it ever being in core, unless it requires some freakish internal
changes. If you do need changes to core, or you can at least frame
them in a way that would provide wider benefits, you'll definitely get
a fruitful discussion on it, at least.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Denormalisation, magic, and is it really that useful?

[Marty Alchin]

On Tue, Sep 23, 2008 at 8:44 AM, Steve Holden <[EMAIL PROTECTED]> wrote:
> This appears to be a proposal to re-implement triggers inside Django.

I suppose it is. But then, perhaps triggers were a re-implementation
of application-based denormalization. ... The chicken? The egg? The
world may never know. :)

> I can see there are benefits if the underlying DB platform won't support
> triggers, but wouldn't triggers be the preferred solution when they're
> available? That way there is no chance that changes can be made outside
> the scope of the denormalization, and hence no need to recompute the
> denormalized values.

Perhaps the biggest downside is that distributed applications can't
make use of it, since triggers aren't cross-platform. Sure, you could
argue that denormalization is best suited for individual projects,
rather than distributed apps, but I could see uses for it. Imagine a
full-blown forum app that wants to denormalize its post counts.

Without some Python-based approach, all I could see is maybe adding a
cross-platform "create trigger" API (ugh) to Django, which an
application could then use to set up its triggers during syncdb.
Otherwise, something like that forum app would have to implement a
trigger for all available backends or just ship with instructions on
how to set it up yourself.

That said, you're right, I would think it's usually safest to put that
stuff directly in the db. That way, if you have other applications
using other languages or perhaps just other ORMs, everything's all
intact. I'm pretty sure I've seen arguments on both sides of the
issue, though, so I'm not sure there's any objective answer to it.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Python 2.6 abstract base classes

[Marty Alchin]

Looking over the new things in Python 2.6, I wonder if there are plans
to inherit from any of the new abstract base classes in Django. It
looks (to me) like it could be backwards-compatible, using an import
like so:

try:
from collections import Sequence
except ImportError:
# Fallback for Python 2.5
Sequence = type('Sequence', (object,), {})

Django obviously works just fine without these at the moment, so
there's no pressing need to do so, but if they catch on and 2.6+
libraries start checking for them, it might be beneficial if some of
our custom types, like QuerySets, inherited the appropriate classes.

I'm not really in favor of using or not using them, but I'd just like
to know if they're in the plan, in case I should add some notes in my
... documentation.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Python 2.6 abstract base classes

[Marty Alchin]

On Fri, Oct 3, 2008 at 7:34 AM, Steve Holden <[EMAIL PROTECTED]> wrote:
> Trendy might be superficially appealing, but I personally felt 2.6 would
> have been just as useful without the ABC backport.

Actually, I don't find it particularly appealing myself. The PEP
specifically states that it isn't intended to replace duck typing, so
I'm having trouble understanding the rationale in the first place. I
just wanted to make sure I knew now if there was any plan to support
them in the near future, so I could mention it in my book if
necessary, while I still have half a chance.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Python 2.6 abstract base classes

[Marty Alchin]

On Fri, Oct 3, 2008 at 9:48 PM, Malcolm Tredinnick
<[EMAIL PROTECTED]> wrote:
> Think "interfaces". Given that you wrote the file storage stuff and are
> no doubt aware that we still have lots of fuzzy issues where something
> may or may not be able to be used where a Python file object is used,
> you can probably see that there are some uses for this.

Yeah well, I notice a conspicuous lack of any File ABC. ;) I have a
feeling they're still working out how many flavors to have
(ReadableFile, WriteableFile, SeekableFile, etc.) before including it,
but I don't know for sure, obviously.

> There may well be a plan. I, personally, don't have a plan right this
> minute because Python 2.6 is too new and is just one of many Python
> versions we support. Trying to predict the future in open source is
> generally going to be hard. All you can really do is put in a note
> saying "this is accurate as of this date" (something that's been
> unfortunately lacking in previous Django books, so please convince your
> editor to put something in of that nature).

Yeah, understood. Having 1.0 out the door helps tremendously, because
I can target a very stable version with a clear number to work from.
I'll make sure it's plainly labeled though. :)

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Python 2.6 abstract base classes

[Marty Alchin]

On Fri, Oct 3, 2008 at 10:39 PM, Eduardo O. Padoan
<[EMAIL PROTECTED]> wrote:
> IIRC, the file-ABCs are in the new `io` library.

Ah, indeed. I was looking for something called "File" or similar, not IOBase. :)

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Lazy-loading model fields

[Marty Alchin]

I won't pretend to know the whole story here, but I will take issue
with some of your points. I'll leave others to people who have history
on the subject.

On Tue, Oct 14, 2008 at 3:14 AM, Jesse Young <[EMAIL PROTECTED]> wrote:
> * If you split them out, you have to manually create one model every
> time you create the other model.

If by "manually create one model" you mean "type out a brand new model
definition in Python" you're flat-out wrong. I've done quite a bit of
work with generating models dynamically[1] and what you're asking for
could be made very simple using that approach. Rather than this:

thesis = models.TextField(lazy=True)

you could use this instead:

thesis = lazy_utils.LazyTextField()

And LazyTextField could create the associated model behind the scenes,
set up a descriptor for loading its contents only when necessary,
cache those contents so they only get queried once, etc. And all this
without a single patch to Django's core and without you having to
manually maintain another model.

> * If a fraction of your queries actually need those large fields, then
> you have to do a join, which can be expensive.

It doesn't address this, of course, but like I said, I'll leave that
to the professionals.

> * If your data access pattern changes after your application has
> actual data, it is much harder to migrate your data than to make a one
> line change to the application code (i.e., by adding lazy=True to the
> field definition).

Admittedly, the relational approach would require a data migration,
but that's life sometimes. There are plenty of tools now to make
changes like this relatively easy to deal with, and I expect a custom
LazyTextField could be written to give extra hints to any of them.

> * Maintaining more tables is more work than maintaining fewer tables.

Again, this depends on what you mean by "maintaining". At a raw
database level, yes, that's true, but if you're talking about the
Python level, you can make that extra table almost non-existent and
work with it as a regular field.

Anyway, I don't expect this information to solve your problem, but I
hope I've cleared up a few inaccuracies in your justification. Maybe
it'll help you with whatever solution you do come up with.

-Gul

[1] http://code.djangoproject.com/wiki/DynamicModels

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Lazy-loading model fields

[Marty Alchin]

On Tue, Oct 14, 2008 at 12:34 PM, Jesse Young <[EMAIL PROTECTED]> wrote:
> A LazyTextField that manages a dynamic model behind the scenes sounds
> like a neat idea. That would certainly make the Django side of this
> easier to manage. (But of course I have a slight bias towards code
> that exists already -- has anyone actually implemented this?)

I don't think it's out there anywhere, but if you like I could whip up
the code for it in about 15 minutes (probably less). I just haven't
bothered because I don't personally need it.

> Anyway, I'm only trying to point out that inline lazy fields are
> *sometimes* useful, and that Django shouldn't force people to do it by
> splitting out fields into separate tables.

Damn it. Now you've gotten me thinking of ways to possibly manage a
field the way you want (one table, lazy loading, no data migration, no
patches to django, one-line change to the model). I'm 98% sure it's
possible, and I know of many of the things that would need to be
addressed to do so, but I really shouldn't be spending my time on it.
I probably won't be able to get it out of my head until I get it
working though, unfortunately. If I do write it down, I'll let you
know.

Of course, it would also have to come with big biohazard signs warning
people that there could (and likely would) be unforeseen side-effects,
particularly in the admin, serializers, modelforms ... actually,
pretty much anything that introspects your model would never see that
field. I'm not sure if that's suitable for your case, but it certainly
wouldn't be good enough for wide distribution.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Final(ish) 1.1 feature list

[Marty Alchin]

The biggest issue is that we can't edit anything. There's no "Edit
Page" button, and when I manually add "?action=edit" to the URL, it
gives me an editor form, but upon submission, I see the culprit: 403
Forbidden (WIKI_ADMIN privileges are required to perform this
operation). Any hope of getting it opened up for the rest of us?

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Final(ish) 1.1 feature list

[Marty Alchin]

*sigh* I was afraid someone would say that. I didn't think I had to
state that I was indeed logged in at the time. Perhaps you have
WIKI_ADMIN priveleges? :) I can edit other articles on the wiki, just
not that one.

-Gul

On Tue, Nov 25, 2008 at 11:49 AM, Jeremy Dunck <[EMAIL PROTECTED]> wrote:
> On Tue, Nov 25, 2008 at 10:34 AM, Marty Alchin <[EMAIL PROTECTED]> wrote:
>>
>> The biggest issue is that we can't edit anything. There's no "Edit
>> Page" button, and when I manually add "?action=edit" to the URL, it
>> gives me an editor form, but upon submission, I see the culprit: 403
>> Forbidden (WIKI_ADMIN privileges are required to perform this
>> operation). Any hope of getting it opened up for the rest of us?
>
> If you log in, you can get an edit button at the bottom.
>  (Or at least, I could.)
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

<>

Re: Final(ish) 1.1 feature list

[Marty Alchin]

Thanks, guys. All that just to delete a question mark from my name
after signed cookies. :)

-Gul

On Tue, Nov 25, 2008 at 12:04 PM, Jacob Kaplan-Moss
<[EMAIL PROTECTED]> wrote:
>
> On Tue, Nov 25, 2008 at 11:02 AM, [EMAIL PROTECTED]
> <[EMAIL PROTECTED]> wrote:
>> Ok, the page is now editable.
>
> Yeah, that's my bad. Thanks for fixing it, Alex.
>
> Jacob
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Custom Fields API Cleanup

[Marty Alchin]

On Tue, Dec 16, 2008 at 1:03 PM, Jacob Kaplan-Moss
 wrote:
> This seems like a good idea to me, and an easy one to implement; I'm
> all for it. Are there any backwards-compatibility concerns here? It
> doesn't look like it -- we're just talking about adding a new hook to
> SubFieldBase, right?

My only concern with this proposal so far is that I fear it'll fall
short of what many people will really be looking to do here. Every
time I've done this dance, I ended up something a bit more
complicated, because I also needed to delay the processing of the db
value until it's actually used. Basically, the same idea behind
implicit ForeignKey lookups and lazy geometries in GIS.

While I do realize that there are use cases where you'd want the
processing to take place up front, when first loaded from the
database, it seems like a slippery slope, since there are also many
cases where you'd want to delay that processing. Since David's
proposal doesn't address delayed processing, we'd have to make sure to
manage expectations so people don't cry foul when it still doesn't do
what they're looking for.

I've been all for expanding how data conversion in fields is handled,
and I think SubfieldBase is a great first step for common cases. I
just wonder how many of the less-common cases we plan to address
before we start getting into support code that's bigger and harder to
maintain than custom solutions. I'm fine with the line being pushed
back somewhere so we can get more support in core, but I do want to
make sure a line gets drawn somewhere.

Given that I've had a few encounters with this type of thing before,
I'll gladly help out where I can, but I'd also want to make sure
Malcolm weighs in, since I think he was intentionally conservative in
determining just how much SubfieldBase was responsible for.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Custom Fields API Cleanup

[Marty Alchin]

On Tue, Dec 16, 2008 at 2:02 PM, David Cramer  wrote:
> Could we just use a lazy proxy for the value value? When it's executed, it
> could then just call the get_value_from_db method (or whatever).

I expect that the lazy approach is what most people would really want,
so I'd be fine with that. I didn't want to assume that's what you
wanted, though, since you may have a good reason to want your data
processed immediately upon loading, rather than on first use.

Personally, I'd recommend that if we go this route, we go with full-on
lazy loading. I'd recommend against updating ForeignKey and GIS code
to use it, though, since they currently use two different approaches.
Merging them would likely introduce some interesting gotchas for
people who are relying on the specific technique used behind the
scenes.

I'm still reserving a bit of judgment, though, until we get some code
together that proves useful in a good half-dozen different situations
(at least). The only thing worse than not implementing as much of the
feature as people want would be to implement all of it but make them
reimplement it because it's not flexible enough to suit their needs.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Signed Cookies Status

[Marty Alchin]

In light of the upcoming feature completion deadline for Django 1.1,
I'd like to take a few minutes to mention where the signed cookies
application stands right now and some of my thoughts about its
inclusion in django.contrib. The code itself is done. I made a few
minor updates to it last week to encourage people to use it as a
decorator if they don't need the full middleware, rather than
providing a separate utils package that would be too easy to
accidentally get things wrong. I think it's in as polished a state it
can be short of adding some major new feature I'm currently unaware
of.

That said, there's still one major concern: security. The whole point
is to try to guarantee that users can't tamper with their cookies and
use them on the site, so security's of primary concern. Unfortunately,
I'm not a security expert. The app was born from a comment Simon
Willison made on djangobook.com during the initial review phase,
mentioning that he uses an MD5 hash to sign cookies for OpenID
authentication without hitting the database. I basically just took
that description and made a middleware to do the job transparently.

However, Simon's been active on Twitter recently, questioning the
security of various hash digest algorithms, specifically with regard
to signing cookies. Again, I'm no security expert, but from what I've
read, MD5 is pretty much at the bottom of the barrel in terms of sheer
security, so I have definite concerns here. Simon, if you're
listening, what have you found out with regard to recommendations on
hashing algorithms?

The app can very easily be altered to use a different algorithm, but
with greater security comes greater hash lengths. And since cookies
are already limited in length by browsers, longer hashes present a
serious threat to the real-world usefulness of the app. If being
completely secure means only short values can be stored in cookies,
I'm not sure anybody would call it a worthwhile endeavor.

Perhaps most importantly, I just read over Jacob's recent post about
what it takes to be a contrib app, and I feel the signed cookies app
fails on two out of three points. It's certainly optional, which is
the only one it gets right. But even though I believe mine to be the
only Django-based signed cookies application, there are WSGI
middleware projects available as well that do the same job without
relying on Django at all, so I think it fails the "de facto standard"
test. The approach taken by the app is (I believe) a best practice,
but the app itself hasn't proven itself as the preferred way to do it.

But it's the other failure that concerns me most: it's hardly a common
pattern. Perhaps it would be more common if included in Django proper,
but it seems like nearly every time I mention the app, I get little
more response than a flurry of questions along the lines of, "but when
would you actually want to sign your cookies?" If the majority of
people can't even find reasons to sign their cookies at all, there's
little hope of it becoming a common pattern anytime soon.

So, I'm all for improving the app as much as I possibly can, and I'll
definitely continue to make it available, but unless it gets a major
boost in both security and popularity in the next few days, I don't
think it's prudent to include it in django.contrib. If others
disagree, I'll be glad to make the minor changes necessary to make it
a proper contrib app and submit it for review, but I personally don't
think it's the best idea at the moment. I'd much rather revisit it in
a future release once it's proven itself a bit better.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Unicode filenames

[Marty Alchin]

While working again with files for model validation, I realized (and
confirmed with Russ, Honza and Alex in IRC) that the tests put in as a
fix for #6009[1] don't actually prove all the behavior that ticket
refers to. They prove that Unicode filenames come through fine from
uploads, but that test doesn't actually save the file at all. The
changes I'm working on for validation cause it to save the file now,
and the second part of that ticket (dropping too many characters from
the filename) comes back to bite us.

I'm not sure if get_valid_filename() needs to be as aggressive as it
currently is, because a bit of light reading suggests that we should
be able to work with Unicode filenames, now that Django passes around
Unicode strings properly throughout. Alas, I'm not an expert in
Unicode or file systems, so I don't know if I'm overlooking some
obvious problem with opening the range up to include Unicode
characters.

I'd like to get this fixed one way or another before putting my file
patch into model validation, because it introduces a mostly unrelated
failure that I'd rather avoid. Does anybody with more experience in
this area have any thoughts on how we can go about it? For reference,
you can spot the failure by applying a simple patch to the test.[2]

-Gul

[1] http://code.djangoproject.com/ticket/6009
[2] http://media.martyalchin.com/6009-test.diff

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Unicode filenames

[Marty Alchin]

On Thu, Jan 15, 2009 at 10:33 AM, Marty Alchin  wrote:
> While working again with files for model validation, I realized (and
> confirmed with Russ, Honza and Alex in IRC) that the tests put in as a
> fix for #6009[1] don't actually prove all the behavior that ticket
> refers to. They prove that Unicode filenames come through fine from
> uploads, but that test doesn't actually save the file at all. The
> changes I'm working on for validation cause it to save the file now,
> and the second part of that ticket (dropping too many characters from
> the filename) comes back to bite us.

Correction: it was actually an error in the test that was causing the
failure; the problem just wasn't evident until the changes I put
through. I've put in ticket #10041 to address the problem with the
test.

> I'm not sure if get_valid_filename() needs to be as aggressive as it
> currently is, because a bit of light reading suggests that we should
> be able to work with Unicode filenames, now that Django passes around
> Unicode strings properly throughout. Alas, I'm not an expert in
> Unicode or file systems, so I don't know if I'm overlooking some
> obvious problem with opening the range up to include Unicode
> characters.

This is still a problem, though it's not the cause of any test
failures. The second part of #6009 is still a potential issue, and I'm
still not sure how best to address it. I'm not going to reopen that
ticket, though, nor will I open a new one until I know a bit more
about how to continue.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Model-validation: call for discussions

[Marty Alchin]

I haven't been following everything, but I do have a couple comments
to make here.

On Fri, Jan 23, 2009 at 8:04 AM, mrts  wrote:
> works both for forms and models with the proposed approach (all_values
> is model_instance.__dict__.copy() in model field validation case).

One thing to consider is that not all data is always available via the
dictionary, particularly with regard to models. Anything that uses a
descriptor (such as relations to other models) only stores a bare
minimum of information in the instance dict. The rest is retrieved
when accessed *as an attribute*, a fact which is entirely lost in your
example. I understand you're trying not to bundle it too much with the
way models work, so maybe this was intentional, but I do think it's
worth noting, just in case it's not intentional.

> Handling this specially for model instances would be really ugly:
>
>   def __call__(self, value, all_values={}, instance=None):
>   if instance is not None:
>   if hasattr(self.other_field, instance) and getattr
> (self.other_field, instance) == self.other_value and not value:
>   raise ValidationError(self.error_message, REQUIRED)
>   elif self.other_field in all_values and all_values
> [self.other_field] == self.other_value and not value:
>   raise ValidationError(self.error_message, REQUIRED)
>
> and entirely unnecessary under the all_values approach.

I agree that it's a bit ugly, but I'd hardly call it unnecessary,
given the possibly of using descriptors on models. Forms aren't
susceptible to the same problem, because their values are stored in a
true dict regardless. Offhand, I can only think of one way to get a
uniform syntax that works the way you seem to want it to, and that's
by bundling forms and models in some other objects with a unified
interface, designed solely for validation. Something like the
following (untested, coding off the top of my head):

class ValidatableModel(object):
def __init__(self, instance):
self._instance = instance
def __getitem__(self, name):
if hasattr(self._instance, name):
return getattr(self._instance, name)
raise KeyError(name)

class ValidatableForm(object):
def __init__(self, instance):
self._instance = instance
def __getitem__(self, name):
return self._instance.cleaned_data[name]

There could be more methods to be able to iterate over fields and
whatever other niceties you would want to include, but that way you
have a uniform dict interface to work with, regardless of which object
type gets passed in, and it still preserves the ability to use
descriptors on model attributes.

That said, I'm not particularly fond of that solution myself, because
I personally feel that models and forms *should* be treated a bit
differently; the field types for each are just too different in their
capabilities. I haven't been involved with this validation stuff
except for improving file support for it, but I'm not sure a unified
approach would be as big of a win as you claim.

-Gul

(and if I'm missing some obvious bit of history here, feel free to correct me)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: reason for missing readline() in uploaded files?

[Marty Alchin]

I suppose I should probably weigh in over here. For those who don't
know, I worked on much of Django's file handling, refactoring
FileField, adding storage options and standardizing the File interface
across models, forms, storage and uploads. If there are methods
missing, I'm almost certainly the man to blame.

First, I want to point out one thing: adding readline() is different
from supporting a catch-all for any methods we might have missed. The
arguments for each are quite different, and this thread has
transitioned from one to the other, making things a bit murky. With
that in mind, I'll slpit the rest of this email into two parts,
addressing each issue.

readline()

I'm not opposed to adding readline(). It's part of the standard file
protocol, and it can be added even without the underlying file needing
to support it. After all, iterating over the file already yields one
line at a time, so we could just stop after the first line is
retrieved. That said, you can also do that yourself, currently.

for line in file:
is_valid = validate_csv_format(line)
break

It's not the prettiest, but it's a reasonable workaround in the
absence of an explicit readline(). In fact, I'd much rather using a
Python approach to readline() that does something similar, rather than
simply deferring to the _file attribute. That way, we're more able to
make sure it's there for other objects that might not implement it
directly. As long as the file is iterable, and the iterator works the
way the protocol describes, a simple Python method will work just
fine.

def readline(self):
for line in self:
return line

Other methods

First, I'll be clear. Not all the missing methods were forgotten.
readline() probably was overlooked, but that's simply because nobody
had expressed a need for it at the time, but I want to make it clear
that Malcolm was right: not all methods make sense in all situations.
We had a lot of back and forth with seek() and tell(), for instance,
because our chunking mechanism for file access assumes content will
always only be read from start to finish. This caused problems with
the zipfile module, because ZIP files don't work that way.

One of the main things we're trying to keep in mind is consistency.
Much line Django's model lookup API should support all features (or as
many as humanly possible) across all available databases, I'd like to
make sure that the provided file methods work on as many file-like
objects as possible. That's made harder by the fact that there's no
finite list of supported file objects like there is for supported
databases. In the end, it may prove to be a losing battle, but I'd
rather not give up on it quite yet, and that's exactly what deferring
to the wrapped file object feels like to me.

In general, I'd like to see any new methods receive a "yes" to each of
the following questions before being implemented in core. I'm not a
core developer, so I don't have any power to enforce this, but here
they are, all the same.

1. Have people actually expressed interest in making it available?
2. Does it make sense in the context of a web application?
3. Can it be implemented in a way that's equally supported across true
files, storage-backed files and anonymous (StringIO-type) files?

As you can see, readline() passes all of these tests. Others, like
truncate(), don't. Just for reference, consider implementing
truncate() on files backed by a storage backend. In order to work
properly for things like S3, you'd need to actually read the specified
number of bytes from the beginning of the file (over the web), then
write them back out (again, over the web) in order to achieve the
documented behavior. That's hardly what I'd call "equally supported"
alongside native filesystem files which have OS functions to do that
directly.

Anyway, those are my thoughts on the subject.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: MarkupField

[Marty Alchin]

I haven't looked too much at the patch, so I can't comment much on the
implementation yet, but reading your description and the ticket, I'd
to offer some thoughts, based on some of my own thoughts I've had on
this in the past.

First, I don't think you actually addressed the question mentioned in
the ticket regarding the 3 fields. It seems the question was whether
there should be three attributes on the Python model instance,
regardless of how many columns are stored in the database. On this
note, though, I do have a thought: specify the markup type as an
argument to the MarkupField. You already do this with a
default_markup_type, but I don't see much use in having users specify
their markup type at the time they enter the text.

Essentially, it comes down to the developer choosing a markup type
during development, rather than a user choosing it in a form. Like it
or not, fewer choices generally makes for a better user experience,
especially since offering just one choice means you can supply some
help text alongside it. Trying to supply useful information for all
the various markup options, while also helping the user decide which
one is best for the (usually brief) text they want to enter ... well,
it just doesn't seem it would scale well.

Besides, specifying the markup type as an attribute of the model means
that you can do away with one of the three database fields. Plus, it
means you can specify any markup type, simply by supplying a callable
instead of a string. We make this same type of differentiation when
specifying the upload_to argument of FileFields. The implementation
becomes both simpler and more powerful, all in one swift stroke.

As for the issue Alex really did bring up in the ticket, I think
there's a question as to whether the different field types can be
contained by a single complex object, rather than individual
attributes on the model. Basically, if you had a model like this:

class InfoModel(models.Model):
title = models.CharField(max_length=255)
description = markup.MarkupField(formatter='markdown')

I would personally rather see something like this, when it comes time
to access that field's content:

>>> info = InfoModel.objects.get(id=4)
>>> info.title
u'Django'
>>> info.description

>>> info.description.raw
u'*The* web framework for perfectionists with deadlines'
>>>info.description.formatted
u'The web framework for perfectionists with deadlines'
>>> unicode(info.description)
u'The web framework for perfectionists with deadlines'

So essentially, one attribute contains both types of content, with the
default Unicode representation being the formatted output. Since
templates call __unicode__() by default, all you'd have to do is use
{{ info.description }} in a template to get it right. But you could
still use {{ info.description.raw }} to get the original content if
necessary. Or, optionally, you can pass that through a different
processor at display time if you wanted to, using {{
info.description.raw|textile }}

That's just one man's opinion, but hopefully that helps the discussion
a bit anyway.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Non-primary key AutoField

[Marty Alchin]

On Sat, Feb 28, 2009 at 11:27 PM, NewSpire  wrote:
> First, this restriction is not there in the database.  Auto-increment
> fields do not have to be primary keys.  Imposing this restriction
> could cause a problem when trying to use legacy tables in a Django
> environment.  Is there a compelling reason for Django to impose this
> restriction?

Django is optimized for the common case. Sure, it may well be that
it's easy to accommodate uncommon cases as well, and when it's
reasonable, we should definitely do so, but the reason it hasn't been
up to this point is simply that not enough people need it. I highly
doubt any of the core developers were actively seeking to restrict
people from using non-primary-key auto-incrementing fields; Django was
simply written to address the vastly most common need.

> Second, is the use case where the primary key is not an AutoField but
> the record insert order needs to be tracked.  For instance, if the
> primary key is a phone number or UUIDField it would be useful to also
> have an AutoField so that records can be sorted in the order they were
> inserted into the table.  This is a real world problem I am faced with
> right now.

Call me crazy, but why would an AutoField help you with that? Since
you later explain that the records are created in satellite systems,
then aggregated, wouldn't you still run into problems ordering by
something that's generated in isolation? It sounds like what you
really want is a DateTimeField(default=datetime.datetime.now). That
way, you get orderable fields that can be combined from any number of
sources without worrying about collisions. Plus, you get the added
bonus of having a real-world value that you can use for more than just
ordering.

Mind you, I'm not trying to say that a non-pk AutoField is a bad idea
in itself. I'm just not convinced that this use case is a very good
justification.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: (Discussion) Ticket 9986: CACHE_BACKEND argument name doesn't match code

[Marty Alchin]

On Mon, Mar 30, 2009 at 1:25 PM, Kurt Grandis  wrote:
> cull_interval or cull_period would absolutely be the most accurate
> description in terms of the algorithm's operation. My contention is that
> parameter name would be confusing to all but those who have seen the actual
> underlying culling algorithm; what if we convert to a random cache culler?.
> All the user should have to worry about is, what fraction of cache do they
> want to have culled. cull_fraction seems descriptive enough working with
> some documentation clarification.

I ran into this same problem when trying to document cull_frequency in
Pro Django. I ended up adding a separate paragraph to explain the
actual behavior, so readers could better understand what value to
supply. I've brought it up in the past on IRC, and there seemed to be
some agreement that it wasn't optimal, but no better alternatives were
found at the time.

Personally, I'd rather see an argument name that tries to tell users
what to supply, rather than solely trying to be true to the algorithm.
Obviously, the goals should coincide, but focusing on the supplied
value brings another option to my mind: cull_passes (or something
similar, like cull_sweeps). Essentially document it as being the
number of times the cull operation would need to be performed to
completely empty the cache.

That way, we could describe what the cull operation actually does,
rather than how it works. If you want it to cull a lot, set it to a
low number so that fewer passes are required to clear the whole thing.
Setting it to a higher number would require more passes to do the same
job. Does that make sense to anyone else?

Of course, I realize that now is quite the worst time to suggest a
change, so I'm not suggesting it be changed now. Changing the name
doesn't fix any bugs in the behavior, nor is it backward-compatible,
so it's well out of scope for 1.1, and possibly the rest of the 1.x
line, but I'll leave that decision up to those who are responsible for
such matters.

I'll also warn that this type of thing easily spins into a bikeshed
thread, so don't be surprised (or upset) if somebody comes along and
shuts it down to focus our efforts on more pressing matters.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Bug or feature: pre_save used on FileField/ImageField doesn't work as in Django 1.0.2

[Marty Alchin]

On Thu, Apr 2, 2009 at 11:45 AM, Karen Tracey  wrote:

> I think it's a bug, and I'm pretty sure it was introduced by r9766.  The
> setting of the actual file name that will be used (which pulls in the
> upload_to path and possibly tacks on some underscores if the uploaded name
> conflicts with already-present file(s) in the upload directory) is done by
> django.db.models.fields.files.FieldFile save().  This used to be called
> fairly early on from the django.db.models.fields.files.FileField
> save_form_data() method but r9766 moved that to much later, namely a
> newly-introduced pre_save() routine in FileField.  Since (from a brief look
> at django.db.models.base.Model save_base()) the field pre_save routine is
> called after the pre_save signal is sent, the real file name has now not yet
> been set when the pre_save signal is sent.

Ugh. Thanks, Karen, for looking into this and so many others that came
up after r9766. I've been fairly negligent on these, and it's starting
to cause some pretty big problems. r9766 may well mark the last time I
try to be clever. :(

> There are a couple of still-open tickets for other side-effects of r9766
> (#10249, #10300) but I don't believe either of the potential fixes for those
> would fix this issue, so this probably needs a ticket with the details you
> provided (an integrated testcase would be even better, though I don't know
> how much signal testing is done by the test suite).  Not sure what the right
> fix would be -- can just the setting of the name be pulled out of save() and
> done earlier by save_form_data()?  Possibly save() would have to also still
> ensure it's done, in cases where save_form_data() is not involved?  Can
> anyone with a better handle on r9766 comment on this (and #10249, #10300)?

For the record, #10044 (which was "fixed" by r9766) was entered as a
precursor to model validation. After some discussions with Honza, we
discovered that validating a model with a file would cause the file to
get saved to storage even if the model itself would never get stored
in the database. Our solution was to allow files to be assigned to
models outside the save() logic, and only store the file on disk when
the model actually hits the database.

Unfortunately, that proved problematic, because if you assigned an
UploadedFile to a model attribute, it didn't have the FieldFile
methods necessary to actually store it in the backend. I briefly
considered leaving that alone, and not relying on FieldFile when
finally saving the delayed file, but that created an inconsistent API.
After all, if you have a function that acts on models with files
attached, but you pass it a model with an *unsaved* file, it would
have a different set of methods than if the file had been saved.

Thus, r9766 includes the task of creating a new hybrid class whenever
a new file is assigned to a FileField attribute. This implementation
detail is the cause of #10249 and #10300 that you mentioned, but this
one stems from the simple fact that file saving is delayed as long as
possible. A related, thus-far-unreported (I think) issue comes up when
attempting to access width_field and height_field attributes on a
model prior to saving the new file.

I have a couple ideas to try out with regard to #10249 and #10300, so
feel free to hit me up on IRC if you'd like to discuss them, but I'm
not as certain how to approach this new one yet. There are certainly
band-aids that would allow the filename to be generated prior to
storing the file, but that wouldn't address the
width_field/height_field concern, nor any other potential future
issues that arise from delaying the file. I'd like to get to a point
where we're confident this sort of thing won't happen again, but if
the only option is to just keep putting out fires and they flare up,
it'll have to do.

As for addressing this one in particular, I think we'll have to add
something similar to the _committed flag, so we can identify whether
the filename has been processed or not. If it has, return it;
otherwise, generate it using upload_to when it's requested. The
trouble with just moving it up into save_form_data() is that if
someone assigns a file manually, save_form_data() won't get the chance
to generate a proper filename and we'll have a separate issue on our
hands. I hate having to add all these flags to determine what part of
the process we're in, but at a glance, that seems the most reasonable
approach.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Bug or feature: pre_save used on FileField/ImageField doesn't work as in Django 1.0.2

[Marty Alchin]

On Mon, Apr 6, 2009 at 1:16 PM, Karen Tracey  wrote:
> We cannot know for sure what the file name is until it is saved to disk, as
> the save operation may tack on underscores when handling race conditions.
> Thus we cannot delay file save to a field pre_save routine and report the
> guaranteed correct file name in a pre_save signal handler, since the latter
> runs before the former.

Yes, very true. I wrote my reply in a hurry to get you some more
information, and didn't really think all the way through that one.

> I feel like I'm going around in circles thinking about this one -- is there
> a way out that someone else sees that I'm blind to?

Welcome to my world! :( I spent a long time on issues like this prior
to getting the new file storage system in place at first, and
apparently skipped much of that step when doing r9766. At this point,
I think the most useful approach is to take a step back and just look
at the high-level options we have available. Trying to determine the
name in advance is a no-go, because that would just make the existing
known race conditions much more prominent. I see two options left:

1. Write something to disk (maybe the whole file, as it was before,
maybe just a placeholder, as the _save() does now) when the file is
first assigned, so that we have a full, proper filename early on.
Then, if the model doesn't get saved, somehow roll that back so we
don't leave stuff lingering on the filesystem. This is currently
something of a "then some magic happens" approach, since I'm not yet
sure if there's a reasonable, reliable way to roll back a file save.
But I'd like to keep an open mind, so there it is. This would also be
preferable if we can detect transaction rollbacks, because there is
also #6456 to consider.

2. Save the file all at once, as soon as possible, and blatantly
document this behavior when model validation goes in. Then, if people
need to validate a model that has a file on it, they have two options:
validate the model *before* saving the file to it, so the model is
known to be valid and can all be saved at once, or (if they need to
validate something in the file, such as its name or contents), add
their own code to delete the file if validation fails. Or, I suppose a
third option is to ignore the lingering files until they suck up too
much space, requiring a manual purge.

I obviously hate to go with number 2, but if we can't come up with
something solid, I think it's the better approach, at least for now.
Documenting unfortunate behavior is certainly preferable to coding
even more unfortunate behavior.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django 1.1 update

[Marty Alchin]

On Thu, May 7, 2009 at 6:43 AM, Jacob Kaplan-Moss  wrote:
> Once this is done we'll be down to blockers for 1.1; many of us at the
> sprint are focusing on these. More help will be appreciated!

I just wanted to add a note here that may have some impact on which
tickets get punted vs. fixed in 1.1. I've been looking over the many
file-related tickets that are in the 1.1 milestone, and several of
them were caused by r9766, where I introduced support for assigning
directly to file fields, deferring the actual save until later.

While I still think that's a valuable feature, and will likely be
required in order to complete Honza's model validation work for GSOC,
it's really a new feature that has so far caused far more bugs than
it's worth. I'd like to recommend that r9766 be reverted, so we can
take the time necessary to get it right without causing all the bugs.

I'll be working for the next couple days on getting a patch together
for not only reverting r9766, but also adding tests for all the
tickets that are currently failing, to make sure not only that
reverting r9766 does the trick, but also that we don't make the same
mistakes next time the feature comes up for implementation.

I'll get the patch ready, regardless, but I wanted to bring this up
now, since it will likely impact how those tickets are dealt with
during this process. I'll need a committer to agree with me before it
can be done, obviously, but I think it's the right call at this point.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django 1.1 update

[Marty Alchin]

On Thu, May 7, 2009 at 8:34 AM, Jacob Kaplan-Moss
 wrote:
> Ugh, I really hate not being able to just assign files to fields. It
> just feels hacky and wrong to call instance.file_field.save(). It'll
> also break a bunch of code folks have written over the last few
> months. I know, no backwards-compat guarantee, but that's super
> annoying if we don't have to.

Yeah, I realize that. Just yesterday, somebody told me they already
refactored some code to take the new behavior into account, so I fully
appreciate that it'd be breaking an expectation that's already been
set for a few months.

> I've reviewed most of the open issues and it looks like they all have
> patches, so can't we just fix the problems and keep the feature?

I'm a bit concerned about the likelihood of getting them all addressed
in reasonable ways in time to not severely block the 1.1 release, but
I'll set aside some time this weekend to look at it all in more
detail. Karen's already done a good bit of legwork on several of
these, so hopefully the existing patches are better in combination
than I fear.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: r9766-related issues

[Marty Alchin]

On Thu, May 7, 2009 at 11:37 AM, Karen Tracey  wrote:
> I noticed the question of what to do about the r9766-related issues came up
> in the 1.1 thread so figured, in case it's helpful, I'll lay out my
> understanding of what/where these are.

You've been doing some great working tracking these issues down, so
your understanding of their current status is immensely helpful.
Especially since I've been slacking on these for quite some time and
could use the refresher.

> So far as I know there 4 open ticks remaining related to r9766.  Three are
> regressions so I believe something really needs to be done about them before
> 1.1; one I think is just a bug in the new function.  Personally I'd rather
> not revert the new function at this point (it's been available in the
> alpha/beta releases, it would be nasty to pull it at this point).  However,
> reverting the part of r9766 that delayed saving the file to the backend
> until model save (in the case where a ModelForm is used to create a model)
> may be necessary to fix the last of the four problems, unless we want to
> document that one as a backwards-incompatible change.

To be clear, I would definitely rather not revert the change if these
can be fixed in a reasonable time frame. I just didn't want 1.1 to
block for ages while we sort this stuff out, given that all the
problems were caused by a new feature (that is, r9766 didn't actually
fix any bugs, the ticket reference was really just a formality).

> #10249: can't create consistent MRO (method resolution order) when assigning
> a File to a FileField.  This is not a regression as I don't think you could
> do this at all before r9766. I added a patch to #10249 that fixes it, but it
> needs review from someone more knowledgeable about the implications of
> messing with class bases than me, particularly since I noticed it introduced
> another problem when combined with the patch for the next problem.  There's
> a note about that in the last comment on #10249 with possible fixes for the
> subsequent problem.  It's fixable, I just don't know what the right fix is,
> and I don't know what other side-effects there might be to this patch.

Yeah, this is the big one, in my opinion, because of the potential for
side effects. r9766 went in with too little thought on the subject
(entirely my fault, not passing blame), and I don't want to make that
same mistake twice. It's not a regression in and of itself, but the
feature is broken, all the same.

> (Marty also mentioned at one point he had some other ideas on how to fix
> this one but I have no idea what they are.)

I apologize for not writing my ideas down at the time, as you probably
would've been able to help refine them by now, but here's the gist of
what I remember. Right now, FieldFile is a direct subclass of File,
overriding some methods and adding others. What I'm planning to try is
to break that stuff out into some kind of FileFieldMixin that actually
provides and overrides methods, then have FieldFile inherit from File
and that new mixin. Then, in the code introduced in r9766, it would
just add the mixin to the class, hopefully avoiding any MRO
consistency problems.

It's still a bit more complicated than that, because #10300 below
highlights the problems with overriding methods on a base class that
may have behavior you weren't expecting. I think it's a good start,
though, and it's where I'll be focusing my efforts this weekend.

> #10300: custom storage backend can't get length of content to save.  This
> one is a regression.  It's also possibly easily fixed -- it's got a patch
> that works for the reporter of the problem, but it needs review.  I also
> don't remember if I added a test, I don't think I did.

Yeah, this definitely needs review, but mostly to see if there are any
similar problems caused by any of the other overridden methods
provided by r9766. Tests are definitely essential, but this one should
be relatively easy to test.

> #10404: height_field and width_field for an ImageField don't always work.
> Specifically, they don't work if they are declared in the model before the
> associated ImageField.  This is a regression,  resulting from the 2nd part
> of r9766, the delaying of the save to storage.  Possibly fixable without
> moving the save back to where it used to be, but I haven't investigated that
> at all due to the next problem.

I remember having a small bit of discussion on this in IRC a while
back, where someone suggested getting the image's width and height
immediately upon assignment. I'm not a big fan of that kind of side
effect when simply assigning to an attribute, but it's certainly less
strain than saving the entire file on assignment, which is essential

Re: r9766-related issues

[Marty Alchin]

On Thu, May 7, 2009 at 12:05 PM, Armin Ronacher
 wrote:
> I'm working with Alex on that right now here in Prague.  We have some
> branches on github related to that.  Basically the idea is to start
> with getting rid of some of the over engineering in the abstract base
> classes and make sure the subclasses call the constructors properly.
> Once that works we should be able to fix all the "method missing on
> temporary file" tickets that are currently just worked around.

Good to not neglect those, indeed.

> The fixing of the problems caused by the changeset will be covered by that
> branch as well.

I'm a little worried about how easy you're making it sound. Either one
of us is thinking way too much on the subject or the other isn't
thinking enough. :)

> We'll report back some status when we get the
> existing testcases passing again :)

I can't be on IRC for about another 7 to 8 hours or so, when I expect
you guys will already be done for the day, so if you have any
questions about background or want to bounce ideas around, I'm
available via email, and should be able to respond fairly quickly. We
could use this thread if you want more input, but I'd rather not spam
all the devs with minute details.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: r9766-related issues

[Marty Alchin]

On Thu, May 7, 2009 at 3:43 PM, Armin Ronacher
 wrote:
> On May 7, 5:37 pm, Karen Tracey  wrote:
>> #10249: can't create consistent MRO (method resolution order) when assigning
>> a File to a FileField.
> This is fixed.

I was reading over the patch Alex mentioned in IRC (yay for
DjangoBot's logger!), and though it looks different than what I
intended, if it works, I'm happy (mostly).

>> #10300: custom storage backend can't get length of content to save.
> This *should* be fixed.  I can't test it, no access to S3.

It should be reasonably simple to set up a dummy backend that exhibits
the same behavior with size, just to test it out at least.

>> #10404: height_field and width_field for an ImageField don't always work.
> This is not really related to that and I have a hard time
> understanding that part of the ORM, but it should be fixable.

It's not explicitly related to the MRO and method stuff you and Alex
have been working on, but it's definitely related to the r9766
discussion, since it's caused by the delayed saving. I have a clear
understanding of the problem, but I don't have time to write it up at
the moment, so I'll try to catch you guys on IRC if you're still up
when I get home.

>> #10788: actual file name not known until after model save.
> That's the problem with the new implementation.  The final filename is
> decided when the file is actually saved.  The cleaner way is to use
> the assigned file and directly read from it when it's still in the
> temporary location or memory.

That only solves the problem if you're trying to access the file's
contents, which I'll admit is the majority case. Other apps may need
the filename for other reasons, though, and it was available in 1.0,
so the current behavior is backwards-incompatible. Of course, all the
uses I can think of would actually be better as post_save handlers
anyway, but the fact is that compatibility was broken, so we should
see if we can restore it.

Thanks for all the work you guys have done on this. We're not out of
the woods yet, but we're much better off than we were.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Posting to the wrong list (was: Re: Need Django Developer urgent)

[Marty Alchin]

On Fri, May 8, 2009 at 7:05 AM, Jacob Kaplan-Moss
 wrote:
> """
> Discussion group for Django developers. This group is used for
> discussion of developing Django itself, not user questions; Please use
> django-users for issues regarding using the framework, questions for
> the Django user community, outreach, etc.
> """
>
> If you've got any suggestions of ways to make that more clear -- other
> than renaming the list, which isn't going to happen -- I'd love to
> hear 'em.

I'm not sure if it'll actually avoid confusion, but could we just do
away with the first sentence entirely? After all, even before reading
the description, they know it's a Google group called
django-developers. Does the first sentence actually add anything
useful? That would allow the description to lead off with the actual
purpose of the group, so hopefully it would do some good.

As for Ned's suggestion of "core", I wonder if that would be better
suited in the description of what we develop. Maybe something like
"...discussion of developing the core Django framework..." perhaps?

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: r9766-related issues

[Marty Alchin]

On Mon, May 11, 2009 at 8:45 AM, Armin Ronacher
 wrote:
> All bugs are fixed now except for #10788.  Now the problem with
> closing that one is that this one requires a design descision.  I
> updated the ticket accordingly for jacob or anyone else to decide on
> it.  My personal opinion is that I consider it bad design for the
> application to depend on the final filename of an uploaded file on the
> filesystem.  This makes it nearly impossible to create "atomic"
> uploads for unique filenames.

I've been thinking this one over a bit, while everybody else was
cleaning up the rest of my mess (thanks, guys!), and I have to agree
with you, at least to a point. I've only been able to come up with two
general uses for having the filename available at that point, and both
of them have a better approach available.

If you're using the filename to store it somewhere else, typically for
denormalization, it'd be better to do that post-save, since then you
know the record actually got saved in the database. Otherwise, you
might be trying to access the content of the file, which would be
better using the direct file access, which can often save yourself a
round-trip to the storage backend (which can be a big win for paid
storage solutions, like S3).

I won't completely discount the possibility that there's a legitimate
use case for having the final filename available in the pre-save hook,
but I just don't see it. I'll admit that it's a change to current
behavior, but I'm not sure there's a good way to get it working
reliably. I think it's possible to do it if we register our own
pre-save hook directly when the FileField is attached to the model
class, but even then, we can't guarantee that it's the first handler
to get registered, much less should we be relying on the ordering of
signal handlers anyway.

Unless somebody sees something I don't, I'm not sure there's a good
way to get it working, nor do I see much to be gained from doing so.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: DurationField, request for help and comments

[Marty Alchin]

Okay, I'm finally back around looking into this situation, and I have
to say I'm not thrilled with the direction it's currently headed. I've
never liked the "1h 30m 10s" syntax in any situation I've ever
encountered it, and I highly doubt that it will be meaningful to much
of anyone. That's personal opinion, of course, but I just don't see it
being a very good solution. My preferred approach would be to simply
separate the days from everything else and let two fields handle it. A
day field would parse an integer, while a time field would parse
standard time syntax ("01:30:10"), which is much more commonly used
for durations.

I'll freely admit that this isn't perfectly intuitive either, but I
think it's much more natural than the current proposal. I have a
slightly modified version of my original patch that works a bit better
now, and I'm working on a JavaScript helper like the current TimeField
uses in the admin, which should help relieve the intuitiveness
problem.

More specifically, can someone explain to me just what problems are
solved by adding a custom TimeDelta class? I can only see two benefits
to it: it accepts years and months as units and it maps the new syntax
into meaningful values. I've already spoken on the syntax issue, but
I'd like to point out that accepting years and months is a very bad
idea.

Basically, there are two ways to deal with years and months, depending
on what situation you're trying to use them in. The first is to not
try to flatten it out to a specific value until applied to a reference
date. This is the approach used by calendar applications, for
instance, when you apply a yearly repeating rule with a start date or
set an alarm for one month in advance of a given event. There are
known reference dates to work with, so years and months can be
calculated accurately.

The other approach is to assign some approximate value to years and
months, which is especially useful when *not* dealing with reference
dates. For example, you might use this approach to describe a
university degree program that takes four years or a prison sentence
that lasts 6 months. When speaking in generic terms, these
approximations are appropriate, but only because they're abstract in
our minds. They're never given an absolute length of time that can be
collapsed to days, hours, minutes or seconds. That can only happen
once a true date can be assigned.

What this ticket is doing, however, is trying to formalize the second
approach by explicitly stating some approximate values for years and
months. The intent is admirable, but it's ultimately doomed. What
you're left with is a "solution" that passably works for the second
situation, but outright fails for the first. When a custom TimeDelta
with years and months is applied to a known date, the result will
nearly always be incorrect. All it really does is formalize something
that shouldn't be formalized anyway.

The third option, which is used by Python and many other computing
systems, is to take years and months out of the equation entirely.
Sure, it's a reduction in functionality, but the benefit is that what
remains *actually works*. Weeks are the largest unit of time that can
be accurately collapsed to seconds on its own and applied to date
calculations in all situations. Thus, Python's timedelta supports
weeks as the largest available unit.

I saw this issue raised on the ticket, with the response of, "I just
wanted to get the implementation in. It's easier to remove than to
add-on." Unfortunately, there have been two more patches from the same
author and the year and month support remains. I hope I've explained
the situation well enough now that we can forget about years and
months in future patches.

I'll address the question of input format once I have a new patch up
and running.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: DurationField, request for help and comments

[Marty Alchin]

On Tue, May 26, 2009 at 2:16 PM, Jerome Leclanche  wrote:
> Right now, removing months and years is a matter of removing two lines
> from the patch. I didn't want to write a patch just to get told "it's
> lacking months/years!"; I'm also in favour of dropping it and
> following Python's style; but up until now no one except yourself and
> Yuri have given feedback, I had no reason to drop it.

My only beef here is that Yuri did mention it, with reasonable
justification in the ticket, and you did acknowledge that it could be
removed, yet it remained. I can understand the tendency to support as
much as possible until someone reigns you in, but once someone does,
it's best to listen. But, I've spoken my peace and it sounds like
we're on the same page now, so we move on.

> As for the current syntax, I believe with a small syntax helper
> somewhere next to the field, this would be and feel completely
> natural. 01:30:10 is a bit meaningless if you're not working in
> seconds. One of my application uses standard milliseconds durations,
> would I have to write 01:30:10.5 to have another 500ms? Would I need
> 1-2 other fields for milli/microseconds - which would make us go back
> to the former problem ("Many input boxes is not a nice user
> experience")?

In my view, yes, 01:30:10.5 (or 01:30:10.50 if you prefer) would
be the better approach. I will, however, freely admit that my support
for that syntax is based on nothing more than personal preference and
a vague idea of what I believe to be more common. I don't have any
research to back it up, but I don't expect any of us do in this area.
A quick search reveals that ISO 8601 is the relevant specification
here, which would leave us with something along the lines of
PT01H30M10S. In addition to being not-so-user-friendly, it seems ISO
8601 doesn't provide for anything smaller than a second, so we're back
to that same problem anyway.

> That's the way I was going until someone came up with the idea of
> parsing a timestring. Looking back, this is much more elegant, as long
> as the user knows what to do. The need for a javascript helper is
> entirely dropped since it's extremely easy to input any kind of value,
> and the user doesn't have to convert any kind of value. If the user
> wants 17 weeks, they just input "17w" instead of having to calculate
> 119 days. If they want "1500 seconds", same thing; so on.

I'll admit, my preferred approach does drop support for weeks, and if
you were to put in more than 24 hours, 60 minutes or 60 seconds, you'd
get a big, fat error message instead of the implicit calculations that
timedelta does behind the scenes. I still disagree with the notion
that the current syntax is "extremely easy to input any kind of
value," but again, neither statement is based on very much, so it's
likely impossible to say which is "better".

We're basically debating between the two representations allowed by
ISO 8601, at least in spirit. It seems an international panel of
experts couldn't decide on one over the other either, so they allow
both. Unfortunately, that's not really in keeping with Python
philosophy ("There should be one-- and preferably only one --obvious
way to do it."), so we need to pick one or the other. Whichever way we
go, it seems some of us will be Dutch, some of us won't. Such is life.

At this point, I'd say we should table the discussion on input syntax
until after 1.1 is out the door, so we can get some input from people
who have more sway in these matters. And yes, I do realize that I'm
the one who brought it up. ;)

> Also remember not everyone has javascript enabled.

I've seen this argument before, and it doesn't hold water. A
JavaScript helper is just that: a helper. It's not required to use the
field, it just makes it a bit easier for those who need a little help.
Those without JavaScript would just get text boxes without the helper,
just like the existing date and time fields.

> As for a wrapper TimeDelta class, that's up to Yuri to explain - I
> don't see the problem with it personally.

I wasn't pointing fingers at anybody in particular, I just don't see
that it adds anything beyond Python's own datetime.timedelta. Once
years and months are removed, it basically just performs unit
conversion and provides an output format in line with the new input
syntax. Python's timedelta already does the unit conversion, so that's
just duplicated effort. As for the output format, I'd rather see that
handled at the widget level anyway, so any other Python code that
deals with timedeltas doesn't get a shock when the output isn't what
Python normally provides.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://g

Re: Template Caching

[Marty Alchin]

I won't speak to most of this, since Russ and Jacob have said it all,
but I do want to respond to one point.

On Thu, Aug 6, 2009 at 3:55 AM, Russell
Keith-Magee wrote:
>> By simply checking that the template has a ``render`` method in
>> ``get_template``, we get the added benefit of allowing users to write
>> loaders that return custom Template instances, or Template instances that
>> use an alternative template language like Jinja or Mako.
>
> I'd be particularly interested in hearing Jinja and Mako users weigh
> in on this particular comment. If we can improve the accommodation for
> external template engines with a relatively simple change to our
> rendering, I'm all for it.

Mike and I talked about this a bit in IRC before he submitted the
patch, because I had looked into the topic fairly thoroughly while
working my way through Pro Django. I actually wrote up a template
loader that wraps up a Jinja template in a Django candy shell, but of
course it didn't work because the engine expected templates to be
strings. Prior to engaging in a long, beastly workaround (which I
admit was funky, but it worked and I'm fairly proud of it,
regardless), I did make a small change that allowed template loaders
to return Template objects, which worked fine for both Django
templates and (wrapped) Jinja templates.

Of course, it was getting down to the wire on Django 1.0 at the time,
so I didn't submit the patch for review or anything, I just went ahead
with the workaround. So, I won't pretend to be from the Jinja or Mako
camp, but I can speak from experience that Jinja templates can be
achieved with just the small change Mike included in the larger issue
here. In fact, the interfaces are so similar, it would *almost* work
even without the extra wrapper around Jinja templates. The only real
trouble is that their contexts work differently, so it's not a perfect
match, but with the right template loader, the conversion is fairly
straightforward.

Anyway, I hope that perspective helps, but I also look forward to
hearing from people with Jinja/Mako/etc experience, to see if this
addresses any needs I may not have considered during my testing. If it
ends up working as well as it did for me last year, I think this one
feature alone will be well worth including, even if the caching stuff
has to be delayed because of concerns over template tag state.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Field.contribute_to_class and contrib fields

[Marty Alchin]

On 5/18/07, Robert Coup <[EMAIL PROTECTED]> wrote:
> I think eventually, eliminating all the get_XXX_something() methods for
> fields (perhaps with the exception of display) would be an ideal case.

Yeah, get_FIELD_display would be fine, since that's not really
specific to any field type.

> What about having a _first_set attribute on the proxy object that only
> allows laziness for the first time the value is set (ie. when it is set
> from the DB row)? Then any later set()s get live validation and coercion
> to geos objects.

Well, I did try something like that at first, but the problem is that
the proxy object is instantiated once, when assigned to the class. So
ultimately, all instances of the Model use the same instance of the
Proxy, which means that any attribute we set for one instance's field
would be set for all instances of that model. I haven't figured out a
way around that one yet.

-Marty

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Field.contribute_to_class and contrib fields

[Marty Alchin]

On 5/18/07, Robert Coup <[EMAIL PROTECTED]> wrote:
> What about having a _first_set attribute on the proxy object that only
> allows laziness for the first time the value is set (ie. when it is set
> from the DB row)? Then any later set()s get live validation and coercion
> to geos objects.

I submitted a new patch on #3982 that does just this, incorporating a
bit of your work on lazy instantiation, and making it more generic.
Basically, fields would have a simple way to create and manage lazy
attributes. I've already adapted DurationField to use this new
technique (#2443), and I've started work on getting FileField and
ImageField to use it as well. It'll be a bit of time before I can say
whether or not those will work properly, but it's looking promising.

The generic system I put in on #3982 allows fields to specify a class
that would be instantiated, as well as a separate function that would
handle the instantiation, and a function to be called after the object
is instantiated. The separate "creation" function is necessary in
order to have a helper funciton perform some extra trickery, but still
return an instance of the provided class. DurationField uses this
helper function to pass the value as seconds to timedelta, since
seconds are the second positional argument, not the first.

For your GeometryField, both the creation function and the init
function would be specified, since you need special handling for each
case. From what I read in your most recent patch, here's the basics of
how it could work for you.

class GeometryField(Field):
...
def contribute_to_class(self, cls, name):
super(GeometryField, self).contribute_to_class(cls, name)
setattr(cls, name, self.lazy_attribute(GEOSGeometry,
self.create, self.set_srid)

def create(self, value):
try:
return GEOSGeometry(value, "hex")
except GEOSException:
return GEOSGeometry(value, "wkt")

def set_srid(self, geom):
if geom.srid is None and self.srid is not None:
geom.set_srid(self.srid)

As you can see, it would greatly simplify the code necessary to
implement lazy instantiation of your field attributes.

I'm not tied to the self.lazy_attribute(cls, ...) syntax. It would be
just as easy to use AttributeProxy(self, cls, ...), it would just mean
that modules that will be using it would have to import AttributeProxy
as well as Field. Let the discussion ensue.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

django-values -> django-policy?

[Marty Alchin]

Django-values has had a long and rocky road thus far, with its name
not being the least of its problems. I finally put it to the masses in
a recent blog post, and I got back the simplest word I'm amazed I
didn't think of: policy. Organizations are very familiar with the idea
of policies and procedures, and I think this encapsulates that fairly
well. Procedures are what would be done in various situations (Python
code), while policies are the rules that define when and how those
procedures should be carried out.

In short, I vote that it be changed to django-policy. I can't change
the URL in Google Code without starting a whole new project (and since
I already have issues in place, I'd rather not do that), but I can
change all the documentation and add a note explaining the
discrepancy.

For the most part, I just wanted to put that to the general population
here to see if policy would be a useful enough name to use for this
app. Unless there are other objections, I figure the rest of the names
can remain the same, using policy.Options and policy.*Value for the
API.

I don't have a new version of it checked in yet, but I've been doing a
lot of work on it, and it should be very near completion later this
week. This next update will also include documentation, which is
another reason I'd like to make sure the name will stick before I go
too far with all of it.

Thoughts?

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

newforms-admin "plugins"

[Marty Alchin]

In working with my recent contrib saga, I've been looking a bit into
newforms-admin, since it's going to be merged someday, and it looks
like it would give me some better options. However, one thing I was
hoping for doesn't seem to be there.

For my policy work, I'd like to be able to add in a separate URL
handler within each model's registration, so that something like
/admin/app/policy/ and /admin/app/model/policy/ would give editors
customized for that location. However, I'm not seeing a way to do
anything like that, as the URL components are all hard-coded in an
if/elif block.

Has there been any talk of making some kind of plugin system for these
URLs, so that contrib apps can add their own functionality to existing
admin structures? Obviously, this would be a fairly considerable
change, and would also include a way to provide links to these new
interfaces from within existing admin templates. But I think it would
be useful, at least for my project.

Thoughts?

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: newforms-admin "plugins"

[Marty Alchin]

On 5/28/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> Aren't you asking the wrong question here? The real issue seems to be
> "how is admin going to handle arbitrary Field classes that it doesn't
> otherwise know about?"

Well, I'm not worried about Field classes. DurationField, for
instance, works just fine in newforms-admin without any modifications.

> For particular models, you can easily do whatever you like as far as
> admin presentation goes with the existing code. You override methods
> like fieldsets(), fieldsets_add(), fieldsets_change() and the *_view()
> (add_view, change_view, etc) methods to do whatever you wish. They are
> all in the ModelAdmin class and look pretty easy to replace with
> whatever behaviour you would like. So the framework for the ultimate
> presentation of the information is already there.

That's true when you have control over the model being represented,
yes. For django-values, however, it can be placed on any model in any
project, without django-values having any control over where or how it
gets used.

> Isn't the difficulty something like django-values faces is how is the
> initial link to the admin page for an options class going to work in the
> first place?

I'm not sure exactly what you're asking here, but I think the answer
is "yes". If you're talking about the HTML link to the value editor,
or the more symbolic "link" of attaching the editor to the ModelAdmin
class, both are what I'm wondering about. If, however, you're
referring to something else entirely, I guess I don't follow.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: django-values -> django-policy?

[Marty Alchin]

On 5/28/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> My understanding of the module is that it doesn't enforce rules and
> procedures and that is certainly what I think of policy as being,too.

Your understanding is correct. It doesn't do any enforcement of any
rules, but simply provides values that can be used to do so within
your own code. I admit that the term "policy" does seem a little
far-reaching, but it seems (to me) more specific than anything I'm
come up with so far.

> I'm not in love with that name. Something based around "values" or
> "parameters" is going to be more self-descriptive, I feel.

Yeah, things like "values", "paramters", "options", etc. are
definitely more self-limiting, preventing it from overstepping its
functional boundaries.

I guess the real problem here is the trade-off between accuracy and
informativeness. There's a valid argument against "values" because it
doesn't immediately describe what it is or how it's useful, while you
also raise a valid argument against something like "policy" because it
overstates what the app can actually deliver.

Maybe a combination of the two would be a bit better. "policyvalues"?
Indicating its usefulness in the scope of policies while still being
clear that its features only extend to the values.

I dunno, I'm stumped.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: newforms-admin "plugins"

[Marty Alchin]

On 5/28/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> It seems that "extra" things on a model in whatever form they may take
> are either instance-specific, in which case they are Field sub-classes
> -- for database-backed data -- or methods, or they are model-specific
> (e.g. django-values's Option() class), in which case, editing them via
> the per-instance page is illogical, because the results apply to all
> instances.

I see what you're saying now. Admittedly, I hadn't done a lot of
research into newforms-admin yet, but you're right that it seems most
appropriate to make use of ModelAdmin (or a subclass or a replacement
thereof) as much as possible, I just didn't know how it would be tied
in to the existing system. Right now, it seems that newforms-admin is
very model-centric (and with good reason), which does make it a bit of
a challenge to add in functionality that's not tied to a particular
model.

As it stands right now, I can make a ModelAdmin subclass that would
allow editing of these values on a per-model basis, but as far as I
can tell, it would require users to go into django-values, then pick
out a model whose values need editing. I'd just rather be able to tie
into the per-model perspective somehow. Something like adding a
"values" link next to "add" and "change".

All in all, this isn't really an urgent need, since django-values
won't really need it until newforms-admin is ready to be merged into
trunk. I'm just trying to get the ball rolling so that once I get to
that point, it's a bit more finalized. I'm probably not the best
person to ask for ideas on how to get it to work though,
unfortunately.

-Gul

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---