Re: Proposal: use SQLAlchemy Core for query generation

[Ivan]

See also syntax evolution of peewee ORM, which initially was Django's 
lightweight clone. 
http://peewee.readthedocs.org/en/latest/peewee/upgrading.html#goals-for-the-new-api

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/_uk05lcy7OoJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: use SQLAlchemy Core for query generation

[Ivan]

Hi all,
I've viewed a few interesting solutions.


   1. Allows to use both, Django-ORM + SQLAlchemy. Single connection. 
   SQLAlchemy's models are generated automatically form Django's models. 
   https://github.com/Deepwalker/aldjemy
   2. Uses only Django's ORM, but allows to use 4 SQLBuilders together 
   https://bitbucket.org/evotech/sqlbuilder :
  1. Dajngo QuerySet
  2. Lightweight "smartsql" SQLBuilder from this library.
  3. SQLBuilder from SQLObject.
  4. SQLBuilder from SGQLAlchemy.
   
Items 3 and 4 are not as well supported as item 2.

So, It is not a problem to use Django with other SQL builders. Django 
QuerySet syntax is really very simple, and convenient for > 85% SQL 
queries. For others, can be used external SQLBuilders.

Sorry for my English.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/WYkvuZCL2jMJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: use SQLAlchemy Core for query generation

[Ivan]

вторник, 4 декабря 2012 г., 0:48:04 UTC+2 пользователь Ivan написал:
>
>
>1. Allows to use both, Django-ORM + SQLAlchemy. Single connection. 
>SQLAlchemy's models are generated automatically form Django's models. 
>https://github.com/Deepwalker/aldjemy
>
>
Sorry for bad link. This is the correct link: 
https://github.com/Deepwalker/aldjemy 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/L0B7gjhBTc8J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Is file based cache is safe for concurrent process?

[Ivan]

Hi all.
Can anyone tell, does django.cache locks file for writing by concurrent 
process?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Is file based cache is safe for concurrent process?

[Ivan]

Yes, you are right. But I asked for another reason.
For example, I see, that werkzeug's cache write cache trough temporary file 
https://github.com/mitsuhiko/werkzeug/blob/master/werkzeug/contrib/cache.py#L660
But Django cache do not use neither, tmp file or file locking.
So, multiple processes can write a file simultaneously. Also, other 
processes can obtain incomplete data.

I think it's a bug, so I asked a question in this group (django-developers).



четверг, 28 марта 2013 г., 0:55:20 UTC+2 пользователь Russell Keith-Magee 
написал:
>
>
> You've already asked this on django-users. Django-developers is for 
> discussing the development of Django itself -- it shouldn't be used as 
> "second tier" support if you don't get the answer you want on django-users.
>
> Yours,
> Russ Magee %-)
>
> On Wed, Mar 27, 2013 at 8:16 PM, Ivan >wrote:
>
>> Hi all.
>> Can anyone tell, does django.cache locks file for writing by concurrent 
>> process? 
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to 
>> django-d...@googlegroups.com
>> .
>> Visit this group at 
>> http://groups.google.com/group/django-developers?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>  
>>  
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Is file based cache is safe for concurrent process?

[Ivan]

> As I said in my response on django-users, the file based...

-- Oh, I didn't see it yet. Thanks.

> if you want to try your hand at a patch, this should be a relatively 
simple patch to create (although tests could be a little bit complex).

-- yes, it's a good idea, but i need first to study the Contributing guide 
https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/

Thanks for answer.



четверг, 28 марта 2013 г., 2:23:52 UTC+2 пользователь Russell Keith-Magee 
написал:
>
>
> On Thu, Mar 28, 2013 at 7:13 AM, Ivan >wrote:
>
>>
>> Yes, you are right. But I asked for another reason.
>> For example, I see, that werkzeug's cache write cache trough temporary 
>> file 
>> https://github.com/mitsuhiko/werkzeug/blob/master/werkzeug/contrib/cache.py#L660
>> But Django cache do not use neither, tmp file or file locking.
>> So, multiple processes can write a file simultaneously. Also, other 
>> processes can obtain incomplete data.
>>
>> I think it's a bug, so I asked a question in this group 
>> (django-developers).
>>
>
> Ok - *that* is a different matter. For future reference, if you're 
> suggesting that something might be a bug, it's helpful to make that clear 
> in your question. 
>
> As I said in my response on django-users, the file based cache is 
> primarily there as a proof of concept that the caching API can be targeted 
> at multiple backends. However, if someone were to propose a patch to add 
> file locking or temp file write-throughs, I don't imagine it would be 
> turned down.
>
> So - feel free to open a ticket for this; and if you want to try your hand 
> at a patch, this should be a relatively simple patch to create (although 
> tests could be a little bit complex). As a helper, the 
> django.core.files.locks module already contains some OS-abstracted code for 
> file locking.
>
> Yours,
> Russ Magee %-)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

ATOMIC_REQUESTS, cache invalidation and concurrent process

[Ivan]

With ATOMIC_REQUESTS, after cache invalidation, the concurrent process can 
re-create cache old data, between deleting of cache and commit.

Perhaps this problem should be solved by repeat cache deleting after commit.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

pychecker catches IndexError exception while importing /django/db/models/base.py

[ivan]

Hi all,

I'm attempting to run pychecker on my django code (an application's
models.py file) and get the following output:

$ pychecker models.py
Processing models...
 Caught exception importing module models:
   File "/var/lib/python-support/python2.5/pychecker/checker.py",
line 619, in setupMainCode()
   File "models.py", line 538, in ()
   File "/usr/lib/python2.5/site-packages/django/db/models/base.py",
line 51, in __new__()
 IndexError: list index out of range

Warnings...

models:1: NOT PROCESSED UNABLE TO IMPORT

My django version is '1.0.2 final'.

The IndexError happens on the final line of code extract below:

# Figure out the app_label by looking one level up.
# For 'django.contrib.sites.models', this would be 'sites'.
model_module = sys.modules[new_class.__module__]
kwargs = {"app_label": model_module.__name__.split('.')[-2]}

I added a 'print model_module.__name__' above this line and I get the
following output before the exception:
django.contrib.contenttypes.models
django.contrib.auth.models
django.contrib.auth.models
django.contrib.auth.models
django.contrib.auth.models
models

Not sure if the last models that triggers the exception is the models
file I ran pychecker on or another file? I have all my django env
variables set up correctly and can do runserver, dbshell ,etc.

Any ideas as to why pychecker on my models file would throw this
exception in the base django models file?


thanks,
ivan.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: pychecker catches IndexError exception while importing /django/db/models/base.py

[ivan]

Thanks for the suggestion. My app is named 'control' and my django
project name is 'geni'. Running pychecker on the app dir raises a
problem in pychecker itself -- it seemed to be that it was unable to
open one of the sources files:

$ pychecker control/
Processing control/...
  Caught exception importing module control/:
File "/var/lib/python-support/python2.5/pychecker/checker.py",
line 620, in setupMainCode()
  self._setupMainCode(file, filename, module)
File "/var/lib/python-support/python2.5/pychecker/checker.py",
line 625, in _setupMainCode()
  self.main_code = function.create_from_file(file, filename,
module)
File "/var/lib/python-support/python2.5/pychecker/function.py",
line 128, in create_from_file()
  file.seek(0)
  AttributeError: 'NoneType' object has no attribute 'seek'

However, digging more into the pychecker code I found out that
pychecker was attempting to open the 'control' dir as a file. Then I
checked my pychecker version and discovered that I'm running version
0.8.17 while the latest version is 0.8.18. I upgraded to 0.8.18 and
ran again:

$ pychecker control
Processing module control (control)...

Warnings...

None

However, this last run leaves me skeptical. It does not seem that it
actually imported any modules and I'm getting no prints of
model_module.__name__ so its not importing any of the django models.
Finally I tried running:

$ pychecker control/*py
Processing module admin (control/admin.py)...
model_module.__name__ :  django.contrib.contenttypes.models
model_module.__name__ :  django.contrib.auth.models
model_module.__name__ :  django.contrib.auth.models
model_module.__name__ :  django.contrib.auth.models
model_module.__name__ :  django.contrib.auth.models
model_module.__name__ :  geni.control.models
model_module.__name__ :  geni.control.models
model_module.__name__ :  geni.control.models
model_module.__name__ :  geni.control.models
model_module.__name__ :  geni.control.models
model_module.__name__ :  geni.control.models
model_module.__name__ :  django.contrib.sessions.models
model_module.__name__ :  django.contrib.sites.models
model_module.__name__ :  django.contrib.admin.models
Processing module forms (control/forms.py)...
Processing module __init__ (control/__init__.py)...
Processing module models (control/models.py)...
model_module.__name__ :  models
  Caught exception importing module models:
File "/usr/lib/python2.5/site-packages/pychecker/checker.py", line
698, in setupMainCode()
File "models.py", line 539, in ()
File "/usr/lib/python2.5/site-packages/django/db/models/base.py",
line 52, in __new__()
  IndexError: list index out of range
Processing module urls (control/urls.py)...
  Caught exception importing module urls:
File "/usr/lib/python2.5/site-packages/pychecker/checker.py", line
698, in setupMainCode()
File "control/urls.py", line 39, in ()
File "/usr/lib/python2.5/site-packages/django/contrib/admin/
__init__.py", line 40, in autodiscover()
File "control/admin.py", line 108, in ()
File "/usr/lib/python2.5/site-packages/django/contrib/admin/
sites.py", line 64, in register()
  AlreadyRegistered: The model Donation is already registered
Processing module views (control/views.py)...


So its hiccuping on the same issue when importing models, and has a
problem in re-registering a specific model class used in the control
app called 'Donation'. For the last issue, I'm importing Donation in
admin.py using: from geni.control.models import Donation, and I don't
have Donation registered anywhere else. So I'm not sure how to fix
this problem either, except that I don't really care much about
debugging the admin portion of the site so I might just comment out
the register calls.

The original problem remains though.

thanks,
ivan.

On Jan 23, 4:02 am, zvoase  wrote:
> I think I might be able to spot where the problem is. Try running
> pychecker on your app folder, rather than models.py.
>
> Let's assume your app is called myapp, so you have a directory
> structure something like this:
>
> myapp/
> -- __init__.py
> -- models.py
> -- views.py
> -- tests.py
> (et cetera)
>
> If pychecker were doing 'import myapp', then the models module would
> be accessible from 'myapp.models'. Therefore, models_module's __name__
> attribute would be 'myapp.models', and the code would run fine;
> 'myapp.models'.split('.') would return ['myapp', 'models'], and the
> [-2] index of that list would be 'myapp'.
>
> However, because you're running pychecker directly on the models.py
> file, its __name__ attribute is just 'models', which raises an error
> in the Django code

Re: Shouldn't custom fields in ModelForms pick up model Field options by default?

[Ivan Sagalaev]

Malcolm Tredinnick wrote:
> If all you want to do is change the widget, you can do that in the
> form's __init__ method by updating self.fields["name"].widget, for
> example.

There's also a ticket waiting for check-in implementing this common 
usecase: http://code.djangoproject.com/ticket/9223

P.S. Sorry for the shameless plug :-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Logging instead of connection.queries

[Ivan Sagalaev]

Hello!

I'd like to revive the discussion about handling debug SQL log since I 
don't feel we've come to any conclusion. To summarize:

- Ticket [5415] deals with implementing signals upon SQL queries. Nobody 
had any objections to it and it helps in solving my original problem of 
logging.

- Malcolm has a patch that limits connection.queries to some finite size 
which will solve perceived "memory leak" problem for newbies. After 
additional thinking I decided to refrain my proposal of disabling 
populating connection.queries by default since it wouldn't pose 
immediate problems in this case.

If I haven't missed anything are we agree then that those two changes 
should be in 1.2 on principal?

If yes, I have another little thing to discuss about ticket 5415 
implementation. It introduces another cursor wrapper (apart from 
CursorDebugWrapper) that actually sends signals. I think we could just 
put this code into the Cursor itself and get rid of the wrapper. It 
makes sense to have signals for both debug and production modes and the 
common knowledge is that signals are cheap enough. Right?

[5415]: http://code.djangoproject.com/ticket/5415

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Shouldn't custom fields in ModelForms pick up model Field options by default?

[Ivan Sagalaev]

Kevin Henry wrote:
>> There's also a ticket waiting for check-in implementing this common
>> usecase:http://code.djangoproject.com/ticket/9223
> 
> Thanks, the accompanying discussion is very interesting and on point.
> But as someone there points out, why stop at widgets?

Because customizing widgets is a common request, unlike other attributes.

I believe that the consistency you're talking about is not very useful 
here. Speaking from my experience overwriting anything other than a 
widget is needed rarely and is easily accomplished with a custom 2-line 
helper function.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Replacing get_absolute_url, I am against it

[Ivan Sagalaev]

drozzy wrote:
> If it is going to be done I propose adding another function, and thus
> have two possible functions on the model:
> get_relative_url
> get_absolute_url

get_relative_url is rather misleading. The proper meaning of "relative" 
is a url relative to current url: "./some/hierarchy", "some/", ".". Put 
simply it's a path that doesn't start with a slash. The problem is that 
on the model layer no notion of a current url exists so this makes 
get_relative_url even more weird.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: media -> admin_media Prefix Change

[Ivan Sagalaev]

drozzy wrote:
> Yes, I Do, How about that he never replied to the Collin Grady's
> commend about HOW it would break the code.
> Seems like the ticket was never discussed but monopolized by a little
> hush-hush discussion...

Though I don't support this attitude I tend to agree on principle with 
drozzy and various people in the bug comments. Having a nice default is 
good and backwards compatible patch is available. Why not do it then?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Replacing get_absolute_url, I am against it

[Ivan Sagalaev]

Jacob Kaplan-Moss wrote:
> * The fact that the returned URL is a string often means I end up
> doing something like `obj.get_absolute_url().split('/')`. Objects are
> better.

I kind of disagree with the last sentence here. An object in place of a 
well-known native type (here, str) adds another abstraction to grok for 
a newbie. It's like when you read WinAPI docs and see a function 
accepting structs of structs of structs and changing some memory buffer 
but you only have a string and want to get an int. I mean it boils down 
to it anyway, just not obvious from the start...

In the rare cases when I do need to split a URL into parts I just use 
urlsplit:

 schema, domain, path, query, fragment = urlsplit(obj.url())

I believe it's a well-known pattern for an experienced pythoneer.

So I'm against of introducing a new Django-specific way of doing 
urlsplit that has chances to confuse both newbies and gurus for 
different reasons.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Replacing get_absolute_url, I am against it

[Ivan Sagalaev]

Ivan Sagalaev wrote:
> In the rare cases when I do need to split a URL into parts I just use 
> urlsplit:
> 
>  schema, domain, path, query, fragment = urlsplit(obj.url())

Oh... And for template authors we could just make 5 filters returning 
those parts:

 {{ obj.url|domain }}

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: media -> admin_media Prefix Change

[Ivan Sagalaev]

drozzy wrote:
> Sorry about the attitude, sometimes I think a bit too much of myself.
> When do you think will get implemented?

I'm not the one who decides :-). I've just raised another voice claiming 
that that decision may be worth reconsidering.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Replacing get_absolute_url, I am against it

[Ivan Sagalaev]

Jacob Kaplan-Moss wrote:
>>>  schema, domain, path, query, fragment = urlsplit(obj.url())
> 
> That's not in any way intitutive for a new user in the way that
> `obj.url().schema` is.

After thinking of it a bit more I agree about new users. My main concern 
though was about not using an established pattern.

>> Oh... And for template authors we could just make 5 filters returning
>> those parts:
>>
>> {{ obj.url|domain }}
> 
> So we need *five* new built-in filters instead of `{{ obj.url.domain
> }}`? Why? What if I want to access the username or password? Do we
> "just" add two more filters?

Looks like I shouldn't have posted that follow-up :-). It was only an 
afterthought. Actually I think we don't need even those 5 filters 
because there are no clear use cases for all of them. If we need schema- 
and host-relative URLs then we need exactly two filters:

 {{ obj.url|schema-relative }} (or better "protocol-relative")
 {{ obj.url|host-relative }}

I think it's better than, say:

 //{{ obj.url.domain }}{{ obj.url.path }}?{{ obj.url.query }}

The filter is also useful for {% url %}-generated URLs using {% filter 
%}. Which I believe is not doable with url-as-an-object thing.

Thoughts?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Replacing get_absolute_url, I am against it

[Ivan Sagalaev]

Zachary Voase wrote:
> If you take a look, you’ll notice that URLObject, being a subclass of  
> unicode, can be used *directly* within the template and it'll render  
> out to the URL without any magic.

Beware! Using the word "magic" too loosely may infuriate certain core 
devs and they will start hunting you all over the globe, or worse, 
strike you out of a Christmas present list for this year!

Honestly, having an object that looks like unicode but at the same time 
able to be /-ed or |-ed with another unicode in my dictionary *is* 
called "magic".

I still fail to see why this clever micro-DSL is needed when all this 
can be done with normal Python and be more clear.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Replacing get_absolute_url, I am against it

[Ivan Sagalaev]

James Bennett wrote:
> Except I can't help thinking this is an awfully arbitrary distinction
> to draw. In effect you're saying that nearly every question about an
> object should be answerable by interrogating it directly, *except* for
> "what's a URL I can use for you?"

May be I can explain this distinction with an example.

We once had two different web sites about "events". They both had a 
"core" set of models but each one had their own set of views & urls. So 
for a core Event model a question "what's your URL" just didn't make 
sense. It had two different URLs depending on the project it was 
imported in.

That said, I do see a value in having a default URL (and a default 
__unicode__ for that matter) simply because it's useful in many (or even 
majority?) of real-world cases.

So Yuri, Patrick, this method is optional so there's not technical need 
to deprecate it. This discussion really is about how it should be done, 
not if.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Ivan Sagalaev]

Hi Simon,

Simon Willison wrote:
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.

In a recent thread[1] on a similar topic Russel has also emphasized that 
we should improve documentation about doing logging.

> 3. Logging in Django core rather than a 3rd party app will encourage
> reusable applications to log things in a predictable way, standard
> way.

Talking about predictable and standard way I want to be sure that we 
don't break existing logging.

I.e. we at Yandex now have many reusable Django apps that don't setup 
loggers themselves but just log things into named loggers and expect 
them to be setup by a project that uses them.

What I gather from your proposal is that you want the same model ("an 
app logs, a project setups") plus a nice declarative syntax in 
settings.py instead of boring creation of handlers, formatters and 
loggers. Right?

> - We could replace (or complement) django.connection.queries with a
> log of executed SQL. This would make the answer to the common question
> "how do I see what SQL is being executed" much more obvious.

In the thread that I was referring to[1] we kind of agreed on using a 
signal there. Then hooking a logger onto the signal is simple.

> 5. People might leave logging on, then find their server disk has
> filled up with log files and caused their site to crash.

We had this problem with standard logging. Then we switched to a 
RotatingFileHandler which wasn't very good however because its behavior 
is simplistic and is not controllable by admins with an interface that 
they know, namely logrotate. Setting up logrotate also wasn't without 
problems. When it rotates a file it should let an app know about it and 
it uses SIG_HUP for that. However this instantly terminates Django's 
flup-based FastCGI server which we use.

Now we've settled on a WatchedFileHandler ported from Python 2.6 logging 
module. It watches for file descriptor change and doesn't require 
SIG_HUP to pick up a new file. May be we should port it to Django and 
use it as a default handler for logging to file system.

> 6. Logging levels are confusing - what exactly is the difference
> between warn, info, error, debug, critical and fatal? We would need to
> document this and make decisions on which ones get used for what
> within the framework.

May be we can just leave it for users to decide. It depends so much on 
how much an app actually wants from logging.

The only standard thing I can think of is to have DEBUG = True imply 
level = logging.DEBUG (which includes everything more sever). DEBUG = 
False will imply logging.INFO then. What do you think?

> # If you want custom log handlers - not sure how these would interact
> with
> # channels and log levels yet
> LOG_HANDLERS = (
> 'django.core.log.handlers.LogToDatabase',
> 'django.core.log.handlers.LogToEmail',
> )

This is a hard problem really. Most handlers require different set of 
arguments. File names, email credentials, system idents for SysLog etc. 
Also there should be different formatters. For example there's no point 
to waste space in a syslog message on a timestamp since syslog tracks it 
itself...

[1]: 
http://groups.google.com/group/django-developers/browse_frm/thread/9d0992e800cf7d68#

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

#9886: HttpRequest refactoring

[Ivan Sagalaev]

Hello everyone!

Ticket [9886] is one of the things that I've been holding for 1.2 time 
frame to ask for some discussion since Jacob marked it DDN.

This has started as a feature allowing to .read() directly from a 
request object. However this has lead to a rather nice refactoring 
removing a lot of code duplication in wsgi and modpython handlers.

This comment[1] in the ticket is a summary of what had changed. Thanks 
for looking!

[9886]: http://code.djangoproject.com/ticket/9886
[1]: http://code.djangoproject.com/ticket/9886#comment:1

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #9886: HttpRequest refactoring

[Ivan Sagalaev]

Ivan Sagalaev wrote:
> This comment[1] in the ticket is a summary of what had changed. Thanks 
> for looking!

Forgot to add... There's no docs & tests changes in the patch yet. I'll 
add them after the decision if this looks good at all.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #9886: HttpRequest refactoring

[Ivan Sagalaev]

Jacob Kaplan-Moss wrote:
> On Thu, Sep 17, 2009 at 10:09 AM, Ivan Sagalaev
>  wrote:
>> Ticket [9886] is one of the things that I've been holding for 1.2 time
>> frame to ask for some discussion since Jacob marked it DDN.
> 
> At first glance, you need docs and tests before you get a second glance :)

I believe my follow up about this that I've posted a bit erlier will 
eventually get to Google Groups :-).

I'm going to make them of course. Now I just wanted a general answer 
along the lines "okay this looks sane" or "no we don't need it".

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Ivan Sagalaev]

Vinay Sajip wrote:
> Actually you don't need much in settings.py, and Django doesn't need
> to grow any code of its own to "wrap" logging. You can either
> configure logging programmatically (for which I use basicConfig, in
> simple setups) or using a configuration file (ConfigParser-based,
> fully documented in Python docs) for more complex setups.

Thanks! Didn't know that. However see my further comment.

>> Now we've settled on a WatchedFileHandler ported from Python 2.6 logging
>> module. It watches for file descriptor change and doesn't require
>> SIG_HUP to pick up a new file. May be we should port it to Django and
>> use it as a default handler for logging to file system.
> 
> Why "port it to Django"? Do you mean, copy it into Django? I'm not
> sure it should be the default - not everybody uses logrotate. I'd
> leave this sort of decision for code in settings.py.

Using WatchedFileHandler is a safe default because it works as 
FileHandler, just doesn't break with logrotate. I don't know of any 
disadvantages of WatchedFileHandler before the old FileHandler. So I 
don't think there's much value in giving people this choice in settings 
because non-default behavior will be rare (and still possible anyway).

One of the reasons why I propose Django's own settings structure for 
logging is because we can choose better defaults for logging and have 
more compact syntax for them. Standard Python logging configuration has 
a noticable gap between very simplistic basicConfig which configures 
only a root channel and a verbose imperative definition of handler 
objects, formatter objects and logger objects. I've found that my usage 
of logging inevitably falls in between: I often need a few logging 
channels but I almost never, say, reuse handler objects between them.

Here's a variant of a simple config that I had in mind lately:

LOGGING = {
 'errors': {
 'handler': 'django.logging.FileLogger', # WatchedFileLogger copy
 'filename': '...',
 'level': 'debug',
 },
 'maintenance': {
 'handler': 'logging.handlers.HTTPHandler',
 'host': '...',
 'url': '',
 'format': ''
 },
}

Top-level keys are logger names. Values are dicts describing handlers. 
These dicts have several keys that Django knows about:

- 'handler': a handler class. It's imported like any other stringified 
classes in settings

- 'level': a level keyword that is translated into logging.* constants. 
This is done to not make users import logging by hand.

- 'format': a format string for the logging.Formatter object. We can 
have a more sensible default for this than the one in Python logging. Or 
not :-)

These keys are pop'd out of the dict and the rest is used as **kwargs to 
the handler class instantiation.

Django's default setup may look like this:

LOGGING = {
 '': {'handler': 'logging.StreamHandler'}
}

This has an advantage of always configuring a root logger to avoid an 
infamous warning from Python logging when the logger doesn't have any 
handlers defined. Users wanting to configure all the logging themselves 
may null-out this using `LOGGING = {}`.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Ivan Sagalaev]

Ivan Sagalaev wrote:
> Standard Python logging configuration has 
> a noticable gap between very simplistic basicConfig which configures 
> only a root channel and a verbose imperative definition of handler 
> objects, formatter objects and logger objects.

Forgot one thing. As it stands now Django has this "historic" behavior 
when it imports and executes settings module twice. This results in 
breakage when you setup loggers and handlers by hand. We now circumvent 
this by having a helper function that memoizes configured loggers and 
call it from settings.py. Having a declarative config we can hide this 
inside of Django and not scare users.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Ivan Sagalaev]

Sorry for taking a bit long to respond. Looks like we aren't disagree 
much anyway but I had to get some thoughts off my chest :-).

Vinay Sajip wrote:
> It's similar to Django's support for, say, simplejson. I think it's
> reasonable for Django to alias WatchedFileHandler so that it's
> available either bound to logging's own implementation (in
> sufficiently recent versions of Python) or else a copy in Django's own
> code. Then people can use it if they want to, even in older Python
> versions.

Agreed.

> I have no big problem with a configuration scheme such as you suggest
> - if it's felt that a lot of Django users are not Python-savvy enough
> and need some hand-holding

Uhm... No this is not about hand-holding or something like that. I 
propose a format that is shorter and more convenient than the 
ConfigParser's. This is the point.

> My reservation with Django's own take on it is simply
> that it goes against TOOWTDI and the Zen of Python, a little at least.

Yes, this is the bad part about it. However the Zen of Python is not a 
dogma and its application is subjective. I think my proposal still 
stands and the decision is left for core devs. As for me I'll most 
probably play with the implementation of parsing of this format for our 
own projects anyway. I'll file it as a ticket if I would still like it 
by that time :-).

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: built-in logging with django.core.log

[Ivan Sagalaev]

Vinay Sajip wrote:
> We should provide a configure_logging(dict) utility function which 
> takes a dict in an agreed format (such as Ivan's suggestion). Then in
> settings.py users can invoke configure_logging(LOGGING) after
> defining the configuration right there, in settings.py itself, or
> obtaining the dict from any other source. This avoids any "magic" (by
> LOGGING being automagically processed) and allows advanced users to
> configure "by hand" if they so wish.

I see one problem with explicitly calling a function from settings.py. 
This will execute instantiation of handlers and all the user staff very 
early when settings aren't fully configured yet. This prevents a user to 
have a handler that will use Django models for example. In general 
settings should be as dumb as possible for easy bootstrapping. This is 
why all those middleware/context processors/backends etc are strings and 
not actual callables.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Best place for code that processes stuff from settings.py once (for logging implementation)

[Ivan Sagalaev]

Benjamin Slavin wrote:
> * Logging may need to come even earlier.  If you truly want to log
> everything, you'll want to run that code first.

Is it really the case that we want to log everything? I believe that 
logging after initialization is enough. And for my example of a logging 
handler that uses ORM it's the only way it can work. Initialization by 
definition shouldn't do anything interesting for an application 
programmer to look for, it should either succeed or fail with an 
exception saying that it "can't run your program, sorry".

As it stands now loading and processing of all the settings is the point 
  that marks success of initialization. So I'm with Simon in putting 
logging somewhere where all the other settings get processed.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #9886: HttpRequest refactoring

[Ivan Sagalaev]

Jacob Kaplan-Moss wrote:
> On Thu, Sep 17, 2009 at 10:09 AM, Ivan Sagalaev
>  wrote:
>> Ticket [9886] is one of the things that I've been holding for 1.2 time
>> frame to ask for some discussion since Jacob marked it DDN.
> 
> At first glance, you need docs and tests before you get a second glance :)

Hello again!

I've finally got back to the ticket and added docs and tests. For docs 
I've extended the reference in request-response.txt which, being a 
reference, is short. I could write a longer topic document with more 
example if needed. I just don't know if it's worth to make a whole new 
topic for a rather advanced and not commonly-used feature.

Also I've proposed this ticket for 1.2 on the wiki[1]. Is this enough to 
get it in the decision process or should it be added into a spreadsheet 
as well?

[1]: http://code.djangoproject.com/wiki/Version1.2Features#HTTP

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #9886: HttpRequest refactoring

[Ivan Sagalaev]

Russell Keith-Magee wrote:
> Any complete feature is potentially a candidate for
> v1.2 - as long as you can convince a core developer to look into the
> ticket.

This is what I'm trying to do, yes :-). The ticket is assigned to 
Malcolm (and in fact it was created after we talked with him if such a 
thing is needed) but he's incredibly busy lately so I decided to discuss 
it here to get someone else interested.

> It's also worth noting that one of the GSoC projects was "HTTP/WSGI
> improvements". If you want to get #9886 into trunk, I suspect you
> would be well advised to see how your patch integrates with the
> changes on the soc2009 branch for that project.

I had that in mind from the beginning of that GSoC but didn't look at 
the code yet. Since the ticket is older than the GSoC branch there were 
chances that it would be committed earlier. But of course I'll  keep an 
eye on the process.

 From a fast superficial look at the code we don't seem to conflict 
anyway :-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Reversing URL for a view with namespaces

[Ivan Sagalaev]

Hello!

I've just stumbled upon a difficult to understand problem. I have an app 
  that has an urlconf which is included in a project under a namespace:

 (r'^blog/', include('app.urls', namespace='blog')),

This namespace allows things like {% url blog:article ... %} which is 
very nice. But reversing doesn't work when an app itself tries to 
reverse its own urls having a callable at hand:

 reverse(views.article, args=[...])

It doesn't work because namespaced URLs are basically excluded from 
resolver's reverse_dict and are put in their own resolvers under 
namespace_dict. And reverse doesn't traverse those when it gets a callable.

On one hand I can see logic here: trying to reverse a namespaced URL 
without specifying a namespace can lead to ambiguous results. But in 
practice it means that one can break any third-party app that reverses 
its URLs (using models.permalink, redirect, or reverse) just by 
including its urlconf under a namespace. Because an app doesn't know its 
namespace and then has no way of constructing a correct URL.

Am I missing something or is it an evil bug?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Reversing URL for a view with namespaces

[Ivan Sagalaev]

Russell Keith-Magee wrote:
> On the 'missing something' front, reverse() now takes a 'current_app'
> argument that gives context to the app lookup, which resolves the
> ambiguity from the point of view of the reverse() function.

I saw this one. It doesn't work in this case exactly beacuse of this:

> This solves the problem as long as applications have been built to
> provide a namespace.

Which means an app knows its exact namespace.

> However, this doesn't fully address the 'bug' you
> describe - an application that hasn't been built to accept namespaces
> will have problems if it is deployed in a namespace.

There's important detail here.

One thing is a namespace-ignorant application that just uses reversing 
the old way. I don't think we can fix it in any way except saying 
somehow "don't install this app under a namespace" (as you suggested).

Another thing is an application that is being written now and that wants 
to be namespace-aware. The problem is that while an application knows 
that it can be included under a namespace it doesn't care under which 
one exactly.

As far as I can tell it can be solved by keeping views in 
resolver.reverse_dict as it was before namespaces were introduced as 
well as in namespace_dict and app_dict. This way they could be found 
both by specifying a name with or without namespace. However it doesn't 
solve the problem when you have two instances of the same application. 
For this we should have a way for an application to know which instance 
of it is "current". Which means tying reversing to a request (right?). 
At this point I didn't come up with anything more useful...

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Reversing URL for a view with namespaces

[Ivan Sagalaev]

Russell Keith-Magee wrote:
> Apologies for taking so long to get back to you on this.

No problem :-) Have busy schedule myself :-)

> For this last case - building a new app that you want to be namespace
> aware - I think the Django admin already implements the model that
> needs to be followed here. I can't think of any obvious way to allow
> for an application to be 'namespaceless', as well as potentially one
> of many in a namespace. Admin solves this by *always* requiring a
> namespace - even if that's just the default application namespace.
> Django's URL namespace lookup makes the simple case easy, (i.e.,
> admin:named_url works, regardless of how you have deployed admin),
> while allowing for multiple namespaces.

Ahha! This is the catch I was missing. So in order for an app to place 
itself generally in the "namespace-aware" category it should define a 
function that will return a triple of (patterns, app_namespace, 
instance_namespace) and document it as the only way of including itself 
in a project's urlconf.

This makes sense now. Thanks!

> All this leaves is catching (and preventing) the case of deploying
> applications that aren't namespace aware inside a namespace. For
> this... patches welcome :-)

I'll try to look into it when someone reminds of a concept known to 
other people as "free time"...

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Reversing URL for a view with namespaces

[Ivan Sagalaev]

Ivan Sagalaev wrote:
> Ahha! This is the catch I was missing. So in order for an app to place 
> itself generally in the "namespace-aware" category it should define a 
> function that will return a triple of (patterns, app_namespace, 
> instance_namespace) and document it as the only way of including itself 
> in a project's urlconf.
> 
> This makes sense now. Thanks!

And now with my newly found understanding I think I've found a small bug 
(and may be a bigger one). Looks like this:

 include(myapp.urls.urls, app_name='custom_app_name')

is supposed to override whatever myapp provides as an app_name with 
user-specified value 'custom_app_name', right?

But now it doesn't. Here's an excerpt from `include`:

 def include(arg, namespace=None, app_name=None):
 if isinstance(arg, tuple):
 # callable returning a namespace hint
 if namespace:
 raise ImproperlyConfigured('Cannot override ...')
 urlconf_module, app_name, namespace = arg

A final `app_name` here is extracted from `arg` that an application 
provides, not from `app_name` parameter. If this is really a bug I'll 
happily make a patch (it's small).

A bigger (possible, provided I understand things right) bug is that 
include explicitly forbids redefinition of `namespace` for an app that 
*does* provide its own namespace. But as we just concluded a 
namespace-ignorant app just can't be included under a namespace. So 
specifying a namespace in include is a) not possible for "old" apps and 
forbidden for "new" ones. How come?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Conventions around plugable backends

[Ivan Sagalaev]

On topic, I'm +0 on class-based approach.

There's actually one passage that reminded me of something that I 
consider a small wart in a couple of places in Django:

Russell Keith-Magee wrote:
> Module-based configuration:
> ---
> 
>  * The aesthetic of user-configuration options is cleaner because
> configuration items are shorter and don't require duplication of
> terms. e.g., django.core.cache.backends.locmem.LocMemCacheBackend
> duplicates the 'locmem' bit for no real end-user gain unless you're
> planning to have a non-locmem backend in the locmem package.

Nothing forces people to duplicate module name in class names. This same 
class could be named "django.core.cache.backends.locmem.Backend". It 
also encourages a good style of importing the entire module and 
referring to its content with a prefix:

 from django.core.cache.backends import locmem
 backend = locmem.Backend()

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: What do people think about the get_absolute_url proposal?

[Ivan Sagalaev]

Mike Malone wrote:
> On Tue, Dec 8, 2009 at 7:52 PM, Russell Keith-Magee
>  wrote:
>>  4. I share Mike's concern about using settings.SITE_ID to determine
>> the current host, but I'm not sure I have any suggestions on how we
>> could practically use request, short of encouraging the use of a
>> template tag like {% obj_url %} that can use data from the request
>> context if it is available.
> 
> It's not exactly pretty, but: http://paste.pocoo.org/show/155827/

All these are variations of the same thing: making request object 
global. Do we really want it?

For one thing it breaks what Django has always got right: being able to 
work in a script outside of a web request loop. So relying on 
contrib.Site may be inconvenient but it's way better than a global 
request. What I take from code like this:

 protocol = getattr(settings, "PROTOCOL", "http")
 domain = Site.objects.get_current().domain
 port = getattr(settings, "PORT", "")

... is not that we should get all these from a request but that we 
should add "protocol" and "port" fields to the Site model.

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: What do people think about the get_absolute_url proposal?

[Ivan Sagalaev]

Mike Malone wrote:
> Well, not really. It's making a way to generate a URL based on the
> request object global. I agree that it's not ideal, but it's not the
> same as just making the request object global.

My main gripe is not globalness of a request object itself (I agree with 
"not ideal" here) but the very idea of constructing a URL from request.

> You wouldn't have any trouble in a standalone script unless you tried
> to call the get_absolute_url() function.

But I kinda want that. Here's two more usecases where using current 
request for creating URLs is broken:

- If I have an API part of the service and human-readable part of the 
service on different hosts and I want to construct a reference to API 
when serving user pages.
- If have several machines behind a load-balancing proxy that's not 
under my control and that's not telling me its hostname I don't want to 
construct URLs with internal hostnames of individual machines in cluster.

In other words there are legitimate real-world cases when "current" 
requests has nothing to do the URL I want to cnstruct.

> I will reiterate that, in practice, this is a huge pain in the ass.

Can you provide an example? My experience doesn't match this. My only 
minor complaint is that I sometimes forget to update default 
"example.com" generated for Site model on new installation (which should 
be fixed differently, IMO).

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: What do people think about the get_absolute_url proposal?

[Ivan Sagalaev]

Michael Richardson wrote:
> With these cases (the former more so than the latter) you are creating
> URLs for an entirely separate project, not your own, essentially.

Not "entirely separate". These are projects sharing much of the 
codebase, just having different SITE_ID in settings.

Anyway there's still my first use-case of constructing URLs from 
standalone scripts.

> We pass around request objects all over the place in order to get this
> functionality.  request.build_absolute_uri is great, but it does
> require that request object and passing it around - which sucks.

request.build_absolute_uri works for you because you don't have those 
use-cases. This is why we see this way differently: you think that 
having a request universally available makes things more convenient 
while in my view it doesn't fix URL construction at all[*] and just adds 
a bit to the mess in thread-locals. And I kind of disagree that my 
use-cases very exotic.

[*] URL construction is really broken right now because I am forced to 
hard-code 'http://' or 'https://' in my script that sends mail by cron.

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: What do people think about the get_absolute_url proposal?

[Ivan Sagalaev]

Mike Malone wrote:
> In my opinion, using the Site module and settings files is damn
> annoying.

My point is that we better fix these annoyances than leave this way of 
doing things altogether.

> I never use the Site module

This is by itself is not the reason why it might be annoying. Django 
actually recommends its usage in documentation and if you don't follow 
it then, why, some level of annoyance is expected.

>, and in my experience having to
> change the "FRONTEND_URL" of your app every time you push to a
> different environment is tedious and a frequent source of subtle
> problems.

Indeed. This is why there's Site model. It's better than a setting 
because it's per-database, not per-installation.

I agree that having to maintain a record in Site is also not ideal. But 
it can be made less error-prone:

- syncdb could interactively ask for scheme/host/port instead of using 
"example.com" (that would fix all *my* problems with SIte)
- correct set of sites can live in initial_data fixture

> Moreover, the request information in your current request
> _should_ always be correct.

Agreed. But it's not the whole point I'm arguing.

> That said, it sounds like there are a number of special cases where it
> would be useful to override these settings. So maybe the best corse of
> action is to try to use the configured Site information and fall back
> on "RequestSite", which uses information from the currently active
> request.

This might work, I think. Anyway I can't bash this idea with anything 
from the top of my head :-). I'll think of it a bit more.

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Ticket #5025 - truncate filter, why hasn't it been accepted?

[Ivan Sagalaev]

Alex Gaynor wrote:
> Adding the ellepsis is as simple as:
> 
> {{ foo|slice:":100" }}{% if foo|length > 100 %}...{% endif %}

Come on Alex, in what universe it qualifies as "simple"?!

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: idea for using RequestContext by default

[Ivan Sagalaev]

Russell Keith-Magee wrote:
> I'm in complete agreement with Simon on this point. I fail to see the
> benefit of decorators in this context.

I can see one valid point that "render_to" guys have. Which is, 
incidentally, has nothing to do with the decorator syntax. An 
HttpResponse with an opaque string as its content is hard to alter after 
it is returned from a view (one of the precedents was the old 
CsrfMiddleware was parsing HTML to inject a token). And most of the real 
use-cases for this are about adding something to the view's context 
before it blends in a template. Right now the only way to do it is with 
context processors but they are global, not per-view, which is not 
convenient.

I think it can be solved another way, by something like a special 
HttpResponse subclass -- TemplateResponse. It can keep a template and 
its context separate until the very last moment when its contents is 
asked and only then do render a template.

This will allow extension of the views:

 # some library:

 def library_view(request):
 # ...
 return TemplateReponse('template.html', context = { ... })

 # user code:

 def my_view(request):
 response = library_view(request)
 response.context.update({ ... })
 return response

Russel, can you see something like this in Django trunk?

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: idea for using RequestContext by default

[Ivan Sagalaev]

Russell Keith-Magee wrote:
> Certainly - and Simon made exactly this proposal during the review of
> the CSRF work. Simon even provided a sample implementation:
> 
> http://groups.google.com/group/django-developers/msg/b1b3f8854b9ae2b1

Thanks! I keep missing nice things on django-dev@ due to many *-dev 
lists traffic :-(

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Ivan Sagalaev]

Tobias McNulty wrote:

I regret and apologize that I'm arriving to this thread rather late.


To support the tradition, I'm apoligizing for this too :-). Though it's 
funny how everyone thinks they're "late" on a couple-of-days-old thread :-).


Anyway...

I too would opt for an implementation that makes model validation 
optional, i.e., a call that developers must explicitly make, if they 
choose, before saving a model.


I'm +1 on some way of validating a form without *fully* validating a 
model. But for a bit different reason that I didn't see in this thread yet.


The thing is that validating a ModelForm may not be strictly related to 
subsequent saving of an instance. A use-case:


I have a PreviewForm that generates a preview for a comment that user is 
writing in a form. It uses model fields to validate currently typed text 
and selected markup filter and then creates a model instance to do 
actual formatting:


class PreviewForm(ModelForm):
class Meta:
model = Comment
fields = ['text', 'markup_filter']

def preview(self):
comment = Comment(
text = self.cleaned_data['text'],
filter = self.cleaned_data['markup_filter'],
)
return comment.html()

It is then used like this:

def preview_view(request):
form = PreviewForm(request.POST)
if form.is_valid():   # <- this now breaks
return HttpResponse(PreviewForm.preview())

The thing is that the code has no intention to save an instance into a 
DB. It just needs it at application level. This is why I think that 
decisions on "full" vs. "partial" validation should not be bound to 
`save()`.


(FYI currently the first patch (Joseph's) in #12521 does fix this 
problem while the second patch (Honza's) doesn't. From a quick look this 
is beacuse the second patch only excludes fields from validation that 
listed in `exclude`).
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Ivan Sagalaev]

Tobias McNulty wrote:

I don't see why model validation should be bound up with forms at all.


I'm not the one who designed it, so it's just me view. I think this is 
just useful: if you have a code validating, say, a CharField at the 
model level why not reuse it at the form level?


What's important is that *entire validity* of a form should not be bound 
to that of a model. This was a bug which Joseph Kocherhans is now fixing.


 The current release notes for model validation state that it won't be 
done unless explicitly requested by the developer, and it seems that 
validating the model inside a form (whether fully or partially) breaks 
that contract.


Well, I think we can fix release notes for the next release :-).
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Logging format decision

[Ivan Sagalaev]

Hello everyone!

There was a discussion here some time ago about logging in Django that 
led to the ticket 12012 [1] where Simon has done most (all?) of the 
implementation.


There is still a disagreement on the format of the configuration dict 
for it. Simon has implemented it from my ad-hoc proposal [2]. Vinay 
Sajip has an alternative suggestion [3]. I think a core-dev ruling is 
due here.


To state my own opinion, I sill like my format better because it defines 
each logging channel in one place without needing to separately define 
formatters and handlers. I understand that it makes it less powerful but 
I'd rather have a very simple format here -- we still can configure 
logging imperatively in advanced cases.


I'll be happy to do some docs and tests for the ticket once it's decided 
(as far as I can see Simon's branch lack them).


[1]: http://code.djangoproject.com/ticket/12012
[2]: 
http://groups.google.com/group/django-developers/msg/7452fac3ae3b8175 
(look for item 4)

[3]: http://mail.python.org/pipermail/python-dev/2009-October/092782.html
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Ivan Sagalaev]

Joseph Kocherhans wrote:

# Run validation that was missed by the form.
p.validate_fields(fields=['user', 'primary_contact'])
p.validate_unique(fields=['user', 'primary_contact'])
p.validate()


Can this be shortcut to

p.full_validate(fields=['user', 'primary_contact'])

?

If not, why not? :-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Logging format decision

[Ivan Sagalaev]

Russell Keith-Magee wrote:

That said - I really do want to see logging in Django's trunk. If we
can sort out the details in the background and get a good
implementation ready, it could easily be the first feature committed
for v1.3.


Agreed!


I like Vinay's suggestion for exactly the reason you dislike it. The
definition of a handler is independent of the loggers that use that
handler. Simon's example in (4) hides a lot of details behind ellipses
- if you expand those ellipses, you discover that there is a lot of
repeated typing of '/tmp/django-sql.log' etc if you want all your logs
to appear in the same file. Under Vinay's proposal, you define a
single file log handler, and instruct each of the loggers to use that
handler.


Actually, I don't see this problem of repetition that you're describing 
so I thought that we could use some examples closer to real world to 
sort this out. May be I just don't see something obvious to you or have 
misunderstood Vinay's schema. I will refer to two formats we're 
discussing as "explicit handlers" (Vinay's) and "implicit handlers" 
(what Simon has implemented). Better suggestions for naming are welcome :-).


1. The first use-case is the development environment where all you want 
is everything logged in one file with DEBUG level.


"Implicit handlers":

LOGGING = {
'': {
'handler': 'django.logging.FileLogger',
'filename': '/home/user/myproject/all.log',
'level': 'debug',
},
}

"Explicit handlers":

LOGGING = {
'formatters': {
'all': '%(asctime)s %(name)-15s %(levelname)-8s %(message)s',
},
'handlers': {
'all': {
'class': 'django.logging.FileLogger',
'filename': '/home/user/myproject/all.log',
'formatter': 'all',
'level': 'debug',
},
},
'loggers': {
'': {
'handlers': ['all'],
},
},
}

2. A production environment. This actually what we have for one of our 
services:


- a file for uncaught server errors that are logged by a function 
listening to `got_request_exception` signal which uses 'exception' label 
to log
- a file for maintenance messages that come from cron scripts that all 
use 'maintenance.*' labels to log

- a file for everything else

"Implicit handlers":

_log_path = '/var/log/service/'

LOGGING = {
'exception': {
'handler': 'django.logging.FileLogger',
'filename': _log_path + 'exception.log',
'format': '%(asctime)s %(levelname)-8s %(message)s',
'level': 'warning',
'propagate': False,
},
'maintenance': {
'handler': 'django.logging.FileLogger',
'filename': _log_path + 'maintenance.log',
'format': '%(asctime)s %(name)-15s %(levelname)-8s 
%(message)s',

'level': 'info',
'propagate': False,
},
'': {
'handler': 'django.logging.FileLogger',
'filename': _log_path + 'info.log',
'format': '%(asctime)s %(name)-15s %(levelname)-8s 
%(message)s',

'level': 'info',
},
}

(Note: "propagate" is not implemented right now in Simon's code but is 
quite easy to add).


"Explicit handlers":

_log_path = '/var/log/service/'

LOGGING = {
'formatters': {
'exception': '%(asctime)s %(levelname)-8s %(message)s',
'default': '%(asctime)s %(name)-15s %(levelname)-8s 
%(message)s',

},
'handlers': {
'exception': {
'class': 'django.logging.FileLogger',
'filename': _log_path + 'exception.log',
'formatter': 'exception',
'level': 'warning',
},
'maintenance': {
'class': 'django.logging.FileLogger',
'filename': _log_path + 'maintenance.log',
'formatter': 'default',
'level': 'info',
},
'info': {
'class': 'django.logging.FileLogger',
'filename': _log_path + 'info.log',
'formatter': 'default',
'level': 'info',
}
},
'loggers': {
'exception': {'handlers': ['exception'], 'propagate': False},
'maintenance': {'handlers': ['maintenance'], 'propagate': 
False},

'': {'handlers': ['info']},
},
}

In both examples "implicit handlers" give shorter code and, to me, are 
more readable.


The main point that I'm trying to argue is that having handlers and 
loggers *separate* is not needed in real world scenarios. If you want 
your logging in several files you will have to define several handlers 
and several loggers anyway. "Implicit handlers" format accept this fact 
and couples both handler and logger definitions in one place.


The only thing that 

Re: Logging format decision

[Ivan Sagalaev]

Russell Keith-Magee wrote:

You've also defined a production server that uses just two loggers -
exception and info - for the entire server. Again, this isn't
especially realistic - in practice, you have a large number of fine
grained loggers for different parts of the system.


In fact, most of our production services use just two loggers: one for 
exceptions and one for everything else. So that one with maintenance was 
the most interesting :-). But see below...



I think I understand what you're advocating - I just think you've
missed some very important use cases.


And I do understand what you and Vinay are saying. I really don't miss 
those use-cases. I intentionally try to avoid them in practice to not 
complicate things.


But since you both feel that they are indeed important so be it, let's 
make it this way.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Master/slave replication in multi-db era

[Ivan Sagalaev]

Russell Keith-Magee wrote:

There are some cases where this shouldn't happen - for example, in a
master/slave setup. I'm tinkering with some code at the moment to
control this sort of database allocation.


Russel, can you share your ideas on the matter? I'm about to port 
(soon-ish) my replication backend to new multi-db reality and if you 
already had some thoughts I'd very appreciate to hear it.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feedback requested: Multi-db database routing

[Ivan Sagalaev]

Russell Keith-Magee wrote:

class MasterSlaveRouter(object):
def db_for_read(self, model, db=None):
# Point all read operations to a random slave
return random.choice(['slave1','slave2'])

def db_for_write(self, model, db=None):
# Point all write operations to the master
return 'master'

def allow_relation(self, obj1, obj2):
# Allow any relation between two objects in the db pool
db_list = ('master','slave1','slave2')
if obj1 in db_list and obj2 in db_list:
return True
return None

Then, in your settings file, add the following:

DATABASE_ROUTERS = ['path.to.AuthRouter', 'path.to.MasterSlaveRouter']

And you're set.


It's more of a side note.

I understand that this is only an example but examples like this one 
might give an impression that Django now solves master-slave replication 
out of the box. It's not.


The hard thing about replication is that switching between master and 
slaves doesn't depend on models and doesn't depend on SQL operators. It 
depends on some vague notion of "operation context": if we are in some 
logical multi-step process of changing state then all writes *and* reads 
should happen on a master or we won't have a consistent transaction.


That said, this routing looks to me like a convenient and may be even 
necessary plumbing. It just doesn't solve the whole problem.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [Python-Dev] PEP 391 Acceptance - Thanks!

[Ivan Sagalaev]

Vinay Sajip wrote:

Guido,

That's great - thanks very much! I'll get on with integrating the PEP
391 implementation into trunk right away, and it should be in good
time for 2.7/3.2.


Thank you for your efforts!

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Backwards-incompat with CsrfViewMiddleware

[Ivan Sagalaev]

Hello everyone!

A couple of days ago we've noticed a (potentially big) backwards 
incompatibility that can bite users upgrading from 1.1 to 1.2.


CSRF doc encourages users to enable CsrfViewMiddleware that will check 
all POST requests for csrf_token and then suggests to alter all HTML 
forms to include one or wrap in @csrf_exempt those views that accept 
POSTs generated elsewhere. This however is no help for views in 
third-party apps that users don't control. So their site would be broken 
until those apps are updated.


This has actually happened lately to users of my OpenID server library 
that was missing @csrf_exempt.


I won't try to estimate the amount of breakage that this will incur so 
I'm just shedding a light on the issue. May be some Big Fat Warning™ 
should be added to docs or even they should be rewritten to suggest 
using individual decorators instead of a middleware.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Backwards-incompat with CsrfViewMiddleware

[Ivan Sagalaev]

Luke Plant wrote:
This case is slightly different because it is down to an interaction 
of a changed default setting with working code, but there will always 
be cases like that, and I think it is much better for developers to 
remember the general principle that they are responsible for whatever 
code they are running, whether they wrote it or not.


It's not the question of responsibility. We're changing a minor version 
which is supposed to be backwards compatible. If a site will break in 
this case people won't go looking for some responsible person they'll 
just blame Django for breaking a common expectation and creating a 
crunch job for them.


I should also point out the possibility of doing the csrf_exempt 
decoration in the URLconf, as with any decorator - I don't know how 
feasible that is in this case.


It's not. Because third-party apps typically provide their own urlconf 
along with their own views.


Again, it's not the matter of how to fix it, there are plenty of ways to 
do it. But if our changes are backwards incompatible we should clearly 
say so in upgrade/release notes.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Backwards-incompat with CsrfViewMiddleware

[Ivan Sagalaev]

James Bennett wrote:

Perhaps that's why, in both the 1.2 alpha release notes and the draft
for the final 1.2 notes, it's listed under a big heading titled
"backwards-incompatible changes".


As we say in Russia "Gee, I didn't notice the elephant" :-).

I still think it would be useful to add an explicit note about the 
possibility of breaking because of third-party apps.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Hladame Django programatorov (Looking for Django developers - Slovakia)

[Ivan Mincik]

We are looking for experienced Django developers for work in our office located 
in Eastern Slovakia (Presov).
For more information please look here: 
http://gista.sk/index.php/o-splocnosti/ponuka-prace
or send email directly to me.

Thanks,
-- 
Ivan Mincik
Gista s.r.o.


signature.asc
Description: This is a digitally signed message part.

Re: Proposal - ChoiceField.choices need to accept callable, not only list or tuple

[Ivan Sagalaev]

George Karpenkov wrote:

http://code.djangoproject.com/ticket/13181


I'm -1 on complicating it this way. The ability to pass a callable for 
choices is covered by passing a generator. Am I missing something?


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal - ChoiceField.choices need to accept callable, not only list or tuple

[Ivan Sagalaev]

George Karpenkov wrote:

Given your experience, the chances are that you're right, however, I
don't see any way how we can pass a generator which will not be called
until the form class is instantiated.


Why should a call matter? The body of the generator is not executed 
until first iteration attempt and can contain whatever lazy logic you need.


Can you provide an example where you need a function returning a list 
and generator wouldn't suffice.



b) it can be way more convenient to pass a function then a generator
expression (generator expression can be very hard to code if it has to
contain some complex logic)


I fail to see the difference here. A function that builds and returns a 
list is trivially converted into generator by replacing ".append(...)" 
with "yield ..." or replacing a list comprehension with a generator 
expression.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: First-class WSGI support in Django 1.3 / twod.wsgi

[Ivan Sagalaev]

On 05/26/2010 03:02 PM, Gustavo Narea wrote:

Among many other things, one of the components alone provides solutions to some 
enterprise requirements for 
Django:

  * It’s now possible to run code at startup time, in a straight-forward yet 
extremely flexible fashion, which will also work on development servers if you 
want it to – not only when deployed on a production server.
  * You can now stop using a Python module to store your application settings, 
in order to use an intuitive and widely used
mechanism that scales up and scales down. It will just work and you won’t have 
to update any other file in your application.
  * By providing full WSGI support in development mode, we are able to  work around 
the differences in the process environment between django development server and 
django hosted using 
mod_wsgi
  * It’s finally possible to run WSGI middleware in development servers, the 
same way you may do it on production servers.

And this is just the tip of the iceberg.


Gustavo,

Could you please give a concise technical overview, in high-level terms, 
on what twod.wsgi actually does to Django code?


From the part that I quoted above I see that there are things that 
either don't have anything to do with WSGI (I'm fine with settings.py 
BTW) or don't explain anything (I didn't have problems with running 
Django under Paste along other WSGI apps).


P.S. I won't say for others but for me, as a developer, the phrase "just 
the tip of the iceberg" sounds scary, not attractive :-).


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: streaming patches reloaded

[Ivan Sagalaev]

arthur debert wrote:

>I have an app that uploads a 4mb file and is taking a ton of memory.
>>From tickets 1484/1569 they are set as fixed, but I cannot find them on
>django's source code. Are these patches active in trunk? If not,
>anyones knows if they're working after the merge? Anyone using them?
>  
>
1569 is applied to the current trunk.

As for 1484 I use Django with it (which is not surprising since I wrote 
it :-) ) and I don't encounter any problems with form uploads, be it 
with files or not.

My guess is that the main thing that holds this patch from checking in 
is lack of documentation and tests. But I dont' feel myself confident to 
do either thing being not native English speaker and not very 
experienced test writer. Anyone?

Two things need to be documented there:
- new boolean setting "STORE_UPLOAD_ON_DISK"
- new key for request.FILES items: item['file'] which is a file-like 
object for accessing uploaded file. The old way item['content'] still 
works loading file in memory on first access.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Suggestion: Better slugifying of scandinavic characters

[Ivan Sagalaev]

Malcolm Tredinnick wrote:

>So either we're going to end up carrying around a fairly large mapping
>table in the Javascript or we need a better solution.
>  
>
Looking at what Wordpress does at the link in the beginning of this 
thread 
(http://trac.wordpress.org/file/trunk/wp-includes/functions-formatting.php) 
it's not that large really.  Especially in javascript and python that 
know what unicode is and you don't need to write things like 
"chr(195).chr(128)" for utf-8. You can  also strip half of that table 
with capital letters since slugs don't use them anyway.

>To put the problem into context: it's only a small generalisation to
>attempt to do the same thing for mapping Japanese characters to
>ASCII-based URLs.
>
This is a whole another issue since there is no "correct" translation 
from non-latin based scripts that wouldn't look plain ugly. So let's 
keep this issue to just "stripping accents and umlauts" without claiming 
an ability to produce slugs for any language. Django will become a 
little bit better for latin-based scripts with no changes for the rest.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Suggestion: Better slugifying of scandinavic characters

[Ivan Sagalaev]

everes wrote:

>So any django application used slugs are no use for none-ascii-charcter
>languages.
>  
>
Exactly. My point is that Django shouldn't try to solve this problem. 
But impossibility to make truly international slugs shouldn't stop 
Django to do it better for couple of languages in Europe.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: streaming patches reloaded

[Ivan Sagalaev]

arthur debert wrote:

>Hi Ivan.
>
>Just to make sure I understand:
>- after patching django I 've set:
>STORE_UPLOAD_ON_DISK = True
>
>I am trying to use this on the admin. Any more steps I must take?
>  
>
Hm... I don't think it would help very much in admin. The patch provides 
an interface to an open file-like object stored on disk but admin still 
accesses it the old way so the file will be immediately loaded in 
memory. However I think that there _will_ be a performance gain even in 
this case since the patch will stream the file on disk when it is 
received by the web server. Without the patch this file is stored in 
memory like 2 or 3 times (including temporary storages when copying).

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: unicode.. reject?

[Ivan Sagalaev]

gabor wrote:

>i'm also willing to help with this task... but..do i understand this 
>correctly, that it's agreed that django is going to switch to unicode?
>  
>
Being one of the proponent of the all-unicode way back when it was 
proposed I should say that the more I think of it the more I'm afraid 
that it can create just as much problems as it will solve. Today there 
is a convention that Django works everywhere with byte strings except 
some rare cases where it is convinient to use Python's built-in unicode 
functions (these cases include counting length in validation code and 
various string conversions in filters like in the unapplied patch in 
ticket http://code.djangoproject.com/ticket/924). Using this convention 
one can write international apps without worries since all messy things 
are made inside the framework.

If we change this convention to using Python's unicode everywhere we 
will hardly win anything except some feeling of a more "pure" approach. 
At least _I_ can't see any gain :-). However there will be some 
disadvantages that I think are serious:

- I heard that many 3rd party libraries don't work with unicode so a 
user will be forced to do some coversions manually
- byte strings is a default string type in Python and is more widely 
used than unicode, I'm afraid many people in ASCII-world will be 
resistant to switching all their coding to unicode because they can't be 
sure that it works right because they can't test it easily themselves

So may be Django should just wait for Python 3000 when unicode string 
will become default ones and developers would be more widely aware of 
this change.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Custom manipulators

[Ivan Sagalaev]

Luke Plant wrote:

>Has anyone tried to use Django's manipulators and forms for the case 
>where you only want a few fields to be editable, and the rest to keep 
>the original values?
>
I'm doing it very often. I have some frankenstein user model which is 
editable partly in one form, partly in another and third form edits data 
from my User model and contrib.auth's User model in together :-)

>In short, I was hoping to do something like this:
>
>class MyManipulator(MyModel.ChangeManipulator):
>   limit_to = ['my_text_field', 'my_date_field', 'my_upload_field']
>  
>
You are looking for undocumented "follow":

class MyManipulator(MyModel.ChangeManipulator):
  def __init__(self, id):
MyModel.ChangeManipulator.__init__(self, id, follow={
  'my_text_field': False,
  'my_date_field': False,
  'my_upload_field': False,
})

Manipulator will look only for fields in its "follow" dict where their 
values evaluates to "true". A default manipulator has its "follow" 
prefilled with editable fields from a model and a "follow" parameter 
that is passed in the constructor will update those values.

Similarly this way you can include non-editable fields that don't get in 
by default, just pass them with "True" value.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Custom manipulators

[Ivan Sagalaev]

Luke Plant wrote:

>Manipulator.get_validation_errors() ignores the 'follow' parameter, so
>you get validation errors for fields that aren't included.
>
Just tested it and it shows that you don't. I have a required field 
switched off with "follow" that is not in the form. It doesn't give an 
error. When I include it in follow with "True" it does give an error.

My quick guess (I'm a bit lazy to check :-) ) is that automatic 
manipulator just don't create field objects based on follow and this is 
why it works for validation.

>  Also, it's
>not simple to strip them out, since you have things like 'mydate_date'
>and 'mydate_time' if you have a DateTimeField called mydate.
>  
>
Follow works on model fields, not manipulator field objects. I 
nonetheless checked it out, this is the beginning of automatic 
manipulator constructor:

def __init__(self, follow=None):
self.follow = self.opts.get_follow(follow)
self.fields = []

for f in self.opts.fields + self.opts.many_to_many:
if self.follow.get(f.name, False):
self.fields.extend(f.get_manipulator_fields(self.opts, 
self, self.change))  # Here's when your date and time fields come into 
existance

>Also, 'follow' is sub-optimal -- you have to calculate (either by hand
>or automatically) the fields that you want to *exclude*, rather than
>pass in simply the ones you want to include.  Maybe some utility
>function on AutomaticManipulator might help.
>  
>
Yes... But I should say that I needed to exclude fields from 
manipulators more often.

>My aim in all of this, is that if I add any type of field to MyModel, I
>shouldn't have to alter MyCustomManipulator at all.
>  
>
Hm... But there are cases when you want include a new field in this 
particular manipulator and there are when you don't. I don't think there 
can be a default behavior.

An example. I have a User model and it has some fields that only an 
administrator can touch (priveleges, rating) and fields editable by a 
user (birth_date, location, etc..). I have two manipulators for this 
model: administrative one with excluded personal fields and a personal 
one with excluded administrative fields. And I can't predict in which 
manipulator any new field should go. I will have to decide it for each 
field anyway.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: unicode.. reject?

[Ivan Sagalaev]

gabor wrote:

>>Using this convention 
>>one can write international apps without worries since all messy things 
>>are made inside the framework.
>>
>>
>
>well, i would replace 'without worries' with 'good enough' :)
>  
>
Ok :-)

>for example, i was building a very simple web-file-manager. you can ask 
>python to get you the os.listdir, etc, data in unicode, which i think 
>the only sensible way, because otherwise you have to watch out for the 
>filesystem-encoding. but because the querystring and httpresponse is 
>bytestring, i have .encode('utf8'), .decode('utf8') all the way.
>  
>
Not at all. You can talk to file system in utf-8 and do no conversions 
at all. I'm doing it in my app and it works just fine.

I don't know exactly how Python handles encoding of data that it 
receives from the OS and I think there can be problems with 
misconfigured OS giving out those characters in a non-UTF8 byte 
encoding. But unicode here won't help either since Python would try to 
convert some arbitrary stream to unicode and will throw an exception. 
With  byte string I suppose you will get the string without errors but 
with the garbage in it. All this anyway just requires configuring 
locales properly.

And as a sidenote: HttpResponse can happily accept unicode with 
convertion to DEFAULT_CHARSET on actual output.

>hmm... is it that hard to get some japanese text and maybe some german 
>names? :)
>  
>
I meant that it's hard when you should install some weird fonts and look 
at symbols you don't understand :-). But you are right. I think every 
developer in ASCII world can work with, say, accented latin characters 
and test if they are uppercased or counted properly.

On the other hand people working closely with non-ASCII characters are 
working out some useful debugging habits that one just can't know only 
by theory. For example I can very quickly distinguish string that is in 
utf-8 shown in windows-1251 locale from string in windows-1251 shown in 
utf-8 locale by just looking at it. For a person not working with 
cyrillic they all will look equaly weird I think... Which is kinda 
supports the idea of all-unicode since Python represents byte-strings 
differently than unciode ones.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Generalisation of exclude/QNot has broken QNot

[Ivan Sagalaev]

I noticed today that in http://code.djangoproject.com/changeset/2997 
there were some changes to QNot after that my code has broken. I was 
using Q() and QNot() to dynamically construct a search query like this:

query = Q()
if :
  query = query & Q()
if :
  query = query & QNot()
   
results = MyModel.objects.filter(q)

After updating Django this stopped working saying with an exception 
saying that "__init__() got an unexpected keyword argument '...'". Which 
looks like a bug since this "q" is not attached to any model and 
shouldn't try to resolve kwargs.

I then changed all this logic from "q = q & Q()" to "queryset = 
queryset.filter()" (and "exclude()" for negation). I even like it better 
but still want to know if Q()'s and QNot()'s are not intended now to be 
used independently?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Generalisation of exclude/QNot has broken QNot

[Ivan Sagalaev]

Luke Plant wrote:

>I think the basic assumption is that undocumented features and
>functions can be changed without warning.
>
Absolutely! I'm not complaining :-). I'm just trying to get general idea 
of how it is supposed to be used.

>  You are of course free to
>re-use any Django code, but you need to keep your eye on django-updates
>if it's undocumented stuff.
>
It's what I did converting my code to use querysets, filters and excludes.

However if you interested here's why I needed this in the first place. I 
have a search interface that uses one set of conditions to search two 
different types of objects: music Albums and Artists. Users select some 
values in form and get two result lists side by side which is in this 
particular example appeared to be useful.

In code I was constructing the query once and then used it for both models:

q = Q()
q = q & Q(...)
...
albums = Album.objects.filter(q)
artists = Artist.objects.filter(q)

One of the conditions happend to check if one of the fields was not an 
empty string so it was QNot(album__path__exact=''). That's why it was 
needed.

As I already said I have it working now with querysets but I can't use 
the same set of conditions for both models (I  actually construct 
querysets twice). I wonder if querysets can do this?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Generalisation of exclude/QNot has broken QNot

[Ivan Sagalaev]

Luke Plant wrote:

>Constructing the query once is pretty neat. I would suggest using your
>original method (you use source control, right?), and then adding
>QNot(Q(album__path__exact=''))
>  
>
Ah... That's the trick! Thanks :-)

I certainly use source control but I won't use the old code since the 
shared part of the query actually doesn't contain QNot. A working code 
for me is now better than a pure one 'cause I'm about to finish the 
project in a few days.

>albums = Album.objects.all()
>albums = albums.filter(...) # your stuff here
>artists = Artists.objects.all()._clone(_filters=albums._filters).
>  
>
I would rather define a function accepting model and building a queryset 
and then call it twice with Album and Artist:

def build_qs(model):
  qs = model.objects.all()
  qs = qs.filter(...)
  qs = qs.exclude(...)
  return qs

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: PROPOSAL: Manager.get_or_create()

[Ivan Sagalaev]

Adrian Holovaty wrote:

>Time and time again I have the following Django code:
>
>try:
>obj = Person.objects.get(first_name='John', last_name='Lennon')
>except Person.DoesNotExist:
>obj = Person(first_name='John', last_name='Lennon',
>birthday=date(1940, 10, 9))
>obj.save()
>  
>
"Me to!" :-)

So +1 on the proposal.

>* I needed to get both the new object *and* a boolean specifying
>whether a new object was created, so get_or_create() returns a tuple
>of (new_object_created, object).
>
My first reaction is that it should be reversed: the "primary" returned 
value is an object itself and a flag is often not important so it feels 
like "secondary" value.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: manipulator.save(data) doesn't return a object from database

[Ivan Sagalaev]

Viktor wrote:

>blabla = models.IntegerField(default=0, editable=False)
>  
>
editable=False means exactly that manipulator won't touch this field, 
this is why it's not updated with the value from data.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Improved FileField ideas?

[Ivan Sagalaev]

Jay Parlar wrote:
> So what I'm thinking, is to allow something like the following:
>
> class User(models.Model):
> username = models.CharField(...)
> avatar = models.ImageField(upload_to="users/" + self.username,
> erase_old=True)
>   
For "upload_to" case I think this will work:

upload_to="users/%(username)s"

with a substitution:

dir = upload_to % obj.__dict__

A (little) problem I see with this solution is that data may be not safe 
for a filename but Django does some safety replacements for supplied 
filenames anyway so it can be used here too.

I like your erase_old. The logic (as I implemented it in my views) can 
be as follows:

old_filename = obj.get_FILED_filename()

if erase_old and
   ( or ) and
   :
  os.remove(old_filename)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: HttpResponseSendFile

[Ivan Sagalaev]

SmileyChris wrote:
> I don't want to go reopening the ticket, but couldn't this still be
> useful functionality?
>   
The regular HttpResponse already can serve files in some fashion:

f = open(filename)
return HttpResponse(f, mimetype='application/octet-stream')

Here the file-like object will work as an iterator sending one line  at 
a time.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

handler404 (and 500?) should use RequestContext

[Ivan Sagalaev]

Currently the default handler404 view uses simple Context. This results 
that often people relying on passing their stylesheet URLs with context 
processors loose this for 404 pages (I have an app that is written 
entirely with generic views except a custom handler404 for the single 
purpose of having sites style on this page too.

Is there a reason to not use RequestContext in handler404? If not I 
could make a patch correcting it.

In the case of handler500 however I'm not sure. General policy in my 
applications that thing handling fatal errors has to contain as few 
logic as possibly to lessen probability having an error in a handler itself.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: handler404 (and 500?) should use RequestContext

[Ivan Sagalaev]

Adrian Holovaty wrote:
> We changed the default 404 and 500 views to use RequestContext a
> couple of days ago, so you may not have seen that change. :)

Ouch! Sometimes this Django thing changes too fast :-)

Thanks!

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Generic Relationships/convert_post_data in forms

[Ivan Sagalaev]

Max Battcher wrote:
> Have you seen Ivan Sagalaev's tag model?  It needs updating to support 
> the new GenericRelation (it currently uses a similar but proprietary 
> clas),

I somewhat missed the whole discussion on generic relation though I get 
the general idea. Do they work for ManyToMany relations? And is there 
any description of how to use them?

> but it has a very neat Tag widget for the Admin.

To be really neat it pretty much requires some more styling to layout 
that bunch of input fields. I didn't include such stylesheets into the 
package since it depends on a page's layout.

> http://softwaremaniacs.org/soft/tags/

Or more likely: http://softwaremaniacs.org/soft/tags/en/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

ManyToManyField looks broken

[Ivan Sagalaev]

I'd like to draw some attention to ticket 
http://code.djangoproject.com/ticket/2232 (mainly russelm's attention as 
  it appeared after his checkin 3195).

Basically ManyToMany relations don't work when you try to get related 
queryset from a model where ManyToManyField is defined. Or is it just me?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: ManyToManyField looks broken

[Ivan Sagalaev]

Malcolm Tredinnick wrote:
> Pretty much "just you", but it's obviously a little trickier than that.
> This is basically ticket #1796 occurring again.

I've dug a bit further and have found out that mentioned changeset 
(3195) has broken only certain relations (for cross-app models and 
another case I have described in the ticket 2232)

> You'll find it will only
> fail in some very specific cases and it's dependent upon both your
> machine and your Python build.

I see the problem in this bug too (just created a very minimal test app 
with two models). It looks like it's not related to the new ticket since 
it fails both before and after changeset 3195.

> Are the tests failing for you, too, Ivan?

No, tests are Ok (except some failures in default filters that I have 
patched locally, but it's not related... hopefully :-) ).

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: ManyToManyField looks broken

[Ivan Sagalaev]

Malcolm Tredinnick wrote:
> I've just checked in r3202 which I believe fixes the root cause of #1796
> in a reasonably correct way. I think it also fixes #2232 for the same
> reason.

It does (just checked). Marking ticket fixed...

Thank you very much, Malcolm and Russel and everyone else!

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: URL confs should take callables as well as strings

[Ivan Sagalaev]

Malcolm Tredinnick wrote:
> Another thing we need to document better here, because this claim is not
> quite correct (didn't you help write this?). The patterns() function
> returns a normal Python list. So you can just append successive calls to
> patterns() to each other:
> 
> urlpatterns = patterns('website.weblog.views',
> # Latest entries.
> (r'^$', 'display.section'),
> ...
> )
> 
> urlpatterns += patterns('',
> (r'^feeds/(?P.*?)$', 
> 'django.contrib.syndication.views.feed',
> {'feed_dict': feeds.feed_mapping}),
> )
> 
> So (Jedi-like wave of hand), these are not the... er.. there is no
> prefix problem.

May be I have some Toydarian relatives and those Jedi things don't work 
for us... But I should say that while adding lists is a known workaround 
  it tends to create unmaintainable urlconfs. This is an example from 
one of my projects:

 urlpatterns = patterns('prolab.orders.views',
   (r'^dashboard/filesources/(?P\d+)/$', ...),
   (r'^dashboard/filesources/(?P\d+)/help/$', ...),
   (r'^dashboard/filesources/progress/$', ...),
   (r'^dashboard/filesources/progress/cancel/$', ...),

   # many others...

 ) + \
 patterns('django.views.generic',
   (r'^dashboard/filesources/$', 'list_detail.object_list', ...),
 )

So just because _some_ of logically related functions happen to be 
simple enough to be handled by generic views they are going into 
separate patterns list. This makes it very hard to find one when you 
need to.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: problem with revers lookup

[Ivan Sagalaev]

Malcolm Tredinnick wrote:
> It's not that we actually suck at fixing this. It's just that realiable
> failures are hard to come by. Once we fix one and think we understand
> the fix, it takes a good three or four hours for Ivan to break it again.

Add the time difference to the problem :-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Unicodification of Django

[Ivan Sagalaev]

Jacob Kaplan-Moss wrote:
> Just one  
> example is the issue of template encodings -- do we need to start  
> indicating that a certain template is UTF-8 or whatever?

May be I don't understand what do you mean...  But this problem is not 
related to internals being in unicode or not. Most templates are fed to 
the browser and they have to be encoded in DEFAULT_CHARSET already 
(which is UTF-8 in most cases, unless someone have to deal with legacy 
code). So this issue is already here because you have to feed string 
data to templates also in UTF-8.

> "... with too little payoff." -- right now it's completely possible  
> to nicely handle Unicode data in Django as long as you're careful.   

With this I agree (with little addition that I still think we should 
commit a patch fixing many template filters for non-ascii letters).

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Unicodification of Django

[Ivan Sagalaev]

Bjørn Stabell wrote:
> What if the patch required everything to be Unicode, meaning:
> 
>  * all programmers would have to become aware of Unicode to some extent
>  * all code would suffer the (minior) performance penalty of encoding
> and decoding all text

The second point is arguable. Currently there are many cases where 
conversion is done twice. For example to check the length of any string 
field in validator it should be decoded to unicode, counted and then 
encoded back to bytes because it's what users expect in views. Same with 
  string processing filters in templates. By converting internals to 
unicode we are requiring conversion everywhere between outside and 
inside but removing these double conversions at the same time. So this 
may be a performance gain as much as a loss. But anyway this seems to me 
absolutely negligible in practice.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

contrib.auth.login doesn't set request.user

[Ivan Sagalaev]

First, a little context. My app uses cookie-based authorization as well 
as HTTP authorization (for download managers mostly). It worked as a 
middleware that sits after Django's auth middleware and tries to 
authorize a user by HTTP credentials if it's not authorized with 
cookies. It goes basically like this (using pre-auth-branch syntax):

 if request.user.is_anonymous():
   # get HTTP credentials
   user = User.objects.get(...)
   if user.check_password(...):
 request.user = user

After getting new authorization API I've redone it this way:

 if request.user.is_anonymous():
   # get HTTP credentials
   user = authorize(username=username, password=password)
   if user:
 login(request, user)

This doesn't work because auth's "login" doesn't set request.user but 
just sets up an environment for the user to be automatically logged in 
on next redirect. But in my case a redirect won't happen because the 
code that checks request.user after all this can't distinguish this 
situation from the actually wrong login when it should say "401 
Unauthorized".

Anyway I beleive that it's worth to make "login" actually set 
request.user for the current session too. Not only for my (edge)case but 
because it just looks more natural: we already know that user is 
successfully authorized.

Thoughts?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: django-unicode conversion, first experiments, questions

[Ivan Sagalaev]

Simon Willison wrote:
> In the absence of anything better than that, I think it's safe to  
> assume that CGI environment variables should always be ASCII encoded.

Talking about QUERY_STRING... While the string itself is in ASCII it has 
urlencoded data and there the encoding matters. As fas as I can see in 
practice browsers tend to encode those data in the same encoding as the 
page from where those links come. So for decoding urlencoded 
QUERY_STRING I think it's reasonable to use DEFAULT_CHARSET.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: django-unicode conversion, first experiments, questions

[Ivan Sagalaev]

Simon Willison wrote:
> We shouldn't be decoding QUERY_STRING in request.META at all - we  
> should leave it as urlencoded ASCII. request.META is meant to give  
> you access to the 'raw data' from the browser.
> 
> We do however need to take charset stuff in to account when creating  
> the request.GET and request.POST arrays - that's where the query  
> string should be parsed and unencoded and turned in to a set of name/ 
> value unicode strings.

Yes, this is what I mean. Sorry for not clarifying that it's not about 
META. I just used QUERY_STRING as a source for GET data.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: django-unicode conversion, first experiments, questions

[Ivan Sagalaev]

Simon Willison wrote:
> request.META should contain unicode strings that directly reflect the  
> underlying raw bytestrings

I just thought that may be they shouldn't. If META is a reflection of 
CGI's environment that is derived from HTTP environment that is 
essentially in byte strings then I think META being unicode is may be 
useless and misleading.

Instead the META can be declared as a low-level stuff that is needed 
only in custom cases. But we also have a public API: request.GET, 
request.POST, request.method. This API reflects the user data 
(urldecoded params, etc.) and should be unicode.

What do you think of this idea?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: django-unicode conversion, first experiments, questions

[Ivan Sagalaev]

Simon Willison wrote:
> I don't see any harm in META using unicode strings, whereas  
> if it were to use bytestrings our documentation ends up being that  
> little bit more confusing (we can't just claim everything is a  
> unicode string).

We can't do it anyway. First example is "raw_post_data" that should be 
in bytes and on the other and are files passed to HttpResponse.

> If there are any practical disadvantages to META containing unicode  
> strings

I confess I can't invent any real disadvantages from the top of my head. 
Unless may be the inconsistency between raw form data submitted via GET 
being in unicode while submitted via POST - in byte string.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: MS-SQL server LIMIT/OFFSET implementation

[Ivan Sagalaev]

DanH wrote:
> 
> Is someone working on MS-SQL support?
> I would like to help if possible with the implementation and testing.

Yes, testing is the main thing that holds it. The patch was developed 
sporadically and as far as I remember all the theoretical problems were 
known how to solve. It's just needed someone to dedicate the time to 
test it.

P.S. I'm not the one of the core devs so it makes sense anyway to wait 
an answer from Jacob or Adrian.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Django sprint / 0.95 release at OSCON

[Ivan Sagalaev]

Adrian Holovaty wrote:
> Most -- all? -- of the magic-removal stuff has settled down

What about model inheritance? This is one feature that was before MR and 
never recovered after. I remember some people in django-users were 
forced to stay with pre-MR syntax because of it.

I also remember that Malcolm was deep in work on it...

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

User subclassing

[Ivan Sagalaev]

James Bennett wrote:
> Which I've never understood. Subclassing User, even when model
> inheritance works again, will be probably the most difficult, least
> efficient and least rewarding way of extending User...

But nobody knows this beforehand. The feature is called "subclassing" 
and looks like it should fit perfectly when one wants to add some 
additional fields for users.

Again, having the solution with OneToOne relation scares newbies away 
because it's become a kind of common wisdom that OneToOne will be 
changed very drastically very soon and "all your code would burn".

I was extending User model before I've read about the "profile" thing 
[1] and I did it in a way that seamed natural: created my own "User" and 
was trying to refer to it all over the place as ForeignKey. Now I 
realize that I've made a mistake because I have to go back and forth 
over this relation all over the code. I believe this is a common 
misconception.

In short, my point is that newbies being Python programmers are familiar 
with extending classes but don't know about user's profiles supported in 
Django.

[1]: 
http://www.b-list.org/weblog/2006/06/06/django-tips-extending-user-model

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Model inheritance API

[Ivan Sagalaev]

First of all: Malcolm, this looks damn cool and very well thought out! 
Thank you!

Malcolm Tredinnick wrote:
> ---
> 2. "Pythonic" Inheritance
> 
> ...
> 
> (in the above classes, Animal
> would have a foreign key reference to the Thing table, similarly for
> Toy).

Why foreign key? This means that data of one Thing object can be shared 
between two different Animal objects. That seems strange, I'd rather 
expected it to be one-to-one relation.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: after save() hook not called in admin?

[Ivan Sagalaev]

James Bennett wrote:
> While we're on the topic, the docs explicitly advise against doing
> 'len' on a QuerySet because of the potential memory overhead of
> loading all the objects, and recommend using the QuerySet's 'count'
> method instead. Would it make more sense, then, to internally change
> '__len__' on QuerySet to just return the result of 'self.count()' and
> remove this potential hazard altogether?

The distinction was made on purpose. Tag {% for %} evaluates length of 
an iterable by doing len() and if this would call .count() then in most 
common case all queries accessed from templates will execute two 
request: "select count" and then "select" to actually output things. Now 
it only makes one select when doing len() and then uses newly cached 
results for output.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Any way to halt big file uploads?

[Ivan Sagalaev]

Todd O'Bryan wrote:
> Would it be better to expose the file-like object that comes with a  
> file upload, rather than reading the file's whole content into memory  
> (or into the server's file system, if the patch gets checked in)?  
> It's easy to retain backward compatibility by just having a call to  
> FILES['file_upload']['content'] simply call FILES['file_upload'] 
> ['file_like_object'].read(), but a developer could, instead, decide  
> how large a file they're willing to allow someone to upload, upload  
> that many bytes, and then raise an exception if the file is bigger,  
> rather than waiting until the whole file is uploaded.

This is not really possible. An input stream contains MIMEd data that 
can contain many files. So to have a _full_ request.FILES you have to 
read it entirely.

In fact you don't want to cancel an upload looking at _files_, you want 
to look at the size of a raw stream. This idea is covered (nicely, I 
believe :-) ) in my proposal in that ticket: 
http://code.djangoproject.com/ticket/2070#change_34

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: bug in django.http.parse_file_upload?

[Ivan Sagalaev]

Todd O'Bryan wrote:
> But aren't backslashes legal characters in unix-like OSes, so that  
> the filename will get truncated if it has a backslash in it?

You're right. But it's better to have nice filenames from Windows 
clients in most cases with a rare occasion of truncated filenames 
uploaded from Unix than the other way around (IMHO).

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: django unicode-conversion, beginning

[Ivan Sagalaev]

First of all, Gabor, thank you very much for doing this!

gabor wrote:
> today i experimented a little with the django source code,
> and here are the results.
> 
> if you apply a very small patch (65lines, attached), you can write a view
> completely in unicode.
> means:
> - GET/POST contains unicode data
> - request.META contains unicode data
> - you can put unicode text into the HttpResponse (this was already possible
> without the patch)

Here's a problem that I didn't know how to solve last time this topic
was discussed.

You can put unicode in HttpResponse. Does it imply that template 
processing should be done in unicode too? I mean, should context data
be in unicode? This would be convenient later because we will get all
the data from DB in unicode also. But this poses a problem of encoding
of actual template files.

We need to know the encoding of a template file. This can be done by
just mandating that they should be in settings.DEFAULT_CHARSET or we
should create a new setting (TEMPLATE_CHARSET). The reason of having
two different settings is that enforcing default UTF-8 in templates
means enforcing people to use unicode-aware text editors that are not
that common.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: If there was massive security hole found in Django, are there plans in place to deal with it?

[Ivan Sagalaev]

James Bennett wrote:
> One would hope that anyone who's using Django is subscribed to
> django-users and/or watches the Django blog

This would be less and less true as time goes because Django will spread 
beyond early adopters to a new forming local communities. For example 
there is russian Django's LiveJournal community where I suspect many 
people read just that list and not django-users because either they're 
already happy with local list or just don't feel comfortable enough with 
English to read it on a dialy basis.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: django unicode-conversion, beginning

[Ivan Sagalaev]

Malcolm Tredinnick wrote:
> I completely agree this is painful and normally I would punt. But my
> crystal ball tells me that you will then get bug reports from Mr
> Sagalaev, who is generally both very diligent in his debugging and likes
> to use some language with a funny alphabet. If whatever you come up with
> works naturally in places like Ivan's setup and maybe somebody who lives
> in Hong Kong or Japan or some other East Asian locale, you could
> consider this "solved" to some extent.

I'm afraid I'm not very good tester with this exact problem. Python on 
my Ubuntu happily says 'UTF-8' when asked 
'locale.getpreferredencoding()'. But indeed I can always try these 
things with my compatriots using Windows or configuring their linuxes 
with old single-byte 'KOI8-R'.

In fact I was under impression that a string returned from this function 
can be safely used for decoding. For example on Russian Windows it 
returns 'cp1251' which works perfectly well while not being a standard 
ISO name which is 'windows-1251' and works well also.

So may be we can just rely on Python's smart little brain and do 
something like this:

- try decoding from locale.getpreferredencoding()
- failing that try something safe like iso-8859-1

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: django unicode-conversion, beginning

[Ivan Sagalaev]

gabor wrote:
> hmmm.. are you sure that the situation with unicode-aware editors is so bad?
> 
> could you name some non-unicode-aware editors?
> for me it seems that from notepad through vim to eclipse everything does 
> unicode fine...

Ok, I should rephrase it. Even if most editors do support utf-8 they 
aren't configured to do so by default. Unfortunately there is some 
notion that unicode is something "new" and "scary" and "who knows what 
problems it will cause". So there is a case when on systems where utf-8 
is not default environment setting (meaning all Windows and many 
Linuxes) if a programmer starts his favorite text editor odds are that 
it will not save a new file in utf-8.

But to be sure I'll better run a poll on my forum about it...

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---