Re: Use case for #14914 (to_db_python)

[Alejandro Dubrovsky]

Thanks. I've had a look at GeoDjango and it did help. I've hacked something 
that works well enough for my purposes, but it assumes that the default 
connection is the one holding the data.

I agree with you that it would be useful if the data mangling/demangling 
stage would be more easily overridable.

On Monday, August 5, 2013 3:02:52 PM UTC+10, Jani Tiainen wrote:
>
> Hi, 
>
> You seem to found kind of an issue which happens with GeoDjango part as 
> well. Most of the geodjango operations require quite heavy to/from data 
> mangling while reading and/or writing data. 
>
> Currently there isn't clean solution to tell (per field) how data should 
> be treated per backend. Django ORM works pretty well for a primitives like 
> numbers, strings and such but when it goes to complex datatypes (like your 
> encrypted field). 
>
> It would be really useful to have something to allow data mangling on a 
> when reading/writing data from/to database per backend basis. Unfortunately 
> such a feature isn't easy to implement due the current way how ORM works. 
>
> If you require such a functionality now, you should take a look how 
> different GeoDjango backends deal with the similiar problem. 
>
> On Thu, 1 Aug 2013 21:23:40 -0700 (PDT) 
> Alejandro Dubrovsky > wrote: 
>
> > Looking around for a way to get the connection on deserialisation, I ran 
> > into #14914  which was 
> closed 
> > wontfix 3 years ago with a request for a use case for the change. 
> > 
> > Here is my use case: 
> > 
> > I'm writing an encrypted char field, with encryption happening at the 
> > database end (MS-SQL). Decryption looks a bit like this: 
> > 
> >  
> > OPEN SYMMETRIC KEY keyname 
> > DECRYPTION BY CERTIFICATE somecertificate 
> > 
> > SELECT CAST(DECRYPTBYKEY(fieldname) AS VARCHAR(2048)) 
> > FROM atable 
> > 
> > CLOSE SYMMETRIC KEY keyname 
> >  
> > 
> > and analogously for the encryption. 
> > 
> > The way I thought of doing that was by modifying get_db_prep_value for 
> > encryption and to_python for decryption, but I ran into the lack of 
> > connection at the to_python stage, and nothing like to_db_python. 
> > Does this constitute a legitimate use case for to_db_python or is there 
> a 
> > better way to go about this? 
> > Note: I'd prefer if any discussion would stay away from the validity of 
> > using DB-based symmetric encryption on specific fields. 
> > 
> > In the more general sense, I'd expect the method to be useful for any 
> > stored-procedure-heavy location where the extracted field has to be 
> > post-processed by some function that runs on the database to be made 
> sense 
> > of. 
> > 
> > Thanks 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "Django developers" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to django-develop...@googlegroups.com . 
> > To post to this group, send email to 
> > django-d...@googlegroups.com. 
>
> > Visit this group at http://groups.google.com/group/django-developers. 
> > For more options, visit https://groups.google.com/groups/opt_out. 
> > 
> > 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Use case for #14914 (to_db_python)

[Alejandro Dubrovsky]

On Wednesday, August 7, 2013 5:22:29 PM UTC+10, Anssi Kääriäinen wrote:
>
> On Monday, August 5, 2013 8:02:52 AM UTC+3, Jani Tiainen wrote:
>>
>> Hi, 
>>
>> You seem to found kind of an issue which happens with GeoDjango part as 
>> well. Most of the geodjango operations require quite heavy to/from data 
>> mangling while reading and/or writing data. 
>>
>> Currently there isn't clean solution to tell (per field) how data should 
>> be treated per backend. Django ORM works pretty well for a primitives like 
>> numbers, strings and such but when it goes to complex datatypes (like your 
>> encrypted field). 
>>
>> It would be really useful to have something to allow data mangling on a 
>> when reading/writing data from/to database per backend basis. Unfortunately 
>> such a feature isn't easy to implement due the current way how ORM works. 
>>
>> If you require such a functionality now, you should take a look how 
>> different GeoDjango backends deal with the similiar problem. 
>>
>
> I agree that there is room for improvement in how data conversions in the 
> ORM work. If I recall correctly different backends work slightly 
> differently, and when to_python() is called isn't consistent. Improvements 
> in this area are welcome.
>
> For the original use case I think it would be better to do the encryption 
> already in the original SELECT. I am not sure if there is any easy way to 
> do that currently...
>
>  - Anssi
>

Yes, doing it in the SELECT would be even better in this case, since then 
the encrypted data wouldn't ever have to reach Python, but I didn't see any 
easy way to do that (although I must admit I was trying to steer clear of 
the SQL compiler section) 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: django.utils.functional.cached_property

[Daniele Procida]

On Thu, Aug 8, 2013, Russell Keith-Magee  wrote:

>One suggestion -- a new special topic guide on performance tweaking



I'd like to take that on, if unless anyone else feels they should be 
responsible for it, since it will involve a certain amount of structural change 
in the documents.

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

InterfaceError: connection already closed Django 1.5

[mikola1717]

Hello! Sorry for bad English.

I met the following error:
Traceback (most recent call last):
  File 
"/usr/local/lib/python2.7/dist-packages/django/core/handlers/wsgi.py", line 
255, in __call__
response = self.get_response(request)
  File 
"/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 
177, in get_response
signals.got_request_exception.send(sender=self.__class__, 
request=request)
  File 
"/usr/local/lib/python2.7/dist-packages/django/dispatch/dispatcher.py", 
line 170, in send
response = receiver(signal=self, sender=sender, **named)
  File "/usr/local/lib/python2.7/dist-packages/django/db/__init__.py", line 
68, in _rollback_on_exception
transaction.rollback_unless_managed(using=conn)
  File "/usr/local/lib/python2.7/dist-packages/django/db/transaction.py", 
line 143, in rollback_unless_managed
connection.rollback_unless_managed()
  File 
"/usr/local/lib/python2.7/dist-packages/django/db/backends/__init__.py", 
line 232, in rollback_unless_managed
self._rollback()
  File 
"/usr/local/lib/python2.7/dist-packages/django/db/backends/__init__.py", 
line 59, in _rollback
return self.connection.rollback()
psycopg2.InterfaceError: connection already closed

The unit of a code where arises an error looks as follows:def 
__call__(self, environ, start_response):
# Set up middleware if needed. We couldn't do this earlier, because
# settings weren't available.
if self._request_middleware is None:
with self.initLock:
try:
# Check that middleware is still uninitialised.
if self._request_middleware is None:
self.load_middleware()
except:
# Unload whatever middleware we got
self._request_middleware = None
raise

set_script_prefix(base.get_script_name(environ))
signals.request_started.send(sender=self.__class__)
try:
request = self.request_class(environ)
except UnicodeDecodeError:
logger.warning('Bad Request (UnicodeDecodeError)',
exc_info=sys.exc_info(),
extra={
'status_code': 400,
}
)
response = http.HttpResponseBadRequest()
else:
response = 
self.get_response(request) the error 
arises here

response._handler_class = self.__class__

try:
status_text = STATUS_CODE_TEXT[response.status_code]
except KeyError:
status_text = 'UNKNOWN STATUS CODE'
status = '%s %s' % (response.status_code, status_text)
response_headers = [(str(k), str(v)) for k, v in response.items()]
for c in response.cookies.values():
response_headers.append((str('Set-Cookie'), 
str(c.output(header=''
start_response(force_str(status), response_headers)
return response


In Django 1.4 it looked:
def __call__(self, environ, start_response):
# Set up middleware if needed. We couldn't do this earlier, because
# settings weren't available.
if self._request_middleware is None:
self.initLock.acquire()
try:
try:
# Check that middleware is still uninitialised.
if self._request_middleware is None:
self.load_middleware()
except:
# Unload whatever middleware we got
self._request_middleware = None
raise
finally:
self.initLock.release()

set_script_prefix(base.get_script_name(environ))
signals.request_started.send(sender=self.__class__)
try:
try:
request = self.request_class(environ)
except UnicodeDecodeError:
logger.warning('Bad Request (UnicodeDecodeError)',
exc_info=sys.exc_info(),
extra={
'status_code': 400,
}
)
response = http.HttpResponseBadRequest()
else:
response = self.get_response(request)
finally:
signals.request_finished.send(sender=self.__class__)

try:
status_text = STATUS_CODE_TEXT[response.status_code]
except KeyError:
status_text = 'UNKNOWN STATUS CODE'
status = '%s %s' % (response.status_code, status_text)
response_headers = [(str(k), str(v)) for k, v in response.items()]
for c in response.cookies.values():
response_headers.append(('Set-Cookie', 
str(c.output(header=''
start_response(status, response_headers)
return response


That is the unit of a code was in section try...finally(in finnaly signal 
was excited signals.request_finished), n

Re: django.utils.functional.cached_property

[Russell Keith-Magee]

On Thu, Aug 8, 2013 at 7:40 PM, Daniele Procida  wrote:

> On Thu, Aug 8, 2013, Russell Keith-Magee  wrote:
>
> >One suggestion -- a new special topic guide on performance tweaking
>
> 
>
> I'd like to take that on, if unless anyone else feels they should be
> responsible for it, since it will involve a certain amount of structural
> change in the documents.


As far as I'm concerned -- go for it. It's not like people are lining up to
do a big rewrite of the docs; if you're volunteering, it's all yours.

Russ %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: InterfaceError: connection already closed Django 1.5

[Aymeric Augustin]

2013/8/8 
>
> That is the unit of a code was in section try...finally(in finnaly signal
> was excited signals.request_finished), now it isn't present.
> There is a feeling that in case of a code redoing it simply forgot to
> consider.
> If everything is made intentionally I ask to report about it (if it is
> possible the link to documentation)ю
>


Hello,

I purposefully moved the request_finished signal from just after the view
function returns to after the response is fully generated.

I documented this change here:
https://docs.djangoproject.com/en/1.5/releases/1.5/#request-finished-signal

How are you triggering this exception?

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: InterfaceError: connection already closed Django 1.5

[mikola1717]

The exception is excited after connection with a database was lost and 
again regenerated(I start the application, execute a command postgresql 
restart, refresh the page of application and receive this exception)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: InterfaceError: connection already closed Django 1.5

[Aymeric Augustin]

2013/8/8 

> The exception is excited after connection with a database was lost and
> again regenerated(I start the application, execute a command postgresql
> restart, refresh the page of application and receive this exception)


Have you turned on persistent connections?

If so, you're seeing the expected behavior.

With persistent connections, if memory serves, the first user hitting a
page after a database restart will encounter an error.

There are rather fundamental reasons why this is hard to fix. (Hard doesn't
mean impossible.)

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: django.utils.functional.cached_property

[Łukasz Rekucki]

Hi,

I have some minor nitpicks:

1. Unlike the standard @property, the current implementation of
@cached_property doesn't allow for a docstring.
2. Calling `del obj.` before accessing the value
or more then once in a row throws an AttributeError.

Should I make a new ticket for that or just send a PR for the old one ?

-- 
Łukasz Rekucki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Security Advisory: BREACH and Django

[Collin Anderson]

>
> I am doing something a little different with my CSRF tokens, and I believe 
it guards against BREACH.

Instead of sending the token in the HTTP response, I am using javascript to 
read (and generate if needed) the CSRF token cookie. The javascript reads 
the token from the cookie and adds it as a hidden field to any forms that 
need it on the page.

This also has two bonus benefits:
- It makes pages easier to cache, because the response doesn't Vary based 
on each person's CSRF Cookie.
- I've been spam-free since I started using this method. Most spam bots 
don't aren't aware of this method (yet).

It could be used as a drop-in replacement for {% csrf_token %} for anyone 
who doesn't care about the javascript requirement. I personally consider 
the javascript requirement a benefit.

This also, of course, is incompatible with CSRF_COOKIE_HTTPONLY.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Proposal: ./manage check to look for models.BooleanField() without default= value.

[Collin Anderson]

The new ./manage check command is for helping people upgrade to a new 
version of django.
https://docs.djangoproject.com/en/dev/ref/django-admin/#check

django1.6 changes models.BooleanField() to not default to False any more. 
This has already bitten me a few times.
https://docs.djangoproject.com/en/dev/releases/1.6/#booleanfield-no-longer-defaults-to-false

Can we add a warning about BooleanFields without default values to the 
"check" command?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Security Advisory: BREACH and Django

[Tim Chase]

On 2013-08-08 09:59, Collin Anderson wrote:
>> I am doing something a little different with my CSRF tokens, and
>> I believe it guards against BREACH.
> 
> Instead of sending the token in the HTTP response, I am using
> javascript to read (and generate if needed) the CSRF token cookie.
> The javascript reads the token from the cookie and adds it as a
> hidden field to any forms that need it on the page.
> 
> This also has two bonus benefits:

but also has the downside that it doesn't work if JS is disabled.
You may or may not care, but it's at least something to consider
(JS-free interactions may come from older devices, or from testing
tools, etc; so I've found enough benefit to code for JS-free and then
add in additional functionality).

-tkc


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Deprecate admindocs?

[Collin Anderson]

Looking at the code, we're only using docutils for parsing reST markup in 
docstrings. What if we made reST parsing (and docutils) optional? Seems to 
me admindocs would still be plenty helpful without it.

>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Proposal: ./manage check to look for models.BooleanField() without default= value.

[Matthew Lauber]

Maybe do this only for 1.6/1.7 to give a grace period to the change?

On Thu, Aug 8, 2013 at 1:25 PM, Collin Anderson wrote:

> The new ./manage check command is for helping people upgrade to a new
> version of django.
> https://docs.djangoproject.com/en/dev/ref/django-admin/#check
>
> django1.6 changes models.BooleanField() to not default to False any more.
> This has already bitten me a few times.
>
> https://docs.djangoproject.com/en/dev/releases/1.6/#booleanfield-no-longer-defaults-to-false
>
> Can we add a warning about BooleanFields without default values to the
> "check" command?
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Proposal: ./manage check to look for models.BooleanField() without default= value.

[Russell Keith-Magee]

On Fri, Aug 9, 2013 at 1:25 AM, Collin Anderson wrote:

> The new ./manage check command is for helping people upgrade to a new
> version of django.
> https://docs.djangoproject.com/en/dev/ref/django-admin/#check
>
> django1.6 changes models.BooleanField() to not default to False any more.
> This has already bitten me a few times.
>
> https://docs.djangoproject.com/en/dev/releases/1.6/#booleanfield-no-longer-defaults-to-false
>
> Can we add a warning about BooleanFields without default values to the
> "check" command?
>

Sounds like a good idea to me - this sort of change is exactly what the
check command is supposed to be identifying (wherever possible). Ticket and
patch welcome!

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.