Re: Django should not use `force_unicode(..., errors='replace')` when parsing POST data.

2012-03-29 Thread Tai Lee
I agree that it is not an ideal user experience to raise an exception while decoding POST data. However, I think the alternatives are arguably just as bad, or even worse. I consider silently replacing characters in user data to avoid an exception while decoding to be a silent data loss / corrup

Re: Django should not use `force_unicode(..., errors='replace')` when parsing POST data.

2012-03-29 Thread Mike Axiak
It should also be noted that it is a terrible user experience to raise an exception in the middle of parsing the multipart parser---a socket error with no error message. Not to say that you can't slurp in the data and raise the exception afterwards, but care has to be given if that strategy is take

Re: Django should not use `force_unicode(..., errors='replace')` when parsing POST data.

2012-03-29 Thread Alex Ogier
On Mar 29, 2012 5:43 AM, "Tai Lee" wrote: > > I've just created an essay *cough* I mean ticket about the way Django > silently mangles POST data that is incorrectly encoded, and how this can send > someone (like me) down the rabbit hole trying to debug it, thanks to "current > transaction abort

Re: Django should not use `force_unicode(..., errors='replace')` when parsing POST data.

2012-03-29 Thread Waylan Limberg
Tai, The idea behind errors='replace' is that any chars that cannot be properly decoded as unicode will be replaced with an acceptable unicode char (something like '?'). So, if you want to avoid crashing your program on bad input, this is the only acceptable approach and, I assume, why the current

Django should not use `force_unicode(..., errors='replace')` when parsing POST data.

2012-03-29 Thread Tai Lee
I've just created an essay *cough* I mean ticket about the way Django silently mangles POST data that is incorrectly encoded, and how this can send someone (like me) down the rabbit hole trying to debug it, thanks to "current transaction aborted" database errors. This shouldn't normally happen,