Re: RFC: Django history tracking

[william]

Uros Trebec wrote:
> Hi, everyone!
>
> First: introduction. My name is Uros Trebec and I was lucky enough to
> be
> selected to implement my idea of "history tracking" in Django. I guess
> at least some of you think this is a very nice feature to have in web
> framework, so I would like to thank you all who voted for my Summer Of
> Code proposal! Thank you!
>
> Ok, to get right to the point: this is a Request For Comment. I would
> like to know
> what you think about my idea for implementation and how can I make it
> better. Here is what I have in mind so far...
>
> (Just for reference: http://zabica.org/~uros/soc/ . Here you can find
> my initial project proposal and some diagrams.)
>
>
> 1. PROPOSAL:
> Main idea is to create a way to have a content history for every
> change in a Model. Current changes tracking is very limited to say the
> least, so I will extend/replace that so one could actually see how
> something was changed.
>
> 1.1 SCOPE:
> Changes will have to be made in different parts of Django. Most of
> the things should be taken care of inside django.db, except diff-ing
> and merging.
>
>
> USAGE
>
> 2. MODELS:
> The easiest way to imagine how stuff will work is to have an actual
> usage case. So, let's see how Bob would use this feature.
>
> 2.1. Basic models:
> To enable history tracking Bob has to create a sub-class for those
> models that he will like to track:
>
>   class Post(models.Model):
>   author = models.CharField(maxlength=100)
>   title = models.CharField(maxlength=100)
>   content = models.TextField()
>   date = models.dateField()
>
>   class History:
>   pass
>
> This works much like using Admin subclass. The difference is that if
> the subclass is present then database will have change to include two
> tables for this class:
>
> (the main table - not changed):
>
>   CREATE TABLE app_post (
>   "id" serial NOT NULL PRIMARY KEY,
>   "author" varchar(100) NOT NULL,
>   "title" varchar(100) NOT NULL,
>   "content" text NOT NULL,
>   "date" datestamp NOT NULL
>   );
>
>
> (and the history table):
>
>   CREATE TABLE app_post_history (
>   "id" serial NOT NULL PRIMARY KEY,
>   "change_date" datestamp NOT NULL,   # required for datetime
> revert
>   "parent_id" integer NOT NULL REFERENCES app_post (id),
>   "author" varchar(100) NOT NULL, # data from app_post
>   "title" varchar(100) NOT NULL,  # data from app_post
>   "content" text NOT NULL,# data from app_post
>   "date" datestamp NOT NULL   # data from app_post
>   );
>
> I think this would be enough to be able to save "basic full" version of
> changed record. "parent_id" is a ForeignKey to app_post.id so Bob can
> actually find the saved revision for a record from app_post and when he
> selects a record from _history he knows to which record it belongs.
>
>
> 2.2. Selective models:
> But what if Bob doesn't want to have every information in history (why
> would someone like to keep an incomplete track of a record is beyond
> me, but never mind)? Maybe the 'author' and 'date' of a post can't
> change, so he
> would like to leave that out. But at the same time, he would like to
> know who made the change, but does not need the information when
> using Post.
>
> Again, this works like Admin subclass when defining which fields to
> use:
>
>   class Post(models.Model):
>   author = models.CharField(maxlength=100)
>   title = models.CharField(maxlength=100)
>   content = models.TextField()
>   date = models.dateField()
>
>   class History:
>   track = ('title', 'content')
>   additional = {
>  "changed_by": 
> "models.CharField(maxlength=100)
>   }
>
>
> In this case "app_post_history" would look like this:
>
>   CREATE TABLE app_post_history (
>   "id" serial NOT NULL PRIMARY KEY,
>   "change_date" datestamp NOT NULL,   # required for datetime
> revert
>   "parent_id" integer NOT NULL REFERENCES app_post (id),
>   "title" varchar(100) NOT NULL,  # data from app_post
>   "content" text NOT NULL,# data from app_post
>   "changed_by" varchar(100) NOT NULL  # new field
>   );
>
>
>
> 3. VIEWS
> 3.1. Listing the change-sets:
> Ok, so after a few edits Bob would like to see when and what was
> added/changed in one specific record.
>
> A view should probably work something like this:
>
>   from django.history import list_changes
>
>   def show_changes(request, post_id):
>   list = list_changes(post_id)
>   return render_to_response('app/changes.html', 
> {'p

Re: RFC: Django history tracking

[william]

Uros Trebec wrote:
> > Sounds nice, this is a feature I'm currently looking for... but I've
> > already started my own implementation.
>
> Nice! Do you have anyting in code yet? Any bottlenecks?
>

sorry not yet. But will come, I need it for my current development.

>
> > I would just share it with you.
> >
> > I've build a single table History with :
> > - "change"; a  text field which will contain a python pickled
> > dictionary: { field: old_value} in case you update a record.
>
> How does this help/work? Why dictionary? Can you explain?
>

I should add first that my history table can contain history of any
other table. In fact I don' t have an history table per "master" table.

This is a kind of "generic" history, thus I don't know the type of data
I will save in it. I solve this by pickling the data and save them into
a textField, that's the goal of the "change" field.
For flexibility reasons, I let possibility to select the field people
want to have in their history table.

Does this is better explained ?


> > - type: type of modification (update, delete, insert).
>
> Is this really necesary? How do you make use of it?
>

I take inspiration from svn log files. Sure this is necessary!!.
History is a kind of record trace from his creation up to his delete
(if occurs). But I'm agreed that most of the case, "insert" and
"update" are really useful. Concerning the "delete", this would be
useful if you want to have "undo" functionalities.



> > - "obj": the table object. This can come from ContentType
>
> I don't understand...
>

As I've explained, my "history table" can record history data of any
table I'm interested. Thus "obj" and "obj_id" give me possibility to
build a link between the history and the "master table".


> > - "obj_id": the id of the impacted object.
> > - create_date: a timestamp automatically set.
>
>
> > I'm using it by sub-classing the save methods in each model I want to
> > see the history.
> > This is quite flexible, because you can decide which field you want to
> > track.
>
> I agree. But I fail to see the need for not versioning the whole record/row.
>
>
> > To facilitate, yet one step further, it would be nice to have a
> > PickledField within Model.models of django.
>
> Can you elaborate on that?

behind the scene, Django will unpickle and pickle when you access the
data in a PickleField. This will avoid to have a model like this:
class History(models.Model):
   change = TextField()
   def get_change(self):
   return loads(change)
   def set_change(self,data):
   change = dumps(data)

Imagine you have 3 fields with Pickling functionalities in the same
table ...



>
>
> > Feedbacks are welcome.
>
> Same here! :) And thanks for your feedback!
>

Thanks for the questions. I hope I'm a bit clearer ;-).
I hope I'll have a bit time this week to tackle this problem, and come
with real code.

William


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: auth.User refactor: reboot

[william ratcliff]

A bit of an odd question--

What practices have other packages adopted?  For example, I was playing
with ASP.NET last weekend to help out a friend and found their decoupling
of authentication from the database to be rather pleasant.  Is it possible
to add a layer of indirection.   In some sense, I want to know:
1)  What is the identify of this "putative" user
2)  If their Identity is verified, what resources are they Authorized
access to.

To determine identity, in the current era, there are many ways--for example:
single factor:
1)  username/password
2)  email address/password

multifactor:
1)  username/password  + PIV card
2)  username/password + RSA token

Depending on the credential, it can be checked in multiple ways:
active directory
database
etc.

To facilitate these different cases, can we have a pluggable
"Authentication Service" where the user provides a "credential" which then
returns a result of whether the credential is valid or not?We could
then implement a few "standard" authentication plugins, such as
username/password, email address/password. Third party apps would run
against the authentication service, so they shouldn't depend on a
particular implementation of user in the database.For legacy users, the
current username/password could be implemented and run as the default , so
they only have to migrate if they want to...The settings file could be used
to register authentication "providers"  (for example EmailAuth or
UsernameAuth) with the authentication service.  If there are multiple
authentication methods (users can sign in  via openauth or email address
for example), then they would fallover in the order they are registered
with the authentication service (using settings.py might require some
effort to preserve order and avoid multiple registrations...).


Is this workable, or would it have too much complexity and too much of a
performance hit?  Sorry to be so vague...One issue that would come to mind
with such an approach is how to deal with things like password recovery in
such a scheme using DRY principles...


Best,
William


On Tue, Mar 20, 2012 at 2:02 PM, Donald Stufft wrote:

> On Tuesday, March 20, 2012 at 2:00 PM, Tom Evans wrote:
>
> On Tue, Mar 20, 2012 at 5:41 PM, Donald Stufft 
> wrote:
>
> What Alex said. If it was _just_ the username then you'd have a good
> argument for
> a setting like that. However there's username, email, some people want to
> add
> fields, some want to get rid of, or combine fields. Ideally any change
> would
> work
> for as many of those use cases without turning into a giant set of boolean
> flags
> that drastically alter the DB schema.
>
>
> No-one is talking about a 'giant set of boolean flags', or drastically
> altering the DB schema.
>
> The email field needs to accept email addresses of up to 254 octets.
> The username field should be able to hold common fields used as a
> username, eg emails, and hence needs to be longer.
>
> That is one boolean flag. No more. If you want more flexibility on the
> model, you would use the proposed pluggable auth models.
>
> What the much maligned setting gives us is a way to provision new
> sites and migrate existing sites that care about that situation in to
> a position where users with long email addresses can sign up to a
> django site. Score.
>
> Cheers
>
> Tom
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>  So you fix the problem for the people who agree that username should be
> longer, and that email
> should be longer, but not the people who feel that email should be longer
> but username is fine?
>
> Those settings do not feel any cleaner to me personally than monkey
> patching the models.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #16779 - a tutorial for first time Django contributors

[william ratcliff]

I develop on windows, linux, and macos--for windows, I have to say that I
tend to use tortoise-git (somehow, I still prefer it to github for
windows), whereas for linux, the command line is greatFrom my
experience leaping between platforms, it's rather painful to try to
shoehorn the way of doing things on one platform into another...It's more
work, but perhaps a linux/windows section would work best for those areas
that are more platform specific (for example, environment variables in
windows, vs linux)?

Anywho, just my 2 cents...


On Sun, Nov 11, 2012 at 4:17 AM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> Le 11 nov. 2012 à 06:53, Shai Berger  a écrit :
>
> > On Sunday 11 November 2012, Tim Graham wrote:
> >>
> >> I think the part that has the most potential to confuse new
> contributors is
> >> the introduction of PYTHONPATH.  Claude suggested we could simply
> instruct
> >> users to run the tests like so:
> >>
> >> PYTHONPATH=/path/to/django ./run_tests.py --settings=test_sqlite
> >>
> >> I'm not particularly in love with that, but it would eliminate the need
> to
> >> try to explain things
>
> I've always been running the tests with:
>
> $ cd tests
> $ PYTHONPATH=.. pythonX.Y runtests.py --settings=test_
>
> It's straightforward and easy to understand: "Python will look for django
> in the parent directory".
>
> If you're just running Django's test suite on a reasonably configured
> system, you're starting with an empty PYTHONPATH; you don't really need
> PYTHONPATH=..:$PYTHONPATH.
>
> The alternatives are:
> - either prone to mistakes and side effects (setting a systemwide
> PYTHONPATH — what if I move my checkout?);
> - or even more complicated to explain (mkvirtualenv djang && pip install
> -e .)
>
>
> > It would leave a lot to explain to Windows users (which I note you are
> still
> > trying to cater for).
>
>
> If you're using the default options of the git installer on Windows,
> you're getting a fairly decent environment (MINGW32). It creates a "Git
> Bash" icon on the desktop, which starts a Bash shell where `git` works.
> After adding `export PATH=/c/Python27:$PATH` to `~/.bashrc`, `python` also
> works in that shell.
>
> If we tell Windows users to use "Git Bash", we can skip most of the
> Windows-specific instructions. It's likely to make the tutorial a better
> experience for them.
>
> Otherwise, `set PYTHONPATH=..` works in `cmd.exe`, but I can't recommend
> `cmd.exe` with a straight face :/
>
> --
> Aymeric.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #16779 - a tutorial for first time Django contributors

[william ratcliff]

And I should say, thanks for the effort on this!


On Sun, Nov 11, 2012 at 10:18 AM, william ratcliff <
william.ratcl...@gmail.com> wrote:

> I develop on windows, linux, and macos--for windows, I have to say that I
> tend to use tortoise-git (somehow, I still prefer it to github for
> windows), whereas for linux, the command line is greatFrom my
> experience leaping between platforms, it's rather painful to try to
> shoehorn the way of doing things on one platform into another...It's more
> work, but perhaps a linux/windows section would work best for those areas
> that are more platform specific (for example, environment variables in
> windows, vs linux)?
>
> Anywho, just my 2 cents...
>
>
> On Sun, Nov 11, 2012 at 4:17 AM, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
>
>> Le 11 nov. 2012 à 06:53, Shai Berger  a écrit :
>>
>> > On Sunday 11 November 2012, Tim Graham wrote:
>> >>
>> >> I think the part that has the most potential to confuse new
>> contributors is
>> >> the introduction of PYTHONPATH.  Claude suggested we could simply
>> instruct
>> >> users to run the tests like so:
>> >>
>> >> PYTHONPATH=/path/to/django ./run_tests.py --settings=test_sqlite
>> >>
>> >> I'm not particularly in love with that, but it would eliminate the
>> need to
>> >> try to explain things
>>
>> I've always been running the tests with:
>>
>> $ cd tests
>> $ PYTHONPATH=.. pythonX.Y runtests.py --settings=test_
>>
>> It's straightforward and easy to understand: "Python will look for django
>> in the parent directory".
>>
>> If you're just running Django's test suite on a reasonably configured
>> system, you're starting with an empty PYTHONPATH; you don't really need
>> PYTHONPATH=..:$PYTHONPATH.
>>
>> The alternatives are:
>> - either prone to mistakes and side effects (setting a systemwide
>> PYTHONPATH — what if I move my checkout?);
>> - or even more complicated to explain (mkvirtualenv djang && pip install
>> -e .)
>>
>>
>> > It would leave a lot to explain to Windows users (which I note you are
>> still
>> > trying to cater for).
>>
>>
>> If you're using the default options of the git installer on Windows,
>> you're getting a fairly decent environment (MINGW32). It creates a "Git
>> Bash" icon on the desktop, which starts a Bash shell where `git` works.
>> After adding `export PATH=/c/Python27:$PATH` to `~/.bashrc`, `python` also
>> works in that shell.
>>
>> If we tell Windows users to use "Git Bash", we can skip most of the
>> Windows-specific instructions. It's likely to make the tutorial a better
>> experience for them.
>>
>> Otherwise, `set PYTHONPATH=..` works in `cmd.exe`, but I can't recommend
>> `cmd.exe` with a straight face :/
>>
>> --
>> Aymeric.
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Adding flash objects

[William Marchand]

I believe you are asking for a template tag to handle Flash content better.

Tell me if I am wrong but it sounds like a good candidate for a third party 
app. I don't think it belongs in Django as a Django contribution.

On Wednesday, March 18, 2015 at 5:06:58 PM UTC-4, Debesh Mohanty wrote:
>
> Hello.
>
> Many web sites use flash contents so as to make it more attractive and 
> many online games are also flash games. Flash is used widely over the web. *I 
> am proposing to make a module in django which will handle importing flash 
> contents into web pages. *
>
> Flash contents can be imported using HTML but the HTML code will be bit 
> long. 
>
> But if this module is created the coding in django will be of only one or 
> two lines for importing flash works in a web page. It will be just like a 
> method taking arguments about the flash objects like size and alignment. So 
> coding will be too short and easy for the web developer
>
> And if this is successful this concept can be extended to other type of 
> files as well
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/451363a6-0677-4211-8ff0-4fb2a7d6dc32%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Feature idea: Add support for PostgreSQL's COPY command in contrib.postgres

[William Chambers]

+1 for this integration. This is a great feature that I've been using on 
projects as well. I'd love to be able to do it within django rather than 
hack my own sql scripts around it. The efficiency/speed up is very real. 

This solves the problem that when working with data where people replicate 
across machines (because of time/space/security/cost issues ) Sending a csv 
via box/dropbox and then loading it in with postgres COP is great way to 
make sure that everyone has the information they need in a way that they 
can query efficiently and build on using django!


On Saturday, July 18, 2015 at 3:19:49 PM UTC-7, Ben Welsh wrote:
>
> Hello,
>
> I am a big fan of PostgreSQL's excellent bulk loader COPY 
> . It can rapidly 
> load a CSV file directly into the database, an oftentimes tedious task that 
> I have to do frequently as part of my job. 
>
> I am not a big fan of having to write my COPY commands in raw SQL. I'd 
> much rather use Django's ORM. 
>
> So last week I put together an app call django-postgres-copy 
>  that 
> attempts to integrate COPY into Django, modeling its design on the 
> excellent LayerMapping 
>  
> utility in contrib.gis, which I also use frequently. 
>
> I wrote a blog post about the approach here
>
> http://www.californiacivicdata.org/2015/07/17/hello-django-postgres-copy/
>
> You can find more complete technical documentation here
>
> http://django-postgres-copy.californiacivicdata.org/en/latest/
>
> And all of the code is up here on GitHub
>
> https://github.com/california-civic-data-coalition/django-postgres-copy
>
> Since Django has already begun to integrate other PostgreSQL-specific 
> features in contrib.postgres, I'm curious if the core developers are be 
> interested in adding COPY support as well. 
>
> I'm not attached to the style of my library and I'd welcome a different 
> approach if it got the job done. I'd be thrilled to have the opportunity to 
> carry the torch and do whatever refactoring and additional coding is 
> necessary to qualify it for a merge.
>
> Please let me know what you think. And if I've overlooked some previous 
> discussion or superior third-party library in this area, please forgive me. 
> I searched around but was unable to find anything.
>
> Sincerely,
>
> Ben Welsh
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0a63377c-7cfe-440e-99fc-2c35592f2381%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Pluggable encryption for django auth (design proposal)

[William Ratcliff]

Hi!  I'm new to the list and have started to look into authentication.   I 
find that I will need to patch it for my own needs, but would like to ask 
the opinions of others who are more familiar with the code-base than I am. 
 I apologize if I make any mistakes in the protocol of the list in matters 
such as including too much code.

SHA1 is not secure.  This is not a nationalism issue.  For example:
http://www.darknet.org.uk/2010/11/sha-1-password-hashes-cracked-using-amazon-ec2-gpu-cloud/

Or, from NIST:
http://csrc.nist.gov/groups/ST/toolkit/secure_hashing.html

where the relevant excerpt is:
*March 15, 2006*: The SHA-2 family of hash functions (i.e., SHA-224, 
SHA-256, SHA-384 and SHA-512) may be used by Federal agencies for all 
applications using secure hash algorithms. Federal agencies should stop 
using SHA-1 for digital signatures, digital time stamping and other 
applications that require collision resistance as soon as practical, and 
must use the SHA-2 family of hash functions for these applications after 
2010. After 2010, Federal agencies may use SHA-1 only for the following 
applications: hash-based message authentication codes (HMACs); key 
derivation functions (KDFs); and random number generators (RNGs). Regardless 
of use, NIST encourages application and protocol designers to use the SHA-2 
family of hash functions for all new applications and protocols.

I have also seen discussions in other venues from people who are worried 
about the security of SHA1 in the event that their system is compromised, 
can an attacker gain the passwords in the database?  The appearance of 
modules such as django-bcrypt:
https://github.com/dwaiter/django-bcrypt/
show that this issue is becoming of more general concern.

Current solutions:
1)  Monkey patch:
put a top level installed_app that has a listener to the class_prepared 
signal that performs monkey patching throughout user.
This is rather ugly and it feels very fragile to me if the auth module 
changes internally.
2)  Rewrite the Backend as Tom suggests by subclassing ModelBackend:
Again, it's not sufficient.  Why?

If we look at the Model Backend, we see that yes, the authenticate method 
currently authenticates against User--but the problem is NOT the 
authentication per se, but rather that the User class has several methods 
such as:
check_password, set_password, etc. that have encryption hard coded.   There 
are admin commands associated with the User class which refer to methods 
with a particular encryption method chosen.

3) For users of Django who cannot (say US government agencies, people who 
have tight security concerns, etc.) use the current module, ignore the auth 
module and roll their own:
This has been attempted before.  However, the problem is that it is easy to 
make mistakes doing this and most of the functionality in the auth module is 
very good and simply copying most of it to make a few changes to User--and 
to maintain those against modules which use the user module seems rather 
excessive.

While my first suggestion was:
Move the encryption related portions of the code that are hard coded to the 
authentication backend.  Make a default which follows best practices (I 
would suggest moving to SHA2 in a backwards compatible fashion) that most 
people will use.  However, for those that want to use bcrypt for example, it 
would be easy for them to simply write their own backend.   

However, there are also merits to Paul's approach of having a mapping in the 
settings file.   What I like about the backend approach is that it allows 
for graceful fallbacks as function of python version, but gives the user the 
ability to change the algorithm in a simple way.
Also, it saves one from distinguishing between sha1 and sha1 with 
stretchingPerhaps a compromise would be to have a backend
which looks to the settings file for the location of a method?

But, if I write a patch that works and maintains backwards compatibility 
would it be accepted?  Is it better to email it here, or to submit a ticket, 
claim the ticket, and then add the patch?


Also, would it be better for me to work from the trunk, or from 1.2.5?  This 
is important because of:
http://code.djangoproject.com/ticket/13969


Thanks,
William







-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Pluggable encryption for django auth (design proposal)

[william ratcliff]

Hi!  The scenario that I am considering is when the attacker has your
database--can they compromise the passwords in it?  While I believe that a
salt will protect you against a "Rainbow Table" attack, I'm not convinced
that it will protect you against the brute-force attacks which are now
possible.  I will try to consult some experts today and see if they are
willing to go on record.

William

On Fri, Feb 11, 2011 at 8:10 AM, Eduardo Cereto Carvalho <
eduardocer...@gmail.com> wrote:

> I'm not an expert on the subject.
>
> But I think that the hashes security issues are olved by the use of a
> "salt", salted hashes are known to be a very secure way to store data.
>
>
>
> On Fri, Feb 11, 2011 at 3:59 AM, William Ratcliff <
> william.ratcl...@gmail.com> wrote:
>
>> Hi!  I'm new to the list and have started to look into authentication.   I
>> find that I will need to patch it for my own needs, but would like to ask
>> the opinions of others who are more familiar with the code-base than I am.
>>  I apologize if I make any mistakes in the protocol of the list in matters
>> such as including too much code.
>>
>> SHA1 is not secure.  This is not a nationalism issue.  For example:
>>
>> http://www.darknet.org.uk/2010/11/sha-1-password-hashes-cracked-using-amazon-ec2-gpu-cloud/
>>
>> Or, from NIST:
>> http://csrc.nist.gov/groups/ST/toolkit/secure_hashing.html
>>
>> where the relevant excerpt is:
>> *March 15, 2006*: The SHA-2 family of hash functions (i.e., SHA-224,
>> SHA-256, SHA-384 and SHA-512) may be used by Federal agencies for all
>> applications using secure hash algorithms. Federal agencies should stop
>> using SHA-1 for digital signatures, digital time stamping and other
>> applications that require collision resistance as soon as practical, and
>> must use the SHA-2 family of hash functions for these applications after
>> 2010. After 2010, Federal agencies may use SHA-1 only for the following
>> applications: hash-based message authentication codes (HMACs); key
>> derivation functions (KDFs); and random number generators (RNGs). Regardless
>> of use, NIST encourages application and protocol designers to use the SHA-2
>> family of hash functions for all new applications and protocols.
>>
>> I have also seen discussions in other venues from people who are worried
>> about the security of SHA1 in the event that their system is compromised,
>> can an attacker gain the passwords in the database?  The appearance of
>> modules such as django-bcrypt:
>> https://github.com/dwaiter/django-bcrypt/
>> show that this issue is becoming of more general concern.
>>
>> Current solutions:
>> 1)  Monkey patch:
>> put a top level installed_app that has a listener to the class_prepared
>> signal that performs monkey patching throughout user.
>> This is rather ugly and it feels very fragile to me if the auth module
>> changes internally.
>> 2)  Rewrite the Backend as Tom suggests by subclassing ModelBackend:
>> Again, it's not sufficient.  Why?
>>
>> If we look at the Model Backend, we see that yes, the authenticate method
>> currently authenticates against User--but the problem is NOT the
>> authentication per se, but rather that the User class has several methods
>> such as:
>> check_password, set_password, etc. that have encryption hard coded.
>> There are admin commands associated with the User class which refer to
>> methods with a particular encryption method chosen.
>>
>> 3) For users of Django who cannot (say US government agencies, people who
>> have tight security concerns, etc.) use the current module, ignore the auth
>> module and roll their own:
>> This has been attempted before.  However, the problem is that it is easy
>> to make mistakes doing this and most of the functionality in the auth module
>> is very good and simply copying most of it to make a few changes to
>> User--and to maintain those against modules which use the user module seems
>> rather excessive.
>>
>> While my first suggestion was:
>> Move the encryption related portions of the code that are hard coded to
>> the authentication backend.  Make a default which follows best practices (I
>> would suggest moving to SHA2 in a backwards compatible fashion) that most
>> people will use.  However, for those that want to use bcrypt for example, it
>> would be easy for them to simply write their own backend.
>>
>> However, there are also merits to Paul's approach of having a mapping in
>> the settings file.   What I like about the backend approach is that it
&g

Re: Pluggable encryption for django auth (design proposal)

[william ratcliff]

Wow--you're ahead of me!  Is your custom auth public source?  If so, may I
see the repo?  Also, for increasing the length of the salt, are you
referring to:
http://code.djangoproject.com/attachment/ticket/13969/better_salting.diff

<http://code.djangoproject.com/attachment/ticket/13969/better_salting.diff>I
thought it was marked as accepted.  But I just checked out SVN and you are
correct that it is not using gen_salt.

Does anyone know when it will be included?

Thanks,
William

On Fri, Feb 11, 2011 at 9:50 AM, Clemens-O. Hoppe <
clemens.o.ho...@googlemail.com> wrote:

> That's a subject which comes up every few months, sadly.
>
> In a nutshell, if something requires python >= 2.5 or a lib for older
> versions of Python, forget about adding it.
>
> See f. e. http://code.djangoproject.com/ticket/5600 which was closed as a
> no-fix 3 years ago (full disclosure: I'm coh in that bug report). There was
> also a discussion on this mailing list a few weeks ago about increasing the
> salt length, but afaik it had no code-change as a result.
>
> I apologize if I sound a bit grumpy, but I've spend the last 5 days with
> monkey-patching a local branch of the auth lib up to the latest in security
> (SHA512, 128-bit salt, pre-stretching, pbkdf2, stronger random token
> generation (salt, csrf, default-password)), now it spreads into other areas
> of the django-lib as well (currently SECRET_KEY in the starproject script).
>
> Of course I would very much welcome such a proposal, yet I just believe the
> odds for it to happen are (very) low.
>
> Cheers,
>
> coh
>
>
> On 02/11/2011 06:59 AM, William Ratcliff wrote:
>
>  Hi!  I'm new to the list and have started to look into authentication.   I
>> find that I will need to patch it for my own needs, but would like to ask
>> the opinions of others who are more familiar with the code-base than I am.
>>  I apologize if I make any mistakes in the protocol of the list in matters
>> such as including too much code.
>>
>> SHA1 is not secure.  This is not a nationalism issue.  For example:
>>
>> http://www.darknet.org.uk/2010/11/sha-1-password-hashes-cracked-using-amazon-ec2-gpu-cloud/
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Pluggable encryption for django auth (design proposal)

[william ratcliff]

Excellent summary!  If the core developers agree to this, I'm happy to
contribute.

William

On Mon, Feb 14, 2011 at 1:37 AM, poswald  wrote:

> Here is an overview of issues on this subject opened over the years.
> Some have existing code:
>
> http://code.djangoproject.com/ticket/3316 (Adding `crypt' to list of
> password hashes for legacy apps. - closed: fixed)
> http://code.djangoproject.com/ticket/5600 (Patch to enhance
> cryptography on django.contrib.auth - closed: wontfix)
> http://code.djangoproject.com/ticket/5787 (BCrypt password hashing
> support in Django - closed: duplicate)
> http://code.djangoproject.com/ticket/6028 (add compatibility with
> glibc2 MD5-based crypt passwords - new )
> http://code.djangoproject.com/ticket/9101 (Improved salt generation
> for django.contrib.auth - closed: wontfix)
> http://code.djangoproject.com/ticket/9194 (Allow additional hashing
> algorithms for passwords - closed: duplicate)
> http://code.djangoproject.com/ticket/13969 (auth module should use
> longer salt for hashing - new)
>
> Some of the arguments being made for this feature have been a bit
> misleading and most of them pre-date the NIST requirements changeover.
> I think the summary I made before gives the most clear overview of the
> current situation: as it currently stands, if you get access to the
> contents of a Django user table, you can decrypt the passwords very
> cheaply/rapidly.
>
> Looking at the code of existing solutions to this it seems like the
> following would be a reasonable approach:
>
> * Django ships with SHA2-256, SHA2-512 or PBKDF2 by default. SHA2 is
> python 2.5 compatible (due to hashlib being added in python 2.5) and
> PBKDF2 is short enough that it could be included into the project.
> This satisfies NIST/US Gov requirements.
> * SHA1 is maintained for backwards compatibility
> * More secure hashing algorithms can be specified by defining the
> functions to be used for 'User.set_password' and 'User.check_password'
> as suggested above.
>
> To use SHA2-512 by default requires a larger password db column. That
> might be reasonable and would be a better choice. Additionally we
> could look into using Drepper's key strengthening algorithm of SHA2 by
> default to make django brute-force resilient out of the box. It should
> also be noted that an algorithm like bcrypt stores the salt in with
> the hash and therefore the salt column is not used.
>
> It seems like we have people motivated to do the work. We need a
> design decision made by a core dev that this is an acceptable
> approach.
>
> -Paul
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Pluggable encryption for django auth (design proposal)

[william ratcliff]

Carl and Russ,

Thanks for the response!  Would you prefer that those of us interested in
working on this (pluggable user cryto-system) proceed from the trunk, or
from 1.2?

Thanks,
William


On Tue, Feb 15, 2011 at 2:03 AM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

> On Tue, Feb 15, 2011 at 2:24 PM, Carl Meyer 
> wrote:
> > Hi Paul,
> >
> > On Feb 14, 1:37 am, poswald  wrote:
> >> * Django ships with SHA2-256, SHA2-512 or PBKDF2 by default. SHA2 is
> >> python 2.5 compatible (due to hashlib being added in python 2.5) and
> >> PBKDF2 is short enough that it could be included into the project.
> >> This satisfies NIST/US Gov requirements.
> >> * SHA1 is maintained for backwards compatibility
> >> * More secure hashing algorithms can be specified by defining the
> >> functions to be used for 'User.set_password' and 'User.check_password'
> >> as suggested above.
> >
> > I'm only one core dev, and not a crypto expert, but I've read the
> > linked material and followed previous conversations, and here's my
> > take:
> >
> > I don't think it's OK for Django to continue shipping with a default
> > password hashing scheme which no crypto expert, as far as I've seen,
> > considers adequate. People I trust to know their crypto, e.g. Thomas
> > Ptacek of Matasano, consider PBKDF2 to be significantly better than
> > salted SHA1 for password storage, if not quite as good as bcrypt. [1]
> > PBKDF2 is simple enough (just SHA1 iterated many times) that including
> > an existing pure-Python implementation in Django seems reasonable,
> > removing the concerns about cross-platform and Python version
> > compatibility. (It would still be best if we could get the PBKDF2
> > implementation reviewed by a cryptographer.) So I'm +1 on switching
> > Django's default password hashing to PBKDF2.
>
> As with Carl -- I'm only one core dev, and I'm not a crypto expert,
> here's my take:
>
> I agree that it's less than ideal for us to continue to use SHA1 given
> it's known inadequacies.
>
> My concern with this approach is that it requires us to either
> maintain or adopt an encryption algorithm. We're not just using
> something from the standard library, we're taking responsibility for
> the holes in a specific implementation. Even if we just adopt code
> from an existing implementation, we are accepting responsbility for
> finding and fixing any holes in that implementation. This is a
> responsibility that can't be taken lightly, and I'm not completely
> convinced that we should pick up that particular gauntlet.
>
> For this reason alone, I could be convinced that a configuration item
> may be called for here -- e.g., registering a user-crypto library that
> the default User object will use, in the same way that you can
> currently register serialization libraries.
>
> However, that said:
>
> > As for the broader configurability question, I'm just fine with
> > requiring a custom auth backend, which really isn't that hard, as a
> > condition for customizing password hashing. So I'm not particularly
> > tempted by proposals to add a new setting for this. The hardcoded
> > stuff in the User model does bug me, though; I'm interested in the
> > proposal to make the User model delegate that to new methods on an
> > authentication backend (with backwards-compatibility fallback for old
> > auth backends that don't have the new methods).
>
> One of the things that I want to tackle in the 1.4 timeframe is the
> general problem of a 'pluggable' User model. Allowing for customizable
> authentication schemes is one (of many) parts of this problem. Right
> now, my focus is on getting the 1.3 release out the door; once the 1.4
> feature phase starts, I'll have a lot more time to discuss this sort
> of thing.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Google Patch Rewards program

[William Vincent]

This is great! To the extent we have this teed up internally, makes it much 
easier for DSF to go to Google and coordinate. And even if Google doesn't 
pan out, having structure makes it easier for us to, for example, find 
funding if some fundraising projects in the works pan out.

On Tuesday, December 31, 2019 at 11:40:15 AM UTC-5, Dan Davis wrote:
>
> Another good security improvement would be to allow the database password 
> and other database information to support AWS Secrets Manager, Goolge 
> Secrets Management, and HashiCorp vault (+ others).
> I have done this in a private package used at the National Library of 
> Medicine, but my package is both private and limited to RDS PostgreSQL and 
> AWS Secrets Manager.
>
> On Tue, Dec 31, 2019 at 11:25 AM Dan Davis  > wrote:
>
>> Taymon Beal writes:
>> > First-class integration with one or more secrets management systems, 
>> both to generally contain secrets better and more specifically 
>> > so people aren't so tempted to check SECRET_KEYs and database passwords 
>> into source control. (I think this was mentioned in the list of GSoC 
>> project ideas.)
>>
>> What secrets management systems are you thinking about?   I wrote 
>> confsecrets, whose Django integration never worked right, assuming that I'd 
>> port it to support the following:
>> * HashiCorp vault
>> * AWS Secrets Manager
>> * Google Secrets Management
>>
>> I also need to switch it from cryptdome to cryptography or make it 
>> agnostic on these.
>>
>> P.S. - I have some very limited applied cryptography knowledge - couple 
>> chapters of Bruce Schneier and some hands-on learning, such as securing 
>> single license features with signatures based on my employer's license 
>> signing private key, and discovering that a customer literally pulled one 
>> set of feature.xml/feature.sig files from one tarball, and another from 
>> another tarball to game my home-grown licensing system.  I helped that 
>> company eventually move to a Secure USB dongle, and that was my last moment 
>> of systems software development :)
>>
>> On Sat, Dec 28, 2019 at 11:23 PM Taymon A. Beal > > wrote:
>>
>>> (Disclosure: I'm on Google's security team, and my views on this topic 
>>> are informed by what kinds of things we tend to look for in Web frameworks, 
>>> but here I don't speak for them, only for myself.)
>>>
>>> Beyond those already mentioned, here are some potential security 
>>> improvements I'd like to see in Django:
>>>
>>>- Support for contextual autoescaping in the template system. The 
>>>current autoescaping behavior is better than nothing, but it still makes 
>>>XSS far too easy, since different kinds of strings are XSS sinks in 
>>>different contexts. 
>>>
>>> https://github.com/google/closure-templates/blob/master/documentation/concepts/auto-escaping.md
>>>  shows 
>>>an example of how to do this sort of thing more securely.
>>>- First-class integration with one or more secrets management 
>>>systems, both to generally contain secrets better and more specifically 
>>> so 
>>>people aren't so tempted to check SECRET_KEYs and database passwords 
>>> into 
>>>source control. (I think this was mentioned in the list of GSoC project 
>>>ideas.)
>>>- Capability-based authorization. Right now, you have to explicitly 
>>>check for all relevant permissions everywhere, and if you forget, or 
>>> even 
>>>if you accidentally include the wrong variable in a template, you can 
>>> leak 
>>>data or worse. It'd be much safer if the allowed permissions could be 
>>>defined at a single choke point, and from there, all model access within 
>>> a 
>>>request could be mediated by the specified authorization rules. Ideally 
>>>data that the current user isn't supposed to see would not even be 
>>> fetched 
>>>from the database.
>>>
>>> Taymon
>>>
>>> On Sat, Dec 21, 2019 at 11:29 PM Asif Saif Uddin >> > wrote:
>>>
 Really good plans Adam!

 On Saturday, December 21, 2019 at 11:51:11 PM UTC+6, Adam Johnson wrote:
>
> I just saw Google is expanding their Patch Rewards program for open 
> source security improvements: 
> https://security.googleblog.com/2019/12/announcing-updates-to-our-patch-rewards.html
>
> They are offering two tiers of rewards - $5,000 or $30,000 - for  open 
> source projects making security improvements. I think Django would find 
> it 
> hard to fit in the "small" tier - we generally fix known vulnerabilities 
> quickly - but we could use the "large" tier to fund a bigger GSoC style 
> project. I suspect it would need active involvement from a DSF member to 
> push it through. Not sure how the funding would work in terms of DSF and 
> paying for development time on the project.
>
> Some projects that could fit:
>
>- 2FA built-in to django.contrib.auth (as suggested for GSoC as 
>well in this thread: 
>
> https://groups.

Proposal: Permit __file__-less migrations packages

[William Schwartz]

Django developers,

The migration loader currently loads migrations only from apps whose 
migrations package has a __file__ 
<https://docs.python.org/3/reference/import.html#__file__> attribute. The 
reason for this check, given both in the source code comments 
<https://github.com/django/django/blob/2a76f4313423a3b91caade4fce71790630ef9152/django/db/migrations/loader.py#L91-L95>
 
and in discussions among maintainers 
<https://groups.google.com/g/django-developers/c/GVHMH2ciAnk/m/vbIPbZuSBQAJ>, 
is to discourage the use of PEP-420 
<http://www.python.org/dev/peps/pep-0420/> namespace packages.

Without affecting that behavior, I propose to permit migrations run from 
Python environments that do not provide even *regular* packages with a 
__file__ <https://docs.python.org/3/reference/import.html#__file__> 
attribute. (This falls under the general umbrella of #30950 
<https://code.djangoproject.com/ticket/30950>, which describes these 
environments a little.) The required change is just a few lines.

If this proposal is something you would consider, I will submit a Trac 
ticket laying out further details, including citations to Python 
documentation that justifies my approach. I will simultaneously submit a PR 
with the change and a test. Documentation changes are not likely necessary, 
but I can add some if desired.

I would love for this feature to land in 3.2 before the feature-freeze 
date, and can do all the work for it if a Django committer can review and 
merge the PR.

Best,
William Schwartz

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1c2cb975-0f78-4ef9-a025-303dcb17a5aen%40googlegroups.com.

Re: Proposal: Permit __file__-less migrations packages

[William Schwartz]

As I said in my original post, I *do not propose to change the behavior of 
Django with respect to namespace packages*. My proposal only makes the 
check for them more specific so that we permit *regular* packages run from 
Python environments that do not set __file__.

On Tuesday, December 22, 2020 at 10:55:01 AM UTC-6 Adam Johnson wrote:

> I've hit several problems with PEP-420 namespace packages in the last few 
> months, which inspired me to create flake8-no-pep420: 
> https://pypi.org/project/flake8-no-pep420/ . In summary, I found pep-420 
> namespace packages are not checked by Django's test runner, coverage.py, or 
> mypy (with its bonus flag --namespace-packages). I imagine other code tools 
> fail to import them too.
>
> I would be inclined to stick with the previous decision unless you can 
> prove there's a great reason to support migrations in namespace packages -- 
> other than a distaste for blank __init__.py files. I can't imagine you 
> actually want to split one app's migrations into several separate file 
> system directories, do you?
>
> On Tue, 22 Dec 2020 at 15:58, William Schwartz  wrote:
>
>> Django developers,
>>
>> The migration loader currently loads migrations only from apps whose 
>> migrations package has a __file__ 
>> <https://docs.python.org/3/reference/import.html#__file__> attribute. 
>> The reason for this check, given both in the source code comments 
>> <https://github.com/django/django/blob/2a76f4313423a3b91caade4fce71790630ef9152/django/db/migrations/loader.py#L91-L95>
>>  
>> and in discussions among maintainers 
>> <https://groups.google.com/g/django-developers/c/GVHMH2ciAnk/m/vbIPbZuSBQAJ>,
>>  
>> is to discourage the use of PEP-420 
>> <http://www.python.org/dev/peps/pep-0420/> namespace packages.
>>
>> Without affecting that behavior, I propose to permit migrations run from 
>> Python environments that do not provide even *regular* packages with a 
>> __file__ <https://docs.python.org/3/reference/import.html#__file__> 
>> attribute. (This falls under the general umbrella of #30950 
>> <https://code.djangoproject.com/ticket/30950>, which describes these 
>> environments a little.) The required change is just a few lines.
>>
>> If this proposal is something you would consider, I will submit a Trac 
>> ticket laying out further details, including citations to Python 
>> documentation that justifies my approach. I will simultaneously submit a PR 
>> with the change and a test. Documentation changes are not likely necessary, 
>> but I can add some if desired.
>>
>> I would love for this feature to land in 3.2 before the feature-freeze 
>> date, and can do all the work for it if a Django committer can review and 
>> merge the PR.
>>
>> Best,
>> William Schwartz
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/1c2cb975-0f78-4ef9-a025-303dcb17a5aen%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/1c2cb975-0f78-4ef9-a025-303dcb17a5aen%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1e41bb54-3502-4508-9f67-262db28fc5f4n%40googlegroups.com.

Re: Proposal: Permit __file__-less migrations packages

[William Schwartz]

I now see that my original post was too vague. I will more thoroughly 
explain myself below, including the actual patch (because it's that 
short!). *My proposal will not change Django's behavior at all for anyone 
not using "freezers" *such as PyOxidizer, cx_Freeze, Nuitka, PyInstaller, 
and the like. No changes to namespace package support. No changes to 
dependencies. In particular I mentioned #30950 
 only in its capacity as a 
tracking ticket, but my proposal supports a *very narrow* subset of that 
ticket's goals.

To filter out namespace packages, 
django.db.migrations.loader.MigrationLoader.load_disk 

 
currently rejects any module missing a __file__ attribute (including __file__ 
is None). Because __file__ is an *optional* 
 attribute of 
modules not provided by some freezers 
,
 
*just testing for __file__ casts too wide a net*. Fortunately, namespace 
packages have other differences with regular packages that can be detected 
at runtime. One of those differences is that their __loader__ is 
importlib._bootstrap_external._NamespaceLoader, but that is undocumented 
and specific to CPython. However, a difference that *is* documented in the 
Python language specification 
 is the 
type of __path__:

> Namespace packages do not use an ordinary list for their __path__ 
attribute. They instead use a custom iterable type

So here is the patch that I propose. For brevity, I am excluding the test, 
but it's only another 17 lines.





*diff --git a/django/db/migrations/loader.py 
b/django/db/migrations/loader.pyindex 95a5062ec9..fd3a92f779 100644--- 
a/django/db/migrations/loader.py+++ b/django/db/migrations/loader.py*@@ 
-88,15 +88,19 @@ *class* MigrationLoader:
 *continue*
 *raise*
 *else*:
-# Empty directories are namespaces.
-# getattr() needed on PY36 and older (replace w/attribute 
access).
-*if* getattr(module, '__file__', None) *is* None:
-self.unmigrated_apps.add(app_config.label)
-*continue*
 # Module is not a package (e.g. migrations.py).
 *if* *not* hasattr(module, '__path__'):
 self.unmigrated_apps.add(app_config.label)
 *continue*
+# Empty directories are namespaces. Namespace packages 
have no
+# __file__ and don't use a list for __path__. See
+# 
https://docs.python.org/3/reference/import.html#namespace-packages
+*if* (
+getattr(module, '__file__', None) *is* None *and*
+*not* isinstance(module.__path__, list)
+):
+self.unmigrated_apps.add(app_config.label)
+*continue*
 # Force a reload if it's already loaded (tests need this)
 *if* was_loaded:
 reload(module) 

I hope the narrow nature of this patch is more acceptable than what you 
initially thought I was proposing. I am happy to answer further questions.

Thanks for taking the time to engage with me about this!

On Tuesday, December 22, 2020 at 11:23:19 AM UTC-6 Mariusz Felisiak wrote:

> Hi,
>
> I can only repeat my arguments 
>  from the ticket. 
> I don't see a strong reason to add an extra dependency for this. Django 4.0 
> will drop support for Python 3.6 and Python 3.7, so we can reconsider this 
> ticket when developing Django 4.0+.
>
> Best,
> Mariusz
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1cc41b79-3bd2-4bbf-9882-0c8d9075b78bn%40googlegroups.com.

Re: Proposal: Permit __file__-less migrations packages

[William Schwartz]

Mariusz,

Thanks for your response!

*> I have the impression that more changes are needed to support 
"freezers" Are we sure this is the only change that is needed?*

Django will indeed require changes beyond today's proposal for full freezer 
support. I am pursuing an incremental approach here—apologies for being 
unclear on this point. My intention for this proposal, like the one in 
#32177  whose PR you merged, 
is to remove one failure mode for frozen Django apps without adversely 
affecting any other users.

I led with this proposal rather than others that would make progress on 
freezer support because it makes Django more correct and compliant with 
Python's import API without affecting any existing users. That is, I 
believe the patch strictly improves Django regardless of PyOxidizer.

(If there's interest, I would be happy to start a new thread to explain 
what I patched/overrode in Django to get my Django app working in 
PyOxidizer.)

*> how we can avoid regressions and don't break this in the future?*

My PR will add the following test case to 
tests/migrations/test_loader.py:LoaderTests.

*def* test_loading_package_without__file__(self):
"""Migrations can be found even if the package's __file__ is 
undefined."""
*from* migrations *import* test_migrations
# __file__ == __spec__.origin or the latter is None and former 
undefined
file = test_migrations.__file__
origin = test_migrations.__spec__.origin
has_location = test_migrations.__spec__.has_location
*try*:
*del* test_migrations.__file__
test_migrations.__spec__.origin = None
test_migrations.__spec__.has_location = False
self.test_load()
*finally*:
test_migrations.__file__ = file
test_migrations.__spec__.origin = origin
test_migrations.__spec__.has_location = has_location

No changes to the existing LoaderTests.test_loading_namespace_package are 
required.
On Tuesday, December 22, 2020 at 1:28:41 PM UTC-6 Mariusz Felisiak wrote:

> Hi,
>
> Thanks for extra details. It's not about the number of LOC. I have the 
> impression that more changes are needed to support "freezers" , see the 
> previous discussion about PyOxidizer 
> . 
> Are we sure this is the only change that is needed? and how we can avoid 
> regressions and don't break this in the future?
>
> Best,
> Mariusz
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/62cf1683-3543-48a0-9363-dc3260b34ac0n%40googlegroups.com.

Re: Proposal: Permit __file__-less migrations packages

[William Schwartz]

In the hope that I have allayed this group's initial concerns, I have 
created ticket #32302 <https://code.djangoproject.com/ticket/32302> and 
pull request GH-13820 <https://github.com/django/django/pull/13820>. If you 
still have reservations, I am happy to discuss this matter further. Getting 
this merged in time for Django 3.2 would be a huge relief!

On Tuesday, December 22, 2020 at 4:27:53 PM UTC-6 William Schwartz wrote:

> Mariusz,
>
> Thanks for your response!
>
> *> I have the impression that more changes are needed to support 
> "freezers" Are we sure this is the only change that is needed?*
>
> Django will indeed require changes beyond today's proposal for full 
> freezer support. I am pursuing an incremental approach here—apologies for 
> being unclear on this point. My intention for this proposal, like the one 
> in #32177 <https://code.djangoproject.com/ticket/32177> whose PR you 
> merged, is to remove one failure mode for frozen Django apps without 
> adversely affecting any other users.
>
> I led with this proposal rather than others that would make progress on 
> freezer support because it makes Django more correct and compliant with 
> Python's import API without affecting any existing users. That is, I 
> believe the patch strictly improves Django regardless of PyOxidizer.
>
> (If there's interest, I would be happy to start a new thread to explain 
> what I patched/overrode in Django to get my Django app working in 
> PyOxidizer.)
>
> *> how we can avoid regressions and don't break this in the future?*
>
> My PR will add the following test case to 
> tests/migrations/test_loader.py:LoaderTests.
>
> *def* test_loading_package_without__file__(self):
> """Migrations can be found even if the package's __file__ is 
> undefined."""
> *from* migrations *import* test_migrations
> # __file__ == __spec__.origin or the latter is None and former 
> undefined
> file = test_migrations.__file__
> origin = test_migrations.__spec__.origin
> has_location = test_migrations.__spec__.has_location
> *try*:
> *del* test_migrations.__file__
> test_migrations.__spec__.origin = None
> test_migrations.__spec__.has_location = False
> self.test_load()
> *finally*:
> test_migrations.__file__ = file
> test_migrations.__spec__.origin = origin
> test_migrations.__spec__.has_location = has_location
>
> No changes to the existing LoaderTests.test_loading_namespace_package are 
> required.
> On Tuesday, December 22, 2020 at 1:28:41 PM UTC-6 Mariusz Felisiak wrote:
>
>> Hi,
>>
>> Thanks for extra details. It's not about the number of LOC. I have 
>> the impression that more changes are needed to support "freezers" , see the 
>> previous discussion about PyOxidizer 
>> <https://groups.google.com/g/django-developers/c/V2Kyah9MoXY/m/-7_kwGcgCAAJ>.
>>  
>> Are we sure this is the only change that is needed? and how we can avoid 
>> regressions and don't break this in the future?
>>
>> Best,
>> Mariusz
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cb2d9b0d-99d8-4833-8bcc-015d5e2fe616n%40googlegroups.com.

Re: Proposal: Drop dependency on pytz in favor of zoneinfo

[William Schwartz]

Just wanted to chime in with a +1 from a user in favor of moving away from 
pytz. Doing so will be very helpful for frozen Python environments: 
https://bugs.launchpad.net/pytz/+bug/1834363

On Sunday, January 3, 2021 at 2:43:01 AM UTC-6 Aymeric Augustin wrote:

> Hi Carlton,
>
> IIUC-at-first-pass: your suggestion differs in jumping straight to the 
> end-point with a fallback for those who need it. (In contrast to getting 
> folks to opt-in early if they want it.)
>
>
> Indeed. The majority will switch when the default changes; better earlier 
> than later.
>
> what did you think about allowing an early opt-in for Django v3.2?
>
>
> The more options we create, the more confusion we may generate e.g. when 
> users try to understand release notes or to ask for support. This is why I 
> tried to keep options to a minimum (1. initial situation, 2. target 
> situation, 3. with shim) and to make as few users as possible go through 
> option 3.
>
> Implementing USE_PYTZ_DEPRECATION_SHIM is Django 3.2 makes the situation a 
> bit more complex, as the setting will mean "use pds instead of pytz" in 3.2 
> and "use pds instead of zoneinfo" in 4.0. We'd have four options instead of 
> three.
>
> I'm pretty confident that the transition will go smoothly and I don't 
> think the flaws in pytz are bad enough to warrant urgent action. As a 
> consequence, offering opt-in in Django 3.2 doesn't seem all that valuable 
> to me.
>
> I'm OK with starting the migration in one year in Django 4.0. I'm also OK 
> with offering the opt-in in Django 3.2 if others have a different value 
> assessment!
>
> Cheers,
>
> -- 
> Aymeric.
>
>
>
> On 3 Jan 2021, at 09:12, Carlton Gibson  wrote:
>
> Hi Aymeric. 
>
> Thanks for inputting! 
>
> I need to read in-depth what you’ve said here, but IIUC-at-first-pass: 
> your suggestion differs in jumping straight to the end-point with a 
> fallback for those who need it. (In contrast to getting folks to opt-in 
> early if they want it.) This sounds better if we can. 
>
> The Autumn hasn’t allowed time to make progress here but, what did you 
> think about allowing an early opt-in for Django v3.2? (I’m not sure we have 
> capacity to get the in in-time now anyway, so it may be moot.)
>
> Kind regards, 
> Carlton.
>
>
>
> On Sat, 2 Jan 2021 at 10:29, Aymeric Augustin <
> aymeric@polytechnique.org> wrote:
>
>> Hello,
>>
>> As the original author of support for timezone aware datetimes in Django, 
>> I've been meaning to review this... for six months... Better late than 
>> never I guess?
>>
>> In this discussion, we're assuming settings.USE_TZ = True.
>>
>>
>> The original design 
>> , which 
>> still matches 80% of the current implementation, was very much based on 
>> advice found in the documentation of pytz. This has three consequences:
>>
>> 1. By default, users manipulate aware datetimes in UTC, which limits the 
>> breakage to cases where they explicitly switch to another timezone. This is 
>> auditable e.g. "look for timezone.activate or timezone.override in your 
>> code".
>>
>> 2. Django provides custom wrappers in order to take care of the pitfalls 
>> of pytz automatically e.g. timezone.make_aware. Backwards-compatibility can 
>> be implemented transparently there.
>>
>> 3. A strategy designed for migrating from pytz to zoneinfo should be 
>> applicable to Django.
>>
>>
>> I'm seeing three areas that need care:
>>
>> A. APIs returning a tzinfo object, currently a pytz timezone (other than 
>> UTC — we switched from Django's custom definition of the UTC tzinfo to 
>> pytz' definition in the past without trouble).
>>
>> B. APIs returning aware datetimes, currently in a pytz timezone (other 
>> than UTC).
>>
>> C. APIs performing datetime conversions in the database, which is 
>> typically used for aggregating by day in a given timezone. This depends on 
>> the timezone name. I think we're fine on this front since we're keeping the 
>> same timezone names.
>>
>> So the primary concern is leaking pytz tzinfo objects (either directly, 
>> or via aware datetime objects), to user code that requested it explicitly. 
>> I may sound like I'm belaboring the point. However, I think we can make a 
>> better backwards-compatibility decision with an accurate estimate of the 
>> extent of the breakage.
>>
>>
>> Django currently references pytz in the following places:
>>
>> - timezone.get_default/current_timezone => see case A above.
>> - timezone.get_default/current_timezone_name => shouldn't be an issue 
>> since time zone names don't change.
>> - BaseDatabaseWrapper.timezone => for timezone conversions in Python code 
>> via make_aware / make_naive; see case B above.
>> - BaseDatabaseWrapper.timezone_name => for timezone conversions in the 
>> database via database-specific SQL functions; see case C above.
>>
>> With SQLite, the SQL conversion functions are implemented in Python and 
>> use pytz, but the end result is the same.
>>
>>
>

PR review request

[William Schwartz]

Django committers,

The docs recommend 
<https://docs.djangoproject.com/en/3.1/internals/contributing/new-contributors/#faq>
 
posting here if a pull request hasn't seen any action in awhile—I'm sorry 
if this nudge is premature! There are a couple of PRs I'd love to get 
merged into 3.2 before the upcoming beta release. Both PRs' tickets were 
approved as cleanups/optimizations before the 3.2 feature freeze.

   - More important: #32316 <https://code.djangoproject.com/ticket/32316>'s 
   PR-13841 <https://github.com/django/django/pull/13841>, which reduces 
   use of __file__ at the module level. The PR has already seen one round 
   of review and fixes, so hopefully it's (close to) done.
   - Less important: #32317 <https://code.djangoproject.com/ticket/32317>'s 
   PR-13842 <https://github.com/django/django/pull/13842>, which refactors 
   the loaddata command. The PR has not yet been reviewed.

Thanks for working with me on these (and several other recent) PRs!

Best,
William

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d6fe9a1d-9b1c-4c2a-b706-540d7e35b329n%40googlegroups.com.

Re: CTRL-BREAK still make sense on Windows for `runserver`?

[William Vincent]

Ok, thanks for the link Matthew. Just thought I'd bring it up in case 
things had changed. Seems status quo is fine for now but if Windows 
keyboards continue to remove the Pause/Break key might make sense to add 
`Control-C` as Adam notes or eventually switch over.

On Tuesday, June 22, 2021 at 6:02:39 AM UTC-4 f.apo...@gmail.com wrote:

> As long as runserver (usually) properly shuts down when CTRL-C is hit, 
> then it imo makes sense to change the text -- especially if that is the 
> only thing accessible on some keyboards.
>
> The usually above is because someone might have a bare except that is hit 
> that moment and as such the process might not shut down.
>
> On Tuesday, June 22, 2021 at 7:58:47 AM UTC+2 carlton...@gmail.com wrote:
>
>> I too realise I don't have a(n obvious) break key but have been happily 
>> hitting CTRL-C. 
>>
>> On Monday, 21 June 2021 at 23:53:15 UTC+2 Adam Johnson wrote:
>>
>>> Would it be bad to have Django respond to both shortcuts?
>>>
>>
>> I think as a matter of fact it does. At least using PowerShell/Win10/Etc 
>> — Looking at Matthew's link, we don't do either of the things that would 
>> inhibit this. 🤔
>>
>> Looking at it from Will's POV (writing introductory guides) the output 
>> text of runserver might be a bit problematic:
>>
>> "Quit the sever with CTRL-BREAK" — Errrmmm... Where's that then? 
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/751d9d0f-a387-498f-89be-8a9c7484eea4n%40googlegroups.com.

Re: Google "Season of Docs"

[William Hakizimana]

Just out of curiosity, I was wondering if we got any traction on this.

On Monday, March 18, 2019 at 5:27:47 AM UTC-5, Carlton Gibson wrote:
>
> Hi all, 
>
> Parallel to GSoC, Google now have this "Season of Docs" programme: 
>
> https://developers.google.com/season-of-docs/
>
> The idea is experienced technical writers can get funding to work with 
> open source projects on their docs. 
>
> There are a lot of open Documentation issues 
> 
> . 
>
> As such if you have, or if you know someone with, strong writing skills 
> and you 
> (they) might be interested here let me know and we can look into this. 
>
> I can't quite work out whether if we just apply we might attract a writer, 
> but it would be 
> awesome if already someone in the community was keen for this.
>
> Thanks. 
>
> Kind Regards,
>
> Carlton
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4fc398b3-151f-49fc-8279-9b2f300aa348%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Make Development More Accessible

[William Vincent]

I think the way Rails does it, aka with well-done newcomers guide 
(https://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html) is 
worth looking at, as Carlton notes. A bit more streamlined than the current 
Django How To Contribute Guides.

Incidentally, Carlton and I will be having a chat with DHH of Ruby on Rails 
fame in September for the DjangoChat podcast so if there are any questions 
we should pose to him on managing the Rails community, do pass along to us. 
Certainly I'll ask him about issue tracking!

On Thursday, August 8, 2019 at 3:08:39 PM UTC-4, Tim Graham wrote:
>
> Although I'm not engaged too much with Django development now, a big 
> drawback of moving to GitHub issues for me would be that I could no longer 
> do a Google search like "something something site:code.djangoproject.com". 
> I could pretty much always find the ticket I was looking for that way. 
> Maybe GitHub issue search would be just as good but I find Google results 
> (with a snippet from the page) more useful.
>
> Another migration consideration is that Trac tickets are formatted using a 
> different syntax than GitHub issues.
>
> A migration feels like weeks of work and it doesn't strike me as a "heck 
> yea!" improvement. There would probably be benefits and drawbacks. As 
> Carlton said, I think it's time that could be more usefully allocated.
>
> I wonder how putting issues on GitHub would increase engagement? Is it 
> that Trac isn't discoverable from GitHub? What if GitHub let us redirect 
> https://github.com/django/django/issues (with the "Issues" tab) to 
> code.djangoproject.com?
>
> On Thursday, August 8, 2019 at 4:48:14 AM UTC-4, Carlton Gibson wrote:
>>
>> Just on this point: 
>>
>> > I agree with Andrew Godwins statement on Django loosing many 
>> contributors over the years and being in largely maintenance mode. 
>>
>> First, I'm not sure Andrew actually said this. Rather I think he reported 
>> is a point raised. However...
>>
>> I hear this kind of thing said. It ties-in with the "Django is boring" 
>> trope, when that's not meant as a compliment. 
>>
>> I think it couldn't be further from the truth. 
>>
>> Yes, these ain't the wild west days of yore. (Granted)
>>
>> But Django has successfully transitioned (with large thanks to the effort 
>> Tim Graham put in as Fellow over that time) from young to mature and it 
>> still growing. 
>>
>> Even if you take out your favourite headline feature — don't say it :) — 
>> v3.0 is going to be an awesome release. 2.2 was too. And 2.1. And it's 
>> continuing forward. 
>>
>> I wish we'd drop negative self-talk. It lowers morale and isn't even 
>> correct. Django is in great shape, and only getting stronger. 
>>
>> Could we do with more contributors? Yes. So let's do what we can to make 
>> that better. 
>>
>> I think of the effort to move from Trac. Maybe it's worth it... But I 
>> think of the same effort spent improving djangoproject.com, and the How 
>> to contribute guides (I think Tobias said the Contributing docs are 
>> X-thousands of words—all great but not necessarily that approachable), and 
>> so on, and I wonder if it's the best ROI? (That's a genuinely open 
>> question: DRF uses GitHub issues I don't see it leading to any more 
>> engagement there...) 
>>
>> Go Django Go! 🙂
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ace34012-e5ac-4bbc-ad5b-d3fbac1a560a%40googlegroups.com.

Re: Process DEP for "official" non-core projects

[William Hakizimana]

Just wanted to thank you so much for your hard work. The documentation is 
really well written!

On Wednesday, May 11, 2016, at 1:29:34 PM UTC-5, Andrew Godwin wrote:
>
>
>
> On Wed, May 11, 2016 at 11:00 AM, Jacob Kaplan-Moss  
> wrote:
>
>> I like this, and +1 on your rough outline.
>>
>> There is one missing thing here though: I think we need to consider the 
>> process/policy for removing things if they're no longer maintained. Without 
>> clear maintainership forks happen, which is bad for pretty much everyone. 
>> So I think we should have a plan in place for what happens if the shepherd 
>> can no longer tend to their flock, and nobody else steps into the role.
>>
>> I'd suggest something like this: if a project goes stale, core calls for 
>> new maintainers. If none arrive within a reasonable period of time (3 
>> months?) we deprecate and remove the package from our org.
>>
>>
> That seems sensible to me; if the project is important enough to Django, 
> I'm sure people will step up to take over, and if not, dropping it is 
> probably the better option rather than letting it linger.
>
> I would be inclined to merely mark it as deprecated and not drop it from 
> e.g. the GitHub org, though, as where would we move it *to*? 
>
> Andrew 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/885bd485-341a-47ad-8766-7ef17df77ada%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANNOUNCE] Django security advisory: Vulnerability in password reset (master branch only)

[William Hakizimana]

While we are at it, could we implement these NIST new password guidelines 

 
into django? Just

On Monday, November 21, 2016 at 3:13:21 PM UTC-6, Tim Graham wrote:
>
> We don't normally give security advisories for issues that affect only
> the master branch, but in this case we've made an exception as the issue
> could be high impact.
>
> Please see the blog post for details:
>
> https://www.djangoproject.com/weblog/2016/nov/21/passwordresetconfirmview-security-advisory/
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/71b646e9-9a39-4e47-9aa9-c6cfacd82b6e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Solicitation of feedback: MetaOptions

[William Waites]

I have released a package called django_options on the Cheeseshop at

http://cheeseshop.python.org/pypi/django_options/r6

It is a class called MetaOptions that enables decoration of Django models 
with meta classes in similar style to Admin and Meta. Options placed
there show up in Model._meta.classname.attribute and overriding the
contribute_to_class method of MetaOptions is a useful place to populate
other attributes on models in the case that straightforward fields are
not appropriate or possible.

Included is an example usage to enable CSV export of any set of models
with an example urls.py entry that will add such functionality to the
Admin. It should be noted that this is a specific example, and perhaps
not the best example, of the usage of a much more general interface.

The package installs into django.contrib.options

Any feedback or suggestions are welcome.

Cheers,
-w

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Lets talk about a dynamic limit_choices_to again :)

[William Waites]

On Fri, Nov 23, 2007 at 02:05:28PM -0800, Dmitri Fedortchenko wrote:
> 
> I found this ticket:
> http://code.djangoproject.com/ticket/2445
> 
> It's not seen much activity for a while, but in september jkocherhans
> posted some news which I found interesting. So since I needed this
> functionality, I wrote a patch for it.
> 
> Basically the idea is that you can define:
> def choices_for__FIELD(self):
>  return ModelClass.objects.filter(relation_name=self.relation)

This looks like quite a clean way to accomplish dynamic choices. I've
needed things like this as well and often have resorted to rather less
elegant hacks.

Any chance of getting this or something similar pulled into the trunk?

Cheers,
-w

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

OSG - Django support for Directed Graphs and OpenSocial

[William Waites]

Greetings folks,

I've released a first cut at some Django models that are designed for
modeling directed graphs, and node metadata as well as some initial
work towards support for OpenSocial data APIs. The hope is that this
will be a useful building block for people wanting to implement social
networking web sites using the Django framework.

It also contains an implementation of the generative algorithm by
Farmer et al. for producing test graphs that can be used to test
traversal algorithms and such.

The snapshot can be downloaded from the Python Cheeseshop at:
http://pypi.python.org/pypi/osg

The mercurial repository for development is at: 
http://www.irl.styx.org/hgweb.py/osg/

Caveat: this software is intended for use with the GIS branch of
Django though with minor changes should work with trunk as well.

Cheers,
-w
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django auth magic

[Brian William Kaplan]

I also suppose you could do something like, in the base class:

_name = None
_password = None

And then in the __init__ function the end user could set _name and
_password, but the get_* functions just seems simpler and more
Django-styled.

Django auth magic

[Brian William Kaplan]

I was looking at the Removing The Magic wiki page, and it seems you
have no plan to remove the magic from the auth module. This came as
quite a disappointment to me. Many things in the auth module are
magical, and the auth model makes many assumptions.

The auth model can still be easy and fun to use, and most importantly
provide a simple means of a login system.

Currently the auth model does the following:
   - Assume that we want to use columns such as "First Name" and "Last
Name".
   - Assume that we want to even have a username field.

That is just some examples of the assumptions.

I propose a dynamic Auth model class, with function helpers that return
objects that the rest of the auth module needs to work. For instance,
consider this example:


class User(models.Model):
"Base class for User model, just a barebones example to show that no
columns are in this class."
objects = UserManager()
class Meta:
verbose_name = _('User')
verbose_name_plural = _('Users')
ordering = ('username',)
class Admin:
fields = (
(None, {'fields': ('username', 'password')}),
(_('Personal info'), {'fields': ('first_name', 'last_name',
'email')}),
(_('Permissions'), {'fields': ('is_staff', 'is_active',
'is_superuser', 'user_permissions')}),
(_('Important dates'), {'fields': ('last_login',
'date_joined')}),
(_('Groups'), {'fields': ('groups',)}),
)
list_display = ('username', 'email', 'first_name', 'last_name',
'is_staff')
list_filter = ('is_staff', 'is_superuser')
search_fields = ('username', 'first_name', 'last_name',
'email')



class MyUser(User):
"Our extended User model"
email = models.CharField(_('e-mail'), maxlength=50, unique=True) #
In this example, we want to login with a e-mail address.
password = models.CharField(_('password'), maxlength=128,
help_text=_("Use '[algo]$[salt]$[hexdigest]'"))

def get_name_object():
  return self.email
 def get_password_object():
return self.password

The login code within the module would then call the function to grab
the object based on "get_name_object" and "get_password_object". This
can probably be done better, with some sort of caching. For example, in
the base User class:

def check_password(self, raw_password):
"""
Returns a boolean of whether the raw_password was correct.
Handles
encryption formats behind the scenes.
"""
got_name = self.get_name()
got_password = self.get_password()
# Backwards-compatibility check. Older passwords won't include
the
# algorithm or salt.
if '$' not in got_password:
import md5
is_correct = (got_password ==
md5.new(raw_password).hexdigest())
if is_correct:
# Convert the password to the new, more secure format.
self.set_password(raw_password)
self.save()
return is_correct
algo, salt, hsh = got_password.split('$')
if algo == 'md5':
import md5
return hsh == md5.new(salt+raw_password).hexdigest()
elif algo == 'sha1':
import sha
return hsh == sha.new(salt+raw_password).hexdigest()
raise ValueError, "Got unknown password algorithm type in
password."






As you can see, this allows for more flexibility. I've not included
groups, but that should probably be in the base class. This is just a
thrown together example, and is infact poorly thought out and written.

Re: Django auth magic

[Brian William Kaplan]

Also I haven't given any thought to the inner Admin class. I'll leave
that as an exercise to the reader, simply because I don't even really
agree with how the inner Admin class is setup.

Re: Django auth magic

[Brian William Kaplan]

Okay Adrian, I will write a patch. Should I upload it to my site and
send it through here or the bug tracker?

Re: Django auth magic

[Brian William Kaplan]

I'm not able to write the patch because Django is placing too many
restrictions on what I can do as far as this.

Re: Django auth magic

[Brian William Kaplan]

Writing a patch just requires too much effort. I give up and I'm
reluctantly trying to hack this together now. And it's not even fun
like the website says it is. :-(

Re: Django auth magic

[Brian William Kaplan]

If you are interested the problem happened when I looked at the
formfields file. I don't know how I would make that custom auth class
friendly...

Re: Django auth magic

[Brian William Kaplan]

And also the Admin and Meta classes.

Re: Django auth magic

[Brian William Kaplan]

Okay so I can probably easily do it but I'm too lazy. Also I need to
get the auth module to use our subclass of the User class and not the
original class. I'm honestly not sure how I should do this in such a
way it conforms to Django standard.

Re: Django auth magic

[Brian William Kaplan]

You guys are completely misintrepreting the point of this posting. It
has to do with the auth MODEL. Nothing else. When you say Model, you
are obviously referring to SQL. It is beyond me how you people come up
with LDAP and OpenID and all these other wishes.

If you want to use LDAP or OpenID, you obviously aren't going to use a
Model for authentication. Especially not a Model which uses a SQL
schema...

Re: Django auth magic

[Brian William Kaplan]

Nevow (http://www.nevow.com/) uses twisted.cred.

Intermittent IntegrityError on Model Save with auto_now and auto_now_add Fields

['William Palin' via Django developers (Contributions to Django itself)]

Hello Django community,


We are reaching out after encountering a persistent and elusive issue that 
manifests as an IntegrityError during the save operation of a Django model. 
This problem has been sporadically occurring since last year and has 
successfully stumped four seasoned Django developers on our team. The error 
seems to involve the non-update of auto_now fields upon model save, leading 
us to suspect a deeper issue within Django, though we are cautious about 
drawing premature conclusions. Given the complex and intermittent nature of 
this bug, we are seeking insights, advice, or any form of guidance from the 
community.

*Issue Overview:*


Our model inherits date_created and date_modified fields from an abstract 
base class designed to standardize these timestamps across our models. Here 
is how the abstract base class is defined:


class AbstractDateTimeModel(models.Model):

"""An abstract base class for most models"""



date_created = models.DateTimeField(

help_text="The moment when the item was created.",

auto_now_add=True,

db_index=True,

)

date_modified = models.DateTimeField(

help_text="The last moment when the item was modified. A value in 
year"

  " 1750 indicates the value is unknown",

auto_now=True,

db_index=True,

)


class Meta:

abstract = True

*The Problem:*

Intermittently, the .save() method triggers an IntegrityError, apparently 
because the auto_now field (date_modified) does not get created. A specific 
instance of this error showed that while date_created is correctly 
populated, date_modified remains null, which directly leads to the 
IntegrityError upon attempting to insert the record:

[datetime.datetime(2023, 10, 2, 14, 33, 49, 99833, 
tzinfo=datetime.timezone.utc), None, ...]

'INSERT INTO "search_docket" ("date_created", "date_modified",... ]

*What We've Tried:*



   - Investigated the possibility of an issue with update_fields being 
   non-null and excluding date_modified during .save(), but confirmed through 
   Sentry logs that update_fields was indeed None in all instances of the 
   error. 
   - Attempted to reproduce the issue in a controlled environment without 
   success, leaving us without a clear direction for a solution. 


*Request for Help:*


We're wondering if this could point to an undocumented edge case in 
Django's auto_now and auto_now_add implementation or a specific database 
behavior under certain conditions. Any advice on further debugging steps, 
experiences with similar issues, or knowledge of potential Django nuances 
that we might not be considering would be incredibly valuable.


We appreciate your time and any feedback or suggestions you can offer.


Thanks


Bill 


django v5.0.2 

python 3.12.2

db is postgres

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a788c065-3a96-472c-9b41-7c2aebca6967n%40googlegroups.com.