date:20150728

Re: Migrations in Django 1.7 make unit testing models harder

2015-07-28 Thread Kevin Tran

I have found a good alternative to the old initial data feature.  It's 
called django-update-initial-data.

https://github.com/ambitioninc/django-dynamic-initial-data

MySQL compatible version:
https://github.com/minervaproject/django-dynamic-initial-data/tree/mysql

It's actually better than the old initial data feature because, "Django 
Dynamic Initial Data also handles when objects are deleted from initial 
data, a feature that Django's initial data fixture system lacks.".

On Tuesday, May 12, 2015 at 9:51:59 PM UTC-7, Pradeek J wrote:
>
> I'm facing the same speed issue here. Like Andrew mentioned above, if 
> initial data is going to be removed and data migrations are the way 
> forward, even having an option to skip migrations is a problem because we'd 
> need that data to be populated. Is there something planned for this for 
> Django 1.9? 
>
> On Saturday, December 27, 2014 at 8:10:10 PM UTC+5:30, Tim Graham wrote:
>>
>> In pull request 3791 , I 
>> started working on the code removals for Django 1.9 to better understand 
>> how the database/migrations code will look and what APIs we still need to 
>> deprecate. The good news is that we can keep support for apps without 
>> migrations (at least for Django's test suite) without adding any additional 
>> code or using any deprecated database creation APIs (as Claude suggested 
>> earlier). If you look at the commit "Removed supported for syncing apps 
>> without migrations" in that PR, you'll see "Run the syncdb phase" 
>> remains in the migrate management command, but now it'll only be executed 
>> when testing (and it's also quite a bit simplified due to other removals 
>> like support for initial data). It seems like it should be pretty simple to 
>> implement something similar to the old *SOUTH_TESTS_MIGRATE* setting in 
>> order to allow disabling migrations for all apps when testing if we wanted 
>> to go that route. I think the remaining deprecations to move forward with 
>> ticket 
>> #22340  (Legacy Table 
>> Creation Methods Not Properly Deprecated) are straightforward and will 
>> leave a comment there about it.
>>
>> On Saturday, December 20, 2014 2:18:17 PM UTC-5, Aymeric Augustin wrote:
>>>
>>> There's django-admin test --keepdb for this purpose (in Django 1.8).
>>>
>>> -- 
>>> Aymeric.
>>>
>>> Le 20 déc. 2014 à 18:41, Bernard Sumption  a 
>>> écrit :
>>>
>>> A proposition: the problem isn't that migrations take 80 seconds, it's 
>>> that this 80 seconds is repeated every time a developer runs tests. How 
>>> about serialising the structure and content of the database to disk after 
>>> applying migrations, then whenever you need a migrated test database, load 
>>> it from disk unless the migration files have been altered since the 
>>> serialised version was created?
>>>
>>> Bernie :o)
>>>
>>> On 20 December 2014 at 10:52, Andrew Godwin  wrote:

 Clause, I believe that line is to allow people to run makemigrations 
 when AUTH_USER_MODEL points to an app that doesn't yet have migrations; 
 before, it would fail hard, as the AUTH_USER_MODEL was not in the migrate 
 state and so nothing could use it, and you couldn't run makemigrations to 
 create it as it would error out.

 I'd test variations of that to see if your patch still works; if not, 
 we'll need a workaround again.

 Andrew

 On Sat, Dec 20, 2014 at 10:47 AM, Claude Paroz  
 wrote:
>
> On Friday, December 19, 2014 6:30:32 PM UTC+1, Tim Graham wrote:
>>
>> Yes. Claude has worked on the deprecation in 
>> https://github.com/django/django/pull/3220 but it requires adding 
>> more migrations to our test suite and he noted that each migration we 
>> add 
>> to Django's test suite adds up to ~3.5 seconds to the run time of the 
>> test 
>> suite. He also worked on some speed ups to mitigate this in 
>> https://github.com/django/django/pull/3484 but there are some 
>> unsolved issues.
>>
>
> I think that the first mentioned patch I worked on (PR 3220) shows 
> that it should be possible to use the new schema infrastructure without 
> requiring migrations, at least for apps that don't depend on other 
> migrated 
> apps. Tell me if I miss anything.
>
> But surely, I'd really like to solve first the speed issue of 
> migrations (#23745, PR 3484). Chris, if you have the opportunity to test 
> that patch with your database, it would be great. The way forward for 
> this 
> patch is to first test the "if app_label == settings.AUTH_USER_MODEL 
> and ignore_swappable:" line which is currently not tested and is 
> probably not addressed by my patch. Andrew, do you remember why you 
> introduced that part of the code?
>
> Claude
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups

@staff_member_required handling of non-staff users

2015-07-28 Thread Tim Graham

We received a ticket [1] noting that when a non-staff user tries to access
an admin page, they will be redirected to the admin login page with no
explanation. A pull request [2] proposes to add this message to login page
if a user is authenticated, "While you are authenticated as {{ username }},
you are unfortunately not authorized to access this page -- would you care
to re-login?" -- the thinking being that any authenticated user viewing the
login page will have reached it via such a redirect. (Logged in staff users
cannot view the login page because they are redirected back to the admin
index page if they try to access it).

I think it's a bit odd for the @staff_member_required to redirect non-staff
users to the login page (which is a side effect of using the
user_passes_test() function) as if many users have two separate "staff" and
"non-staff" accounts. Instead I propose to raise PermissionDenied so that
developers can use handler403 to customize the behavior.

What are your thoughts on this issue?

[1] https://code.djangoproject.com/ticket/25163
[2] https://github.com/django/django/pull/5044

--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/7c79d452-977a-48db-9613-6d9f912f1fff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

A new setting for custom gettext domains?

2015-07-28 Thread Tim Graham

Since adding new settings is sometimes controversial, I wanted to ask for 
thoughts about this ticket [1] / pull request [2] which proposes a new 
LOCALE_FILENAMES 
setting to allow using additional gettext domains besides "django". I don't 
use translations myself, so thoughts about whether the usefulness of the 
feature justifies a new setting or if there is some other solution to the 
problem described in the ticket would be welcome.

[1] https://code.djangoproject.com/ticket/6376
[2] https://github.com/django/django/pull/5053

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/17c0ba39-47b6-4ce5-988c-808d877c8a61%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: @staff_member_required handling of non-staff users

2015-07-28 Thread Carl Meyer

On 07/28/2015 12:36 PM, Tim Graham wrote:
> We received a ticket [1] noting that when a non-staff user tries to
> access an admin page, they will be redirected to the admin login page
> with no explanation. A pull request [2] proposes to add this message to
> login page if a user is authenticated, "While you are authenticated as
> {{ username }}, you are unfortunately not authorized to access this page
> -- would you care to re-login?" -- the thinking being that any
> authenticated user viewing the login page will have reached it via such
> a redirect. (Logged in staff users cannot view the login page because
> they are redirected back to the admin index page if they try to access it).

This solution seems reasonable to me.

> I think it's a bit odd for the @staff_member_required to redirect
> non-staff users to the login page (which is a side effect of using the
> user_passes_test() function) as if many users have two separate "staff"
> and "non-staff" accounts.

I think that probably varies quite a bit by project. There are certainly
projects (especially on staging sites where a lot of testing occurs that
sometimes requires multiple accounts) where I do have multiple accounts,
one staff and others not, and I've used the re-login ability.

> Instead I propose to raise PermissionDenied so
> that developers can use handler403 to customize the behavior.

I guess this should be somewhat practical now that we have
https://github.com/django/django/commit/70779d9c1cab77791c73b72e8a63f60184d8f2b0
-- without that it would be hard to distinguish one type of 403 from
another.

It feels a bit ugly to me in the cases where you want to return
something other than a 403 (e.g. the 302 that is returned currently)
that you'd have to do that via `handler403`. And you'd still have the
`got_request_exception` signal sent, even though you are choosing to
handle it in a non-exceptional fashion.

> What are your thoughts on this issue?

I think the proposed PR for a UX hint is a good improvement on the
status quo, regardless.

In theory I like the idea of being able to customize what happens when a
`staff_member_required` check fails. For that matter, I'd like it if
`user_passes_test` itself were more customizable, so it could be
configured to return either a redirect, a 403, or a 404.

I am not convinced that `handler403` is the right place to do this
customization. To me that seems like the place to customize your 403
responses, not to make policy decisions that may result in returning
something that isn't a 403 at all.

The obvious place for this type of customization to happen is in an
optional parameter to `@staff_member_required`. The problem in the admin
is that the decorator is applied internally, not by the developer.
Perhaps an `AdminSite` attribute?

Carl

> [1] https://code.djangoproject.com/ticket/25163
> [2] https://github.com/django/django/pull/5044

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55B7E329.90503%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.

signature.asc
Description: OpenPGP digital signature

Re: Feature idea: Add support for PostgreSQL's COPY command in contrib.postgres

2015-07-28 Thread Aaron Williams

+1 for this feature addition.

I work with a lot of public data and I almost always go through the steps 
of loading data into PostgreSQL and building from there. COPY reduces data 
load time significantly so a core load command for Django is welcome.

I've used LOAD DATA INFILE for MySQL on occasion too, but I'd rather stick 
to Django's ORM (and PostgreSQL) instead of writing raw SQL as Ben 
mentioned.

Another neat idea is to borrow from GeoDjango's ogrinspect 

 and 
do the same thing for CSVs (I wrote a simple implementation here 
).
 
These two tools together will be huge leap in productivity for the data 
journalism / open data communities.

I should add I just used django-postgres-copy for a project I'm working on 
and it ran like a charm.

On Saturday, July 18, 2015 at 3:19:49 PM UTC-7, Ben Welsh wrote:
>
> Hello,
>
> I am a big fan of PostgreSQL's excellent bulk loader COPY 
> . It can rapidly 
> load a CSV file directly into the database, an oftentimes tedious task that 
> I have to do frequently as part of my job. 
>
> I am not a big fan of having to write my COPY commands in raw SQL. I'd 
> much rather use Django's ORM. 
>
> So last week I put together an app call django-postgres-copy 
>  that 
> attempts to integrate COPY into Django, modeling its design on the 
> excellent LayerMapping 
>  
> utility in contrib.gis, which I also use frequently. 
>
> I wrote a blog post about the approach here
>
> http://www.californiacivicdata.org/2015/07/17/hello-django-postgres-copy/
>
> You can find more complete technical documentation here
>
> http://django-postgres-copy.californiacivicdata.org/en/latest/
>
> And all of the code is up here on GitHub
>
> https://github.com/california-civic-data-coalition/django-postgres-copy
>
> Since Django has already begun to integrate other PostgreSQL-specific 
> features in contrib.postgres, I'm curious if the core developers are be 
> interested in adding COPY support as well. 
>
> I'm not attached to the style of my library and I'd welcome a different 
> approach if it got the job done. I'd be thrilled to have the opportunity to 
> carry the torch and do whatever refactoring and additional coding is 
> necessary to qualify it for a merge.
>
> Please let me know what you think. And if I've overlooked some previous 
> discussion or superior third-party library in this area, please forgive me. 
> I searched around but was unable to find anything.
>
> Sincerely,
>
> Ben Welsh
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a128fbdb-181f-400b-b810-8c451f735a62%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Making max_length argument optional

2015-07-28 Thread Podrigal, Aron

Hi,

I am using postgresql and I prefer my VARCHAR columns not to have a length
limit. Is there any particular reason why max_length arg to fields is
required. If for  compatibility with other database backends we can have
some sane default if it is None.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANJp-yjwzpk%2BN2%3D4WWbmj5zu2Gwu67G0Sme1f8xQa%2BKrzSQ2Rw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

default values on database level

2015-07-28 Thread Podrigal, Aron

I would like to propose making Fields.default to rely on the database
generated defaults where possible. This could be implemented by having
some  constants like fields.CURRENT_TIMESTAMP, etc.  The tricky part here
is to detect database backend specific capabilities and have a python
equivalent fallback.

Any thoughts?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANJp-yi5%2Bu%3DW8PdUXtcot%2BO8%3Df35dVLbStwVArJhU7gDnaNXoA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making max_length argument optional

2015-07-28 Thread Loïc Bistuer

Hi Aron,

I'm +1 on addressing this, I often don't care about max_length, but I still 
want a terse form representation. In my projects I use a subclass of TextField 
that sets a TextInput wiget in its formfield() method. But that's not very 
elegant: it requires a custom field for a common use-case, and it's tightly 
coupled to Django forms (i.e. won't work with other form libraries and non-HTML 
forms).

Now how we address this depends on the meaning we want to give to models fields 
like CharField or TextField, more specifically whether the emphasis is on end 
user's perspective or on mapping 1:1 to database concepts.

If we see CharField and TextField as respectively "short/single line field" and 
"long/multiline field" from a user's perspective, then it makes sense to lift 
the max_length requirement and use `db_type='TextField'` when max_length is 
None.

However if we want to stay close to their database equivalents, then we could 
have a display hint on TextField (e.g. TextField(display_multiline=False)) and 
Field.formfield() would use that hint to provide a TextInput widget instead of 
a Textarea.

Personally I'd much prefer option 1, it's significantly less verbose and more 
often than not I don't want a length constraint, especially considering that 
there is no performance penalty in doing so with all the databases that I care 
about. Also considering we want first class virtual field support (and plan on 
turning existing concrete fields into virtual fields) I don't think it's a big 
problem to break from the existing models.Field/db types mapping.

Cheers,
Loïc

> On Jul 29, 2015, at 11:27, Podrigal, Aron  wrote:
> 
> Hi,
> 
> I am using postgresql and I prefer my VARCHAR columns not to have a length 
> limit. Is there any particular reason why max_length arg to fields is 
> required. If for  compatibility with other database backends we can have some 
> sane default if it is None.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CANJp-yjwzpk%2BN2%3D4WWbmj5zu2Gwu67G0Sme1f8xQa%2BKrzSQ2Rw%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/EB67D8D3-1633-4CE7-9A7D-E22FA1EB4A47%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: default values on database level