Re: How to disable system check framework?

[Marcin Nowak]

Thank, Josh.

I'm upgrading mature and complex project, which have some tricky parts 
implemented like a monkey-patched user model, which is applied after system 
chceck execution. So Django complains about missing fields in ModelAdmin, 
but patch works fine. There were some complains about doubled mappings to 
fields, but I've removed them. Generally speaking I don't need System 
Check, because I can find soultion myself other way.

Migrations are similar - they are too simple/limited for my needs. They 
operate on models, but I'm working mainly with database and I'm creating 
mappings for tables and views for easy querying. So I don't need builtin 
migrations feature.

I'm using commit_manually() frequently, but this decorator was removed... 

And so on...

That's why I feel that working with Django is harder. It's more strict, 
useful tools/helpers are missing, there is more features which should be 
bypassed/omitted. I feel that is harder to do simple things... Or maybe I'm 
getting old. ;)

Cheers,
Marcin


On Wednesday, August 5, 2015 at 3:08:26 AM UTC+2, Josh Smeaton wrote:
>
> Your "scare quotes" for "feature" are really disappointing. Especially 
> considering that the checks were hardcoded and couldn't be silenced prior 
> to the "system check framework".
>
> > Working with Django is getting harder with every new version.
>
> If you want to turn off new features or extension points, then I'd imagine 
> there'd be more work than for the regular user. Otherwise, I couldn't 
> disagree more.
>
> Anyhow, you want to turn off system checks for some reason. Here are some 
> potential ways forward:
>
> - Target the checks that are incorrectly spitting out errors or warnings, 
> and fix those.
> - Provide another setting "DISABLE_CHECKS" or something similar that will 
> prevent checks running altogether (probably not ideal, new settings are 
> hard to get into core, especially when the benefit here is still 
> questionable).
> - Provide a special value for the existing SILENCED_SYSTEM_CHECKS setting 
> that causes all checks to be silenced.
>
> You haven't really said much about the issue other than it bothers you. 
> The main purpose of checks are for development - so it doesn't have any 
> runtime or deployment time penalties. Perhaps if you took some time to 
> justify why users other than yourself might want to disable the check 
> framework, it'd be easier to start looking at solutions to problems.
>
> Cheers
>
>
> On Wednesday, 5 August 2015 01:16:21 UTC+10, Marcin Nowak wrote:
>>
>> Thanks, Carl. I know about silencing but I want to disable unwanted 
>> "feature". Working with Django is getting harder with every new version. 
>>
>>
>> On Tuesday, August 4, 2015 at 3:25:22 PM UTC+2, Carl Meyer wrote:
>>>
>>>
>>> > On Aug 4, 2015, at 9:18 AM, Marcin Nowak  
>>> wrote: 
>>> > 
>>> > I need to upgrade project to Django 1.8 but the SystemCheck Framework 
>>> bothers me. 
>>> > It complains about thigs that should work, even if they are a little 
>>> tricky. 
>>> > 
>>> > I need to disable system checks designed for blogs and other simple 
>>> sites. How can I do that? 
>>>
>>> The third paragraph of the system checks documentation [1] links to [2]. 
>>>
>>> Also, this is a question about using Django, not developing Django, so 
>>> it belongs on the django-users mailing list, not here. 
>>>
>>> Carl 
>>>
>>> [1] https://docs.djangoproject.com/en/1.8/topics/checks/ 
>>> [2] 
>>> https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SILENCED_SYSTEM_CHECKS
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/47132c06-4617-4119-b8f0-b0f166df2103%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Aymeric Augustin]

On 5 août 2015, at 11:09, Marcin Nowak  wrote:

> I'm using commit_manually() frequently, but this decorator was removed... 

Hi Marcin,

Sorry for derailing the discussion a bit, but could you clarify why you were
using this API? I removed it after failing to find a use case where it would
be preferrable to commit_on_success(). I’d love to know if I missed something
in order to avoid making similar mistakes again.

Using commit_manually() properly required great care -- and a try/finally
construct in practice -- to ensure that no code path escaped without a commit
or a rollback. It also required a massive amount of boilerplate to avoid
masking exceptions:

- https://groups.google.com/d/msg/django-developers/S3a5UAqAsmg/7C27UGl6G_0J
- https://code.djangoproject.com/ticket/6623

If you replace it with transaction.atomic() and remove the boilerplate, you
get the same behavior with ten times less code.

In case removing the boilerplate creates too much code churn, I documented a
real drop-in replacement that doesn't have the drawbacks of commit_manually:
https://docs.djangoproject.com/en/1.6/topics/db/transactions/#id5

What could I have done better?

If you don't want any changes in Django -- your posts to this list are often
after-the-fact complains about new features -- perhaps you could stick with
whatever version of Django you're currently running?

-- 
Aymeric.




-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/76ACD788-8644-40E3-A01E-5C8BEB81FCD4%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

Hi Aymeric,

This project is complex, as I said before, and works well with Django 1.4. 
But we need to upgrade it to newer version because of lack of support, 
security issues, and simplifcation of maintenance and growth (we're 
currently using three different Django versions, from 1.4 to 1.7, and we've 
decided to upgrade all subprojects to 1.8)

commit_manually() is used within long-live processes running as a mgmt 
commands, where is used multiprocessing too.
Whole process is complicated and hard to maintain. I would like to avoid 
logical / flow changes, whose are required to replace commit_manually() 
with atomic().
Second use case is a extract-transform-load subsytem, which also requires 
precise development of queries, commits and rollbacks. I'm afraid that 
upgrade of it may be also hard.

I think that there are rare cases, when manual transaction handling is 
required and should be available for punctilious developer.
*BUT* I can't say that atomic() does not fit my needs. I know how it work 
(I belive), but it may not cover current cases (implementation).

I must try to do replacement or bring old behaviour of commit_manually() 
with my custom wrapper. I'm trying to change code as less as possible.

You're asking what could be done better.. Generally speaking - an API 
compatibility layer. As a maintainer of big and old project I would like to 
import "commit_manually" from special module delivered with Django, which 
will guarantee stability and compatibility. This may be even undocumented. 
Manual commit() and rollback() functions are provided in transaction module 
(thanks so much for that!), so making compat layer should be possible, I 
think.

Kind Regards,
Marcin

On Wednesday, August 5, 2015 at 1:16:38 PM UTC+2, Aymeric Augustin wrote:
>
> On 5 août 2015, at 11:09, Marcin Nowak > 
> wrote: 
>
> > I'm using commit_manually() frequently, but this decorator was 
> removed... 
>
> Hi Marcin, 
>
> Sorry for derailing the discussion a bit, but could you clarify why you 
> were 
> using this API? I removed it after failing to find a use case where it 
> would 
> be preferrable to commit_on_success(). I’d love to know if I missed 
> something 
> in order to avoid making similar mistakes again. 
>
> Using commit_manually() properly required great care -- and a try/finally 
> construct in practice -- to ensure that no code path escaped without a 
> commit 
> or a rollback. It also required a massive amount of boilerplate to avoid 
> masking exceptions: 
>
> - 
> https://groups.google.com/d/msg/django-developers/S3a5UAqAsmg/7C27UGl6G_0J 
> - https://code.djangoproject.com/ticket/6623 
>
> If you replace it with transaction.atomic() and remove the boilerplate, 
> you 
> get the same behavior with ten times less code. 
>
> In case removing the boilerplate creates too much code churn, I documented 
> a 
> real drop-in replacement that doesn't have the drawbacks of 
> commit_manually: 
> https://docs.djangoproject.com/en/1.6/topics/db/transactions/#id5 
>
> What could I have done better? 
>
> If you don't want any changes in Django -- your posts to this list are 
> often 
> after-the-fact complains about new features -- perhaps you could stick 
> with 
> whatever version of Django you're currently running? 
>
> -- 
> Aymeric. 
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3257c139-f52e-422f-8eb4-5a97771ca367%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Carlton Gibson]

Sorry to be a pain but, could this thread be continued (if it's going to be) on 
Django Users?




(sorry)



—

On Wed, Aug 5, 2015 at 3:59 PM, Marcin Nowak 
wrote:

> Hi Aymeric,
> This project is complex, as I said before, and works well with Django 1.4. 
> But we need to upgrade it to newer version because of lack of support, 
> security issues, and simplifcation of maintenance and growth (we're 
> currently using three different Django versions, from 1.4 to 1.7, and we've 
> decided to upgrade all subprojects to 1.8)
> commit_manually() is used within long-live processes running as a mgmt 
> commands, where is used multiprocessing too.
> Whole process is complicated and hard to maintain. I would like to avoid 
> logical / flow changes, whose are required to replace commit_manually() 
> with atomic().
> Second use case is a extract-transform-load subsytem, which also requires 
> precise development of queries, commits and rollbacks. I'm afraid that 
> upgrade of it may be also hard.
> I think that there are rare cases, when manual transaction handling is 
> required and should be available for punctilious developer.
> *BUT* I can't say that atomic() does not fit my needs. I know how it work 
> (I belive), but it may not cover current cases (implementation).
> I must try to do replacement or bring old behaviour of commit_manually() 
> with my custom wrapper. I'm trying to change code as less as possible.
> You're asking what could be done better.. Generally speaking - an API 
> compatibility layer. As a maintainer of big and old project I would like to 
> import "commit_manually" from special module delivered with Django, which 
> will guarantee stability and compatibility. This may be even undocumented. 
> Manual commit() and rollback() functions are provided in transaction module 
> (thanks so much for that!), so making compat layer should be possible, I 
> think.
> Kind Regards,
> Marcin
> On Wednesday, August 5, 2015 at 1:16:38 PM UTC+2, Aymeric Augustin wrote:
>>
>> On 5 août 2015, at 11:09, Marcin Nowak > 
>> wrote: 
>>
>> > I'm using commit_manually() frequently, but this decorator was 
>> removed... 
>>
>> Hi Marcin, 
>>
>> Sorry for derailing the discussion a bit, but could you clarify why you 
>> were 
>> using this API? I removed it after failing to find a use case where it 
>> would 
>> be preferrable to commit_on_success(). I’d love to know if I missed 
>> something 
>> in order to avoid making similar mistakes again. 
>>
>> Using commit_manually() properly required great care -- and a try/finally 
>> construct in practice -- to ensure that no code path escaped without a 
>> commit 
>> or a rollback. It also required a massive amount of boilerplate to avoid 
>> masking exceptions: 
>>
>> - 
>> https://groups.google.com/d/msg/django-developers/S3a5UAqAsmg/7C27UGl6G_0J 
>> - https://code.djangoproject.com/ticket/6623 
>>
>> If you replace it with transaction.atomic() and remove the boilerplate, 
>> you 
>> get the same behavior with ten times less code. 
>>
>> In case removing the boilerplate creates too much code churn, I documented 
>> a 
>> real drop-in replacement that doesn't have the drawbacks of 
>> commit_manually: 
>> https://docs.djangoproject.com/en/1.6/topics/db/transactions/#id5 
>>
>> What could I have done better? 
>>
>> If you don't want any changes in Django -- your posts to this list are 
>> often 
>> after-the-fact complains about new features -- perhaps you could stick 
>> with 
>> whatever version of Django you're currently running? 
>>
>> -- 
>> Aymeric. 
>>
>>
>>
>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3257c139-f52e-422f-8eb4-5a97771ca367%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1438783672435.73354761%40Nodemailer.
For more options, visit https://groups.google.com/d/optout.

Re: View permissions to admin

[petr . dlouhy]

Hi Josh,

thanks for your reply. I was a little bit afraid of making PRs of 
interviewed code. So I made it now and the PR is 
here: https://github.com/django/django/pull/5108

Dne středa 5. srpna 2015 3:15:22 UTC+2 Josh Smeaton napsal(a):
>
> Hi Petr,
>
> Thanks for working on that ticket. I'm probably not the person to be 
> reviewing work in the admin, but here are a few ideas for getting your 
> patch noticed.
>
> 1. Create a pull request against the Django repository. Even if it's not 
> ready to be merged, a pull request makes it a lot easier to provide review 
> and comment on your changes.
> 2. Provide a description of what your patch is aiming to do in the pull 
> request comments. "view_permissions" for admin don't tell me a whole lot, 
> why do we need view permissions? Answering that question will help review.
>
> Writing to the mailing list here is also good, but having your code 
> available as a PR makes it easier for people to jump from here to your code 
> and provide feedback where it counts.
>
> Regards,
>
> On Wednesday, 5 August 2015 02:37:01 UTC+10, petr@auto-mat.cz wrote:
>>
>> Hi,
>>
>> 4 months ago, I have implemented view permissions for Django admin and 
>> posted it under following ticket:
>> https://code.djangoproject.com/ticket/8936
>>
>> The patch in my branch contains tests and documentation.
>>
>> I am willing to give my time to make this commited to official Django 
>> branch. But I would need to get some commits, suggestions and advises from 
>> more expert Django developers.
>>
>> Petr Dlouhý
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/10a8b3c8-a68e-4130-9bde-4364e62021d8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

It's ok. I think that the thread can be closed, except the suggestion of 
making compatibility layer, which is rather a kind of Django`s philosophy 
and may be continued in a new thread (if needed).



On Wednesday, August 5, 2015 at 4:08:53 PM UTC+2, Carlton wrote:
>
> Sorry to be a pain but, could this thread be continued (if it's going to 
> be) on Django Users?
>
> (sorry)
>
> —
>
>
>
> On Wed, Aug 5, 2015 at 3:59 PM, Marcin Nowak  > wrote:
>
>> Hi Aymeric,
>>
>> This project is complex, as I said before, and works well with Django 
>> 1.4. But we need to upgrade it to newer version because of lack of support, 
>> security issues, and simplifcation of maintenance and growth (we're 
>> currently using three different Django versions, from 1.4 to 1.7, and we've 
>> decided to upgrade all subprojects to 1.8)
>>
>> commit_manually() is used within long-live processes running as a mgmt 
>> commands, where is used multiprocessing too.
>> Whole process is complicated and hard to maintain. I would like to avoid 
>> logical / flow changes, whose are required to replace commit_manually() 
>> with atomic().
>> Second use case is a extract-transform-load subsytem, which also requires 
>> precise development of queries, commits and rollbacks. I'm afraid that 
>> upgrade of it may be also hard.
>>
>> I think that there are rare cases, when manual transaction handling is 
>> required and should be available for punctilious developer.
>> *BUT* I can't say that atomic() does not fit my needs. I know how it 
>> work (I belive), but it may not cover current cases (implementation).
>>
>> I must try to do replacement or bring old behaviour of commit_manually() 
>> with my custom wrapper. I'm trying to change code as less as possible.
>>
>> You're asking what could be done better.. Generally speaking - an API 
>> compatibility layer. As a maintainer of big and old project I would like to 
>> import "commit_manually" from special module delivered with Django, which 
>> will guarantee stability and compatibility. This may be even undocumented. 
>> Manual commit() and rollback() functions are provided in transaction module 
>> (thanks so much for that!), so making compat layer should be possible, I 
>> think.
>>
>> Kind Regards,
>> Marcin
>>
>> On Wednesday, August 5, 2015 at 1:16:38 PM UTC+2, Aymeric Augustin wrote:
>>>
>>> On 5 août 2015, at 11:09, Marcin Nowak  wrote: 
>>>
>>> > I'm using commit_manually() frequently, but this decorator was 
>>> removed... 
>>>
>>> Hi Marcin, 
>>>
>>> Sorry for derailing the discussion a bit, but could you clarify why you 
>>> were 
>>> using this API? I removed it after failing to find a use case where it 
>>> would 
>>> be preferrable to commit_on_success(). I’d love to know if I missed 
>>> something 
>>> in order to avoid making similar mistakes again. 
>>>
>>> Using commit_manually() properly required great care -- and a 
>>> try/finally 
>>> construct in practice -- to ensure that no code path escaped without a 
>>> commit 
>>> or a rollback. It also required a massive amount of boilerplate to avoid 
>>> masking exceptions: 
>>>
>>> - 
>>> https://groups.google.com/d/msg/django-developers/S3a5UAqAsmg/7C27UGl6G_0J 
>>> - https://code.djangoproject.com/ticket/6623 
>>>
>>> If you replace it with transaction.atomic() and remove the boilerplate, 
>>> you 
>>> get the same behavior with ten times less code. 
>>>
>>> In case removing the boilerplate creates too much code churn, I 
>>> documented a 
>>> real drop-in replacement that doesn't have the drawbacks of 
>>> commit_manually: 
>>> https://docs.djangoproject.com/en/1.6/topics/db/transactions/#id5 
>>>
>>> What could I have done better? 
>>>
>>> If you don't want any changes in Django -- your posts to this list are 
>>> often 
>>> after-the-fact complains about new features -- perhaps you could stick 
>>> with 
>>> whatever version of Django you're currently running? 
>>>
>>> -- 
>>> Aymeric. 
>>>
>>>
>>>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/3257c139-f52e-422f-8eb4-5a97771ca367%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-develo

Re: How to disable system check framework?

[Florian Apolloner]

On Wednesday, August 5, 2015 at 3:59:39 PM UTC+2, Marcin Nowak wrote:
>
> 
>
I must try to do replacement or bring old behaviour of commit_manually() 
> with my custom wrapper. I'm trying to change code as less as possible.
>

To be honest the "issues" you outlined here seem to suggest a bigger 
problem within your own code. I am aware that it is really hard and 
annoying to change from old style transactions to the new ones in big 
projects, but given the fact that atomic() just works and I have yet to see 
a project which used old style transactions properly (see Aymerics 
presentation on transactions), I think it is worth it.
 

> You're asking what could be done better.. Generally speaking - an API 
> compatibility layer. As a maintainer of big and old project I would like to 
> import "commit_manually" from special module delivered with Django, which 
> will guarantee stability and compatibility.
>

And defer upgrading to a proper transaction management for another few 
years? Seriously, if your project is to big to upgrade stick with 1.4 and 
use a distribution which provides LTS support for it (but yes, ultimately 
you will have to switch, but from what I hear Redhat is going to support 
1.4 for another while :D)=…

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ade4b6f6-14e2-412e-bd42-ad725df96bf2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Marcin Nowak]

No, there is not problem in this code only, but yeah - it is a big problem. 
Real problem is that newer versions of Django are pushing developers to do 
things by the only one way, which is good for blogs or cms, or maybe new 
and small projects. As the result it is getting harder to maintain complex 
project based on Django unless you stay with old-and-unsupported version. 
And I'm not talking about atomic(), which is good and simple.

> And defer upgrading to a proper transaction management for another few 
years?

No. Just provide old-style wrappers (at the python-level api compatibility) 
for old guys maintaining old projects ;)

I can say something similar about generics views based on CBV - function 
based wrappers can be built top of CBV and can be provided in django.compat 
or django.legacy package. Just suggesting.
 
Cheers,
Marcin

On Wednesday, August 5, 2015 at 5:28:39 PM UTC+2, Florian Apolloner wrote:
>
> On Wednesday, August 5, 2015 at 3:59:39 PM UTC+2, Marcin Nowak wrote:
>>
>> 
>>
> I must try to do replacement or bring old behaviour of commit_manually() 
>> with my custom wrapper. I'm trying to change code as less as possible.
>>
>
> To be honest the "issues" you outlined here seem to suggest a bigger 
> problem within your own code. I am aware that it is really hard and 
> annoying to change from old style transactions to the new ones in big 
> projects, but given the fact that atomic() just works and I have yet to see 
> a project which used old style transactions properly (see Aymerics 
> presentation on transactions), I think it is worth it.
>  
>
>> You're asking what could be done better.. Generally speaking - an API 
>> compatibility layer. As a maintainer of big and old project I would like to 
>> import "commit_manually" from special module delivered with Django, which 
>> will guarantee stability and compatibility.
>>
>
> And defer upgrading to a proper transaction management for another few 
> years? Seriously, if your project is to big to upgrade stick with 1.4 and 
> use a distribution which provides LTS support for it (but yes, ultimately 
> you will have to switch, but from what I hear Redhat is going to support 
> 1.4 for another while :D)=…
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c2fc9c5a-dc87-49e2-aed3-064a3e2135b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[multidb] emit_pre_migrate_signal / emit_post_migrate_signal in Django 1.8

[Marcin Nowak]

Hi, it's me again.

I'm managing migrations myself without using builtin migrations.

My project is using many databases and I'm calling schema migration 
separately for each database, providing database alias as a command 
argument. 
I'm also emitting pre- and post- migrate signals.

The problem is with new Django (teste with 1.8.3). 

When updating schema for 'default' database (which contains all 
Django-related tables, like auth, contenttypes, and so on), evertyhing 
works well.

But when updating other databases (without auth, contenttypes, etc), and 
when emitting post migrate signal with database alias other than 'default', 
Django is reading all registered apps and calls (for example) 
"django.auth.management.create_permissions", even if Permission model (or 
ContentType) is routed to 'default' alias. As a result Django tries to 
insert data into tables whose does not exists in specified database.

I can bypass emitting signals for databases other than 'default', but the 
problem still remains in Django itself. 
I think that Django should use database routers to check whether model is 
managed/writeable.

Let's assume that we have two databases: 'default' an 'other'.
Signal is emitting with db alias 'other'.
Django calls 
https://github.com/django/django/blob/1.8.3/django/contrib/auth/management/__init__.py#L82
 
with alias 'other'.
Then tries to get ContentType using='other'. But the 'other' database has 
no such table.

Is that feature or bug?
Should I create contrib.* tables for each database?

Kind Regards,
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8547aa2b-ae44-4cc7-b000-0683f4533789%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [multidb] emit_pre_migrate_signal / emit_post_migrate_signal in Django 1.8

[Tim Graham]

There is a check a few lines above the snippet you linked for 
"allow_migrate_model('other', Permission)". Is that returning the correct 
result? I don't think you can create permissions in the 'other' db without 
content types since they are linked by a foreign key.

p.s. "Is it a bug" questions should go to django-users, thanks!

On Wednesday, August 5, 2015 at 1:08:05 PM UTC-4, Marcin Nowak wrote:
>
> Hi, it's me again.
>
> I'm managing migrations myself without using builtin migrations.
>
> My project is using many databases and I'm calling schema migration 
> separately for each database, providing database alias as a command 
> argument. 
> I'm also emitting pre- and post- migrate signals.
>
> The problem is with new Django (teste with 1.8.3). 
>
> When updating schema for 'default' database (which contains all 
> Django-related tables, like auth, contenttypes, and so on), evertyhing 
> works well.
>
> But when updating other databases (without auth, contenttypes, etc), and 
> when emitting post migrate signal with database alias other than 'default', 
> Django is reading all registered apps and calls (for example) 
> "django.auth.management.create_permissions", even if Permission model (or 
> ContentType) is routed to 'default' alias. As a result Django tries to 
> insert data into tables whose does not exists in specified database.
>
> I can bypass emitting signals for databases other than 'default', but the 
> problem still remains in Django itself. 
> I think that Django should use database routers to check whether model is 
> managed/writeable.
>
> Let's assume that we have two databases: 'default' an 'other'.
> Signal is emitting with db alias 'other'.
> Django calls 
> https://github.com/django/django/blob/1.8.3/django/contrib/auth/management/__init__.py#L82
>  
> with alias 'other'.
> Then tries to get ContentType using='other'. But the 'other' database has 
> no such table.
>
> Is that feature or bug?
> Should I create contrib.* tables for each database?
>
> Kind Regards,
> Marcin
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/41ba2a32-426f-4345-9f75-faf0f1401dcc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Tim Graham]

If you have a big need to use old APIs, feel free to implement them in your 
own package or even in a reusable app so that others can benefit too. There 
have been other efforts like https://github.com/arteria/django-compat to do 
so. We can't maintain them forever, lest Django become bloated and 
difficult to maintain.

If you followed the discussion about the new deprecation policy, you'll 
know that we are trying to make it easier to upgrade from one LTS to the 
next, but some things are inevitably going to change or be removed in an 
effort to improve Django. You can help us by testing preleases and giving 
feedback before final releases.

On Wednesday, August 5, 2015 at 12:49:15 PM UTC-4, Marcin Nowak wrote:
>
> No, there is not problem in this code only, but yeah - it is a big 
> problem. Real problem is that newer versions of Django are pushing 
> developers to do things by the only one way, which is good for blogs or 
> cms, or maybe new and small projects. As the result it is getting harder to 
> maintain complex project based on Django unless you stay with 
> old-and-unsupported version. And I'm not talking about atomic(), which is 
> good and simple.
>
> > And defer upgrading to a proper transaction management for another few 
> years?
>
> No. Just provide old-style wrappers (at the python-level api 
> compatibility) for old guys maintaining old projects ;)
>
> I can say something similar about generics views based on CBV - function 
> based wrappers can be built top of CBV and can be provided in django.compat 
> or django.legacy package. Just suggesting.
>  
> Cheers,
> Marcin
>
> On Wednesday, August 5, 2015 at 5:28:39 PM UTC+2, Florian Apolloner wrote:
>>
>> On Wednesday, August 5, 2015 at 3:59:39 PM UTC+2, Marcin Nowak wrote:
>>>
>>> 
>>>
>> I must try to do replacement or bring old behaviour of commit_manually() 
>>> with my custom wrapper. I'm trying to change code as less as possible.
>>>
>>
>> To be honest the "issues" you outlined here seem to suggest a bigger 
>> problem within your own code. I am aware that it is really hard and 
>> annoying to change from old style transactions to the new ones in big 
>> projects, but given the fact that atomic() just works and I have yet to see 
>> a project which used old style transactions properly (see Aymerics 
>> presentation on transactions), I think it is worth it.
>>  
>>
>>> You're asking what could be done better.. Generally speaking - an API 
>>> compatibility layer. As a maintainer of big and old project I would like to 
>>> import "commit_manually" from special module delivered with Django, which 
>>> will guarantee stability and compatibility.
>>>
>>
>> And defer upgrading to a proper transaction management for another few 
>> years? Seriously, if your project is to big to upgrade stick with 1.4 and 
>> use a distribution which provides LTS support for it (but yes, ultimately 
>> you will have to switch, but from what I hear Redhat is going to support 
>> 1.4 for another while :D)=…
>>
>> Cheers,
>> Florian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4bb9107e-767c-465e-974c-f24848f9d41d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [multidb] emit_pre_migrate_signal / emit_post_migrate_signal in Django 1.8

[Marcin Nowak]

On Wednesday, August 5, 2015 at 7:42:16 PM UTC+2, Tim Graham wrote:
>
> There is a check a few lines above the snippet you linked for 
> "allow_migrate_model('other', Permission)". Is that returning the correct 
> result? I don't think you can create permissions in the 'other' db without 
> content types since they are linked by a foreign key.
>
>
Probably you've got the point! I didn't pay enough attention to this method 
and didn't saw this check.
 

> p.s. "Is it a bug" questions should go to django-users, thanks!
>
>>
>>
Yes, sir! Just thought that there is a better place to ask about internals, 
but I was wrong I suppose.
 

Thanks, Tim!
Marcin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c8c2bac2-831d-46a1-a1dd-fb573974cf43%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to disable system check framework?

[Aymeric Augustin]

Hi Marcin,

Thanks for the clarification, that's helpful.

The backwards compatibility wrapper looks like this in this case:

@contextlib.contextmanager
def commit_manually(using=None):
transaction.set_autocommit(False, using)
try:
yield
finally:
transaction.set_autocommit(True, using)

(typed on a phone and not tested)

We recently decided against shipping compatibility wrappers with Django because 
the interaction with the deprecation policy is messy. However, like Tim said, 
we're trying to make it easier to have such wrappers.

-- 
Aymeric.

> Le 5 août 2015 à 15:59, Marcin Nowak  a écrit :
> 
> Hi Aymeric,
> 
> This project is complex, as I said before, and works well with Django 1.4. 
> But we need to upgrade it to newer version because of lack of support, 
> security issues, and simplifcation of maintenance and growth (we're currently 
> using three different Django versions, from 1.4 to 1.7, and we've decided to 
> upgrade all subprojects to 1.8)
> 
> commit_manually() is used within long-live processes running as a mgmt 
> commands, where is used multiprocessing too.
> Whole process is complicated and hard to maintain. I would like to avoid 
> logical / flow changes, whose are required to replace commit_manually() with 
> atomic().
> Second use case is a extract-transform-load subsytem, which also requires 
> precise development of queries, commits and rollbacks. I'm afraid that 
> upgrade of it may be also hard.
> 
> I think that there are rare cases, when manual transaction handling is 
> required and should be available for punctilious developer.
> BUT I can't say that atomic() does not fit my needs. I know how it work (I 
> belive), but it may not cover current cases (implementation).
> 
> I must try to do replacement or bring old behaviour of commit_manually() with 
> my custom wrapper. I'm trying to change code as less as possible.
> 
> You're asking what could be done better.. Generally speaking - an API 
> compatibility layer. As a maintainer of big and old project I would like to 
> import "commit_manually" from special module delivered with Django, which 
> will guarantee stability and compatibility. This may be even undocumented. 
> Manual commit() and rollback() functions are provided in transaction module 
> (thanks so much for that!), so making compat layer should be possible, I 
> think.
> 
> Kind Regards,
> Marcin
> 
>> On Wednesday, August 5, 2015 at 1:16:38 PM UTC+2, Aymeric Augustin wrote:
>> On 5 août 2015, at 11:09, Marcin Nowak  wrote: 
>> 
>> > I'm using commit_manually() frequently, but this decorator was removed... 
>> 
>> Hi Marcin, 
>> 
>> Sorry for derailing the discussion a bit, but could you clarify why you were 
>> using this API? I removed it after failing to find a use case where it would 
>> be preferrable to commit_on_success(). I’d love to know if I missed 
>> something 
>> in order to avoid making similar mistakes again. 
>> 
>> Using commit_manually() properly required great care -- and a try/finally 
>> construct in practice -- to ensure that no code path escaped without a 
>> commit 
>> or a rollback. It also required a massive amount of boilerplate to avoid 
>> masking exceptions: 
>> 
>> - https://groups.google.com/d/msg/django-developers/S3a5UAqAsmg/7C27UGl6G_0J 
>> - https://code.djangoproject.com/ticket/6623 
>> 
>> If you replace it with transaction.atomic() and remove the boilerplate, you 
>> get the same behavior with ten times less code. 
>> 
>> In case removing the boilerplate creates too much code churn, I documented a 
>> real drop-in replacement that doesn't have the drawbacks of commit_manually: 
>> https://docs.djangoproject.com/en/1.6/topics/db/transactions/#id5 
>> 
>> What could I have done better? 
>> 
>> If you don't want any changes in Django -- your posts to this list are often 
>> after-the-fact complains about new features -- perhaps you could stick with 
>> whatever version of Django you're currently running? 
>> 
>> -- 
>> Aymeric.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3257c139-f52e-422f-8eb4-5a97771ca367%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com