Re: Database pooling vs. persistent connections

[Aymeric Augustin]

Hi Shai,

Thanks for compiling this list ! My answers below.

> First -- the very mention of this topic reminded me of 
> https://code.djangoproject.com/ticket/9964, which was my pet bug for about 
> two 
> years. The point of that bug, though, was to make sure that transactions are 
> properly closed before (and regardless if) the connection is closed, so while 
> related, it should be unaffected.

If anything, persistent connections will *require* proper transaction 
termination
at the end of each request cycle.

> Second -- Tasks out of requests. Core developers participating in this 
> discussion already have this in mind, Aymeric has commented about it in 
> https://code.djangoproject.com/ticket/17887, but it was left out of the 
> discussion on the thread. The suggested changes, AFAICT, modify the behavior 
> around the end of requests only -- for tasks, nobody closed the connection 
> before, and nobody is going to do anything different now; so that's not 
> "something missed" either.

Yes, this patch must not affect behavior outside of the request handler.

> Third -- different use patterns of multi-db. AFAICT the change treats all 
> connections as equals, so no new problems should arise, but I can't remove a 
> gnawing suspicion that some interaction may pop up. In particular, I'm 
> worried 
> because not all such patterns are automatically tested; I know my pet pattern 
> (two aliases for one db, for different transaction behavior -- for logging 
> into 
> db, which shouldn't be reverted when the main transaction is rolled back) has 
> had problems unnoticed by others (see e.g.  
> https://code.djangoproject.com/ticket/14415).

The problem described in that ticket occurred because of the invalid
assumption that each connection was to a different database.

As far as I can tell I'm not making this assumption anywhere. Transaction state
is managed per-connection, and the representation of a connection in Django
is very straightforward.

> And last (and probably least) -- coordinated distributed transactions and two-
> phase-commit are not really supported by Django, and if you want them you 
> need 
> to write backend-specific code (probably a custom backend -- I haven't had 
> the 
> [mis]fortune to need to do this yet). I suspect such code would be affected, 
> I'm not sure if it becomes easier or harder to write. I know the issue is 
> mostly theoretical, and even if there is a problem with it, that's not a 
> blocker; still, I thought it was worth a mention.


I've never thought about this and I'm not sure what the interactions are. At 
worst,
you can restore the previous behavior by setting the connection's max age to 
zero.

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Database pooling vs. persistent connections

[Anssi Kääriäinen]

On 22 helmi, 11:21, Aymeric Augustin
 wrote:
> > Second -- Tasks out of requests. Core developers participating in this
> > discussion already have this in mind, Aymeric has commented about it in
> >https://code.djangoproject.com/ticket/17887, but it was left out of the
> > discussion on the thread. The suggested changes, AFAICT, modify the behavior
> > around the end of requests only -- for tasks, nobody closed the connection
> > before, and nobody is going to do anything different now; so that's not
> > "something missed" either.
>
> Yes, this patch must not affect behavior outside of the request handler.

The way connections are handled in requests could be useful for task
queues, too. The tasks should be separated from each other, so the
wanted properties are:
  - Make sure you have fresh connection state (transaction state for
example) per task
  - Make sure that if a connection is closed inside some task, the
connection will be reopened for the next task.
  - For performance reasons always closing connections after each task
isn't wanted.

This is exactly what is wanted in request processing, too.

For example django-celery does already have their version of
persistent connections, see: 
https://github.com/celery/django-celery/blob/master/djcelery/loaders.py#L84

So, persistent connections will be usable outside request processing
as long as they are opt-in. Supporting persistent connections outside
request processing isn't needed for the initial implementation, of
course.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Ticket #17093 -- quarantine global state in django.template

[Tom Christie]

Hi Christopher,

> OK, let mi introduce the idea for Google Some Of Code of this year. The 
idea is stolen from Pyramid [3]. The main new concept is a renderer. A 
renderer may be current TemplateEngine as well as JSON serializer or mako 
renderer.

I can't really comment on if it'd be appropriate for GSoC or not, but it'd 
probably be worth you taking a look at Django REST 
framework's 
renderers, which do pretty much exactly what you described.

Simple example of rendering to JSON:

class MyView(APIView):
renderer_classes = (JSONRenderer,)

def get(self, request, *args, **kwargs):
return Response({'desc':'my response to AJAX request'})

Rendering to HTML, using template & template context.

class MyView(APIView):
renderer_classes = (HTMLTemplateRenderer,)

def get(self, request, *args, **kwargs):
return Response({'desc':'my response to AJAX request'}, 
template_name='example.html')

There's a bit more to it than that, but I thought I'd mention it since it's 
so similar to the API you outlined.


On Thursday, 21 February 2013 15:16:09 UTC, Christopher Medrela wrote:
>
> Hello. I would like to report progress of my work (not too much new 
> things, rather cleaning and polishing), discuss some issues and propose an 
> idea for next Google Summer Of Code.
>
> Progres from last time:
>
>- Solved problem with TemplateResponse -- it receives optional keyword 
>argument called `engine` (if omitted, then it calls get_default_engine). 
>During pickling the engine is lost. During repickling get_default_engine() 
>is called.
>- Dropped idea of invalidating caches: base.invalid_var_format_string 
>and context._standard_context_processors. These settings are used in about 
>a dozen places in tests and it's not really necessary now.
>- Renamed _Template back to Template. The Template class accepts 
>optional keyword argument called `engine`. If omitted, then it calls 
>get_default_engine().
>
>
> I've also rebased commits so the branch forks from updated master. I've 
> reordered and squashed commits and rewritten description of commits so all 
> work ended in less then twenty well-described commits.
>
> I tried to make all tests green before next weekend (Django Sprint is 
> coming!), but I failed. There are some failing tests (exactly 450 xD); they 
> are heisenbugs -- they appear only when you run all tests; if you run only 
> one test suit, then they disappear. I guess that the problem is that the 
> default_engine should be invalidated between some tests. I also bisected 
> commits and I found out that the commit introducing failures is [1]. I will 
> try to fix these problems before the sprint on Saturday, because the 
> current state is almost ready to review (and to merge, I hope) and I would 
> like to have something that can be reviewed at the weekend.
>
> I will sum up what I've done. I've not introduced new features. I've 
> gathered all pieces of global state into one (the default_engine). I've 
> introduced TemplateEngine class and I've rewritten most tests so they 
> create their own TemplateEngine instances instead of using the global one. 
> I tried not to introduce backward incompatibilities.
>
> I've rewritten history so in order not to break existing links I won't use 
> ticket_17093 branch any more. Instead, see ticket_17093v2 branch [1].
>
> [1] 
> https://github.com/chrismedrela/django/commit/ed578d64fff8c8eb58898548d5ef1c0815c25f24
> [2] https://github.com/chrismedrela/django/tree/ticket_17093v2
>
> > I agree that it doesn't mean less global state. But there is good global
>
> > state and bad global state. For example, we could make DjangoApplication
>
> > class to avoid global state but there is no need. Global settings is OK
>
> > since we don't need the ability to create more than one django
>
> > application instance. Even tests doesn't require more than one instance
>
> > at the same time, so we can just reinitialize the application when there
>
> > is need for that. That's done by change_settings decorator and it works
>
> > well. And there is bad global state, like current template system.
>
> ...
>
> > I didn't know about Jinja2 and that it's more suitable for use outside
>
> > Django. So it seems like the only goal is refactoring -- we need to
>
> > quarantize global state, but not dependencies on settings. We also don't
>
> > need to split the template system into django-dependent and
>
> > django-independent parts.
>
>
>> I'm not sure the distinction between "good global state" and "bad global
>
> state" is so clear, or even exists. A major part of the motivation for
>
> #17093 was that it could be a step along the path towards a
>
> global-state-free Django (using something like a DjangoApplication or
>
> ApplicationConfig object), which would allow running multiple Django
>
> sites in a single process (which is indeed something that some p

Re: 'Change' button for related fields.

[charettes]

There's already tickets for this feature request 
#13163and #
13165 .

I've been planning to propose a patch based on 
django-admin-enhancerfor 
1.6.

Simon

Le mercredi 20 février 2013 21:07:25 UTC-5, Mason Staugler a écrit :
>
> I was going to monkey patch this into our system until a coworker 
> suggested contributing. This patch will add a 'change' link (next to the 
> 'add' button) for related fields.
>
> Is it a trivial patch? I have a feeling it's a Design Decision Needed and 
> the docs suggested post it to this group.
>
> Here's the patch:
>
> https://github.com/mqsoh/django/commit/fedfe2dd41d8b74ec80240efab5b01b076fa05d4
>
> ...and this is what it looks like. The pencil will link to the change form 
> for the selected object.
>
>
> 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

contrib.auth.forms need to rewrite UserCreationForm for custom users

[luan]

Hello guys,

When I had to customize my users in Django 1.5 where UserProfile got 
deprecated,
I found out I'd have to rewrite the UserCreationForm (contrib.auth), as 
described in docs[1].

Is it really necessary to rewrite it all? What if the function 
get_user_model from contrib.auth was used instead of calling the auth User 
model directly? Wouldn't it work?
That's how I imagine it [2]. UserChangeForm also has a "model = User". I 
believe it fits in the same case above.

[1] https://docs.djangoproject.com/en/1.5/topics/auth/customizing/
[2] 
https://github.com/LuanP/django/commit/a6ed3aa292f103ba193407dc96919adb17361f32

Cheers,
Luan.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: 'Change' button for related fields.

[Mason Staugler]

I spent a long time looking for something like that. I must not have been 
using the correct terms and I'm kind of annoyed by that.

I'll take a look at the tickets and send my patch for consideration.

Thank you, Simon.


On Friday, February 22, 2013 9:17:34 AM UTC-5, charettes wrote:
>
> There's already tickets for this feature request 
> #13163and #
> 13165 .
>
> I've been planning to propose a patch based on 
> django-admin-enhancerfor 
> 1.6.
>
> Simon
>
> Le mercredi 20 février 2013 21:07:25 UTC-5, Mason Staugler a écrit :
>>
>> I was going to monkey patch this into our system until a coworker 
>> suggested contributing. This patch will add a 'change' link (next to the 
>> 'add' button) for related fields.
>>
>> Is it a trivial patch? I have a feeling it's a Design Decision Needed and 
>> the docs suggested post it to this group.
>>
>> Here's the patch:
>>
>> https://github.com/mqsoh/django/commit/fedfe2dd41d8b74ec80240efab5b01b076fa05d4
>>
>> ...and this is what it looks like. The pencil will link to the change 
>> form for the selected object.
>>
>>
>> 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Is there a buildbot? Is there a waterfall? Do the tests pass against all backends?

[Shai Berger]

On Sunday 17 February 2013, Florian Apolloner wrote:
> Hi,
> 
> you can see the tests at http://ci.djangoproject.com/ -- currently Oracle
> is untested cause it's a major pita to setup on ubuntu.
> 

The easiest way I know to set up Oracle on Debian/Ubuntu is to use Oracle's 
"Dev Days" VM appliance. It places non-trivial hardware requirements (about 
1.5G memory just to be able to run the VM under virtualbox -- it is set up to 
ask for 1G "internally" and VB has its own overhead), but other than that, it 
was quite easy to get going. I use it on my laptop (which is easily strong 
enough) for development.

http://www.oracle.com/technetwork/community/developer-vm/index.html

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Is the solution in #12235 correct?

[Gordon]

 

Hello,

I don't think the issue is fixed.

I am trying to use a UUID primary key.  Per 
https://code.djangoproject.com/ticket/12235, I've marked my field as the 
autofield in contribute_to_class.  However, now when I try to save a new 
instance, django is trying to insert an integer into my uuid because it 
thinks it is an autofield.  

This causes a ValueError at /admin/…/…/add/ because django auto assigns an 
integer for the field value.

If forcing my uuid field to be an autofield is actually the correct 
approach, how do I override what django inserts as the automatic value?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Is there a buildbot? Is there a waterfall? Do the tests pass against all backends?

[Anssi Kääriäinen]

I did some Oracle fixing today. 1.5.x should now pass all tests. I
didn't run the full test suite after committing the fixes, as it takes
nearly an hour to do so. It would be really good if some of the Oracle
users could run the full test suite (not including GIS which is known
to be broken) on the stable/1.5.x to confirm that all tests are
passing. Oracle is somewhat configuration dependant, so checking
against different configurations would be good.

On 18 helmi, 22:30, Skylar Saveland  wrote:
> If anyone is interested, I get 11 failures and 5 errors on master against
> Oracle with ./runtests right now:
>
> ERROR: test_lookup_date_as_str (modeltests.lookup.tests.LookupTests)

Confirmed.

> ERROR: test_gfk_to_model_with_empty_pk
> (regressiontests.generic_relations_regress.tests.GenericRelationTests)

Fixed.

> ERROR: test_get_key_columns
> (regressiontests.introspection.tests.IntrospectionTests)

Confirmed.

> ERROR: setUpClass (regressiontests.views.tests.i18n.JavascriptI18nTests)

Doesn't seem Oracle specific, likely a random Selenium failure. The
Selenium tests do that sometimes...

> ERROR: test_query_encoding
> (regressiontests.backends.tests.LastExecutedQueryTest)

Fixed. This might be configuration dependant, so retesting at least
this one would be valuable.

> FAIL: test_generic_relations_values_list
> (modeltests.generic_relations.tests.GenericRelationsTests)

For some reason I can't find this test at all. It doesn't seem to be
present on current master (f565c6f9994b2559a5164dc709a926eac7ebd5fe),
and generic_relations tests do pass.

> FAIL: test_extra_method_select_argument_with_dashes_and_values
> (modeltests.basic.tests.ModelTest)

Confirmed.

> FAIL: test_year_lookup_edge_case (modeltests.basic.tests.ModelTest)

Confirmed.

> FAIL: test_foreign_key (modeltests.one_to_one.tests.OneToOneTests)

Works for me.

> FAIL: test_field_types (regressiontests.inspectdb.tests.InspectDBTestCase)

Oracle is an expected failure for this test now. Inspecting Oracle
fields is bit hard, there was various failures from getting CharField
length wrong to wrongly detecting FloatField as DecimalField. Many of
these are likely impossible to fix. And, inspectdb isn't high
priority, the output needs to be hand-edited anyways.

> FAIL: test_custom_project_template_from_tarball_by_url
> (regressiontests.admin_scripts.tests.StartProject)

Works for me, and doesn't look Oracle specific.

> FAIL: test_project_template_tarball_url
> (regressiontests.admin_scripts.tests.StartProject)

Same here.

> FAIL: test_sql_all (regressiontests.commands_sql.tests.SQLCommandsTestCase)

Confirmed.

> FAIL: test_sql_create
> (regressiontests.commands_sql.tests.SQLCommandsTestCase)

Fixed.

> FAIL: test_sql_delete
> (regressiontests.commands_sql.tests.SQLCommandsTestCase)

Fixed.

> FAIL: test_related_null_to_field
> (regressiontests.many_to_one_regress.tests.ManyToOneRegressionTests)

Fixed.

This leaves still 5 confirmed and some "works for me" items. The rest
of the confirmed items are likely a bit harder to fix.

The current situation is somewhat problematic regarding Oracle. We
don't have many core developers who are interested in Oracle and also
have time to fix issues. I am fixing Oracle issues mainly because I
think it is good for the ORM to have a core DB that does things
somewhat differently than the other core DBs. Forces the ORM to be
more generic.

And yes, continuous testing would be really good. Unfortunately Oracle
didn't work well in the CI environment so it was uninstalled. There is
some hope in using the VM image in the CI for Oracle testing, and also
the djangocore-box might give developers somewhat easy way to test
against Oracle. So, there is some hope that the situation will
improve.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Is the solution in #12235 correct?

[Gordon]

After a chat on IRC I submitted ticket 
https://code.djangoproject.com/ticket/19888

On Friday, February 22, 2013 5:07:25 PM UTC-5, Gordon wrote:
>
> Hello,
>
> I don't think the issue is fixed.
>
> I am trying to use a UUID primary key.  Per 
> https://code.djangoproject.com/ticket/12235, I've marked my field as the 
> autofield in contribute_to_class.  However, now when I try to save a new 
> instance, django is trying to insert an integer into my uuid because it 
> thinks it is an autofield.  
>
> This causes a ValueError at /admin/…/…/add/ because django auto assigns an 
> integer for the field value.
>
> If forcing my uuid field to be an autofield is actually the correct 
> approach, how do I override what django inserts as the automatic value?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Multi Client Django System

[Delcio Torres]

Dear Sirs,

Maybe this is a concept question, but here we go.

I'm doing test development to learn django and using admin for everything.

This is a Company/Employee/HeathInsurance CRUD system.

The main ideia is that I want to provide this for different companies and 
still not allow them to see each others registers.

A Django Admin user , should belong to Company or be associated with it 
somehow and only see the registers created by other members of the same 
company.

My question is Django Admin Groups from Auth Model would do the task or not.

I've thought about overriding save_mode and get the group to which the user 
belongs , then save it into some group field at HeathInsurance and Employee.

Then on admin.py thought about overriding the queryset and use a filter so 
you would only see registers from your company.

Is this feasible ? Is this the way?

Thanks everyone!

Best Regards,

Delcio

Here some model example
## models.py ##
class Company(models.Model):
name = models.CharField(max_length=200, null=False)


class HeathInsurance(models.Model):
nome = models.CharField(max_length=200, null=False)


class Employee(models.Model):
name = models.CharField(max_length=200, null=False)
company = models.ForeignKey(Company)
health_insurance = models.ForeignKey(HeathInsurance)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Multi Client Django System

[Jacob Haslehurst]

This list is for the discussion of django itself, this question would be 
perfect for django-users. 

- Jacob


On Saturday, 23 February 2013 at 11:50 AM, Delcio Torres wrote:

> Dear Sirs,
> 
> Maybe this is a concept question, but here we go.
> 
> I'm doing test development to learn django and using admin for everything.
> 
> This is a Company/Employee/HeathInsurance CRUD system.
> 
> The main ideia is that I want to provide this for different companies and 
> still not allow them to see each others registers.
> 
> A Django Admin user , should belong to Company or be associated with it 
> somehow and only see the registers created by other members of the same 
> company.
> 
> My question is Django Admin Groups from Auth Model would do the task or not.
> 
> I've thought about overriding save_mode and get the group to which the user 
> belongs , then save it into some group field at HeathInsurance and Employee.
> 
> Then on admin.py thought about overriding the queryset and use a filter so 
> you would only see registers from your company.
> 
> Is this feasible ? Is this the way?
> 
> Thanks everyone!
> 
> Best Regards,
> 
> Delcio
> 
> Here some model example
> ## models.py ##
> class Company(models.Model):
> name = models.CharField(max_length=200, null=False)
> 
> 
> class HeathInsurance(models.Model):
> nome = models.CharField(max_length=200, null=False)
> 
> 
> class Employee(models.Model):
> name = models.CharField(max_length=200, null=False)
> company = models.ForeignKey(Company)
> health_insurance = models.ForeignKey(HeathInsurance)
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> (mailto:django-developers+unsubscr...@googlegroups.com).
> To post to this group, send email to django-developers@googlegroups.com 
> (mailto:django-developers@googlegroups.com).
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>  
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.