ModelForm._post_clean could simply call Model.full_clean

2012-01-27 Thread Chris Wilson
Hi all,

I proposed this in a comment on #16423 , and then found that it had come up before:

https://code.djangoproject.com/ticket/13100
https://code.djangoproject.com/ticket/13121

and was rejected thus:

 I think model-validation is just not what you want it to
be. It's a step on the way there, but we can't start completely
validating models before we save them without an extended deprecation
plan for the current non-validating behavior. All of the authors and
people familiar with the implementation are busy until after 1.2 is
out, but we should have a conversation on django-dev a week or so
after that happens.

 I'm going to side with Joseph on this one -- he and Honza
spent a long time developing model validation, and I have a lot of
confidence in his design decisions... if you disagree with a decision
we have made, start a thread on django-developers.

I read this, not as a design decision to never do this, but not to do
it before 1.2 was out. It's now well and truly out :)

The only difference, as far as I can see, between
ModelForm._post_clean and Model.full_clean is that validating
uniqueness is optional in the former. It only happens if:

# It is False by default so overriding self.clean() and
failing to call
# super will stop validate_unique from being called.

Is this the right thing to do? The comment doesn't justify the
behaviour, it just describes it, and the logic isn't apparent to me.

ModelForm._post_clean would be much simpler if it just called
Model.full_clean. Yes the documentation would have to be changed again
.

I hope you will consider this change.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Feature request: read-only admin view

2012-01-27 Thread Chris Wilson
Hi all,

I really like how the admin interface does a lot of the work for me in
developing a site with basic CRUD functions, and a few free bonuses
like pagination and list filtering.

I agree with all the comments on #820  requesting this feature (as well as the "view" permission
which goes with it).

This ticket and several others were closed because a design decision
has been made that the admin interface is for admins only, and such a
feature is out of scope. A lot of people believe that it should be in
scope. Therefore I would like to politely request a reconsideration of
this decision.

There are a few very simple things that Django could do to make this
much easier:

* a simple way (e.g. a switch) to make AdminChangeForm or
fieldset.html produce all fields as read-only, without including them
in CustomAdminForm.readonly_fields; (1-2 lines)

* a way to override the default template selection on
ModelAdmin.render_change_form (1 line)

* an easy way to add URLs to AdminSite.get_urls (5 lines if
intergrated? currently requires about 20 lines of external code for a
generic solution)

* an easy way to override the buttons rendered by the submit_row
template tag, such as wrapping it in a {% block submit_block %} in
change_form.html (currently requires copying the whole of
change_form.html to make this change)

In total the (generic) solution requires a few hundred lines of code,
mainly because those features are not present, and would be about 20
lines of code if they were.

Thanks for your consideration,

Cheers, Chris.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Adding fields to admin forms

2012-02-09 Thread Chris Wilson

Hi all,

I want to be able to add extra fields, that I control, to the admin forms. 
For example, a read-only list of documents authored by the user, which is 
ManyToMany, so an InlineAdmin doesn't work.


The admin module docs don't really say whether this is possible or not. 
They have much more to say about excluding, reordering and making fields 
read-only. However there are hints that it's possible (for ModelForm):


* "Also, if you manually add the excluded fields back to the form, they 
will not be initialized from the model instance."


* "you can extend and reuse ModelForms by inheriting them. This is useful 
if you need to declare extra fields"


And the admin interface uses ModelForm and allows you to specify your own 
form. So I create a custom subclass of ModelForm with the new field:


from django_tables2 import tables
class DocumentsAuthoredTable(tables.Table):
...

class DocumentsAuthoredWidget(widgets.Widget):
def render(self, name, value, attrs=None):
table = DocumentsAuthoredTable(value)
return table.as_html()

class DocumentsAuthoredField(forms.Field):
widget = DocumentsAuthoredWidget


class IntranetUserReadOnlyForm(ModelForm):
class Meta:
model = models.IntranetUser

documents_authored = DocumentsAuthoredField()

Now I have to do some ugly things. The widget has no access to the 
instance (model), so I have to provide some data for it, by adding a 
method to either the IntranetUserReadOnlyForm or the model. I don't want 
to introduce a circular dependency in the model, so I have to do this in 
the form:


def __init__(self, data=None, files=None, auto_id='id_%s', 
prefix=None,

initial=None, error_class=ErrorList, label_suffix=':',
empty_permitted=False, instance=None):

"""
Add a "method" to the initial data which is used to lookup the
value of documents_authored by BoundField.
"""

if initial is None:
initial = {}
from django.utils.functional import curry
initial['documents_authored'] = curry(self.get_documents_authored, 
instance)


super(IntranetUserReadOnlyForm, self).__init__(data, files,
auto_id, prefix, initial, error_class, label_suffix,
empty_permitted, instance)

def get_documents_authored(self, instance):
# print "get_documents_authored(%s)" % instance
return instance.document_set.all()

And now I have to do a really horrible hack in the Admin class, to add 
this field into the form:


class IntranetUserAdmin(AdminWithReadOnly):
documents_authored = None

This is because AdminReadOnlyField tries to call label_for_field() on the 
field, which insists that it be a member of the Model or the Admin, not 
the Form. It doesn't really care what type of attribute it is, but it 
throws an exception if it's not there at all:


  File "/usr/lib/pymodules/python2.6/django/template/defaulttags.py", line 
227, in render

nodelist.append(node.render(context))
  File "/usr/lib/pymodules/python2.6/django/template/defaulttags.py", line 
190, in render

values = list(values)
  File "/usr/lib/pymodules/python2.6/django/contrib/admin/helpers.py", 
line 103, in __iter__

model_admin=self.model_admin)
  File "/usr/lib/pymodules/python2.6/django/contrib/admin/helpers.py", 
line 135, in __init__

label = label_for_field(field, form._meta.model, model_admin)
  File "/usr/lib/pymodules/python2.6/django/contrib/admin/util.py", line 
252, in label_for_field

raise AttributeError(message)
TemplateSyntaxError: Caught AttributeError while rendering: Unable to 
lookup 'documents_authored' on IntranetUser or IntranetUserAdmin


So please can we have an easier way to add fields to an admin form? 
Ideally I wouldn't have to change the Model or the Admin at all, just add 
a new attribute to the custom Form.


Also please can the widget have access to the request? Rendering out a 
DjangoTable (with proper links) needs this.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 760887
The Humanitarian Centre, Fenner's, Gresham Road, Cambridge CB1 2ES

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Why recommend to include project name in settings?

2012-02-10 Thread Chris Wilson

Hi all,

I'd like to inquire about the design decision to recommend users to 
include the project name in DJANGO_SETTINGS_MODULE and ROOT_URLCONF, for 
example:


When using django-admin.py, you can either set the environment variable 
once, or explicitly pass in the settings module each time you run the 
utility...


export DJANGO_SETTINGS_MODULE=mysite.settings




A string representing the full Python import path to your root URLconf. 
For example: "mydjangoapps.urls"




But "mysite" is a package, and Django wants us to be able to import 
"mysite.settings" and "mysite.urls" (in fact, the latter is the default 
ROOT_URLCONF setting). In order to make that possible, setup_environ 
does some magic... This bit of clever causes a boat-load of problems...




My question is, why exactly Django wants us to be able to import 
"mysite.settings" and "mysite.urls"? Wouldn't it be much simpler and 
better if we could just import settings and urls?


As an experiment, I've removed the "setup_environ(settings_mod)" from my 
manage.py, which adds the project's parent directory to the PYTHONPATH, 
and it seems to work so far. I think that mod_wsgi uses an environment 
without this addition as well, leading to tickets like this:





Is there any reason not to drop the project name part of 
DJANGO_SETTINGS_MODULE and ROOT_URLCONF, to reduce the risk of confusion, 
pollution, double imports and bad practice app coding in Django?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 760887
The Humanitarian Centre, Fenner's, Gresham Road, Cambridge CB1 2ES

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Why recommend to include project name in settings?

2012-02-10 Thread Chris Wilson

Hi Carl,

On Fri, 10 Feb 2012, Carl Meyer wrote:

Is there any reason not to drop the project name part of 
DJANGO_SETTINGS_MODULE and ROOT_URLCONF, to reduce the risk of 
confusion, pollution, double imports and bad practice app coding in 
Django?


If you follow that quoted thread to the end, you'll find that this 
problem has already been resolved in trunk :-) See 
https://docs.djangoproject.com/en/dev/releases/1.4/#updated-default-project-layout-and-manage-py 
for the release-notes summary.


Thanks, I had seen the patch on Trac and noticed that it was being 
resolved in the direction of the "status quo", by enforcing the presence 
of a top-level directory/module/package. This is the opposite of what I 
was suggesting: entirely removing the top-level thing. I was curious about 
the reasons for this decision, and whether the (apparently simpler) 
alternative had been considered and rejected, and if so why?


For the default project layout, I chose to resolve it in favor of having 
an outer "project package", as this namespaces the project better and 
avoids potential top-level namespace clashes with generic names like 
"settings" and "urls". However, it's easy enough to adjust the layout 
and imports for your own project to eliminate that project package and 
just use "import settings" and "import urls", if you prefer that.


Can you tell me more about why "potential top-level namespace clashes with 
generic names like "settings" and "urls"" would be a problem?


We can always import .settings and .urls if we we want the top-level 
modules, can't we?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 760887
The Humanitarian Centre, Fenner's, Gresham Road, Cambridge CB1 2ES

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Allow changing form field properties after form creation

2012-04-05 Thread Chris Wilson

Hi all,

I've added this as a ticket but I wanted to make sure that the core 
and forms developers have a chance to see and interact with it, so I'm 
posting it here too. You can find the ticket at: 



Currently, if I want to tweak the properties of some fields in a 
ModelForm, I have to replace them in the subclass like this:


{{{
class DocumentForm(ModelForm):
title = 
models.Document._meta.get_field('title').formfield(required=False)
programs = 
models.Document._meta.get_field('programs').formfield(required=False)
authors = 
models.Document._meta.get_field('authors').formfield(required=False)
confidential = 
models.Document._meta.get_field('confidential').formfield(widget=AdminYesNoWidget)
uploader = 
models.Document._meta.get_field('uploader').formfield(required=False)
file = 
models.Document._meta.get_field('file').formfield(widget=AdminFileWidgetWithSize)

}}}

This is very bulky to just change some properties. The problem is that the 
field copies some of its properties into weird places like the widget 
during initialisation:


{{{
class Field(object):
...
def __init__(...)
...
if help_text is None:
self.help_text = u''
else:
self.help_text = smart_unicode(help_text)
widget = widget or self.widget
if isinstance(widget, type):
widget = widget()

# Trigger the localization machinery if needed.
self.localize = localize
if self.localize:
widget.is_localized = True

# Let the widget know whether it should display as required.
widget.is_required = self.required

# Hook into self.widget_attrs() for any Field-specific HTML 
attributes.

extra_attrs = self.widget_attrs(widget)
if extra_attrs:
widget.attrs.update(extra_attrs)

self.widget = widget
}}}

If we refactored this so that all the property initialisation was done in 
setters, then we could just write:


{{{
class DocumentForm(ModelForm):
def __init__(...)
self['title'].required = False
self['programs'].help_text = 'Hold down Ctrl to select multiple 
options'

self['authors'].required = False
self['confidential'].widget = AdminYesNoWidget
self['uploader'].required = False
self['file'].widget = AdminFileWidgetWithSize
}}}

Which is more concise, much clearer, and does not override things 
unnecessarily.


I can write the code, but first I want to:

* see if any core developers object, to avoid wasting effort
* know how to run the Django tests and which suite(s) I should be running
* know where to put the tests.

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Allow changing form field properties after form creation

2012-04-05 Thread Chris Wilson

Hi Nate,

On Thu, 5 Apr 2012, Nate Bragg wrote:


They don't "have" to be replaced in a subclass in the way you showed.
Perhaps it isn't perfectly DRY, but whats so bad about naming the field
explicitly?


The fact that we're building reusable components and we want people to be 
able to subclass our forms and:


* change field and widget properties
* do so depending on run-time configuration (e.g. the database or
  settings)
* even if the inherited field type changes
* or the inherited options change
* and get an error if they reference a field that doesn't exist any more

Besides which, it's unnecessary, repetitive and unclear what one is doing.

 Anyhow, when it comes specifically to widgets, the Meta class already 
has a 'widgets' attribute already that lets you specify that.


Thanks for reminding me about that. It's a bit difficult to inherit 
through multiple levels of subclasses, and doesn't generate errors if one 
refers to a field or property that doesn't exist, but it meets most of my 
requirements.



I would sooner have "smart" Meta class attributes to perform this
behavior declaratively than to have to do it at the __init__ level:

class Meta:
    title__required = False
    programs__help_text = 'Hold down Ctrl to select multiple options'
    authors__required = False
    confidential__widget = AdminYesNoWidget
    uploader__required = False
    file__widget = AdminFileWidgetWithSize


Yes, that would work too, although it also doesn't generate errors if one
refers to a field or property that doesn't exist,


... and I don't like *that* particularly either.


Do you really dislike the idea of making form field properties settable 
after creation using @property and behave sensibly? It seems neat, simple, 
small, powerful and pythonic to me.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Allow changing form field properties after form creation

2012-04-05 Thread Chris Wilson

Hi Brian,

On Thu, 5 Apr 2012, Brian Neal wrote:


On Thursday, April 5, 2012 6:49:20 AM UTC-5, Chris Wilson wrote:

  class DocumentForm(ModelForm):
       title =
  models.Document._meta.get_field('title').formfield(required=False)

You can already replace and tweak the fields in a way similar to what you are 
proposing. For example, you can create a Form or ModelForm, and in the 
__init__() you can do:

def __init__(self, *args, **kwargs):
    super(MyForm, self).__init__(*args, **kwargs)
    self.fields['title'] = models.Charfield(max_length=30, required=False)


Thanks, yes, I can replace the entire field but I can't tweak its 
properties. I.e. I can do exactly what I was doing before, in __init__, 
like this:


class DocumentForm(ModelForm):
def __init__(self, *args, **kwargs):
super(MyForm, self).__init__(*args, **kwargs)
		self.fields['title'] = 
models.Document._meta.get_field('title').formfield(required=False)


But that's even longer. I'm trying to propose a simple way to tweak 
certain properties of a field after it's created, without replacing the 
field, in a way that meets my requirements as described in the previous 
email and that's pythonic and simple and likely to be accepted by Django.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Allow changing form field properties after form creation

2012-06-08 Thread Chris Wilson

Hi Simon,

On Fri, 6 Apr 2012, Simon Meers wrote:

On 6 April 2012 07:26, Beres Botond  wrote:


Isn't it this what you are trying to do?

class DocumentForm(ModelForm):
       def __init__(self, *args, **kwargs):
               super(MyForm, self).__init__(*args, **kwargs)
               self.fields['title'].required = False

Which works perfectly fine. You can tweak pretty much any property at
runtime like that, without replacing the field entirely.


This discussion is sounding more like something that belongs in
django-users. I agree that there are some improvements that can be
made to make field-tweaking easier and more elegant, as proposed in
[1] which Keryn pointed out in [2]. It would probably be better to
continue this conversation in those tickets.

Simon

[1] https://code.djangoproject.com/ticket/17924
[2] https://code.djangoproject.com/ticket/18064


Now unfortunately both tickets have been closed by a core developer, so I 
apparently have no choice but to try to persuade people here if I want to 
reverse that.


There is an important use case on my ticket [2] which is not covered by 
[1], which is controlling properties depending on run-time information. 
For example:


* we have drop-down list controls whose choices depend on database 
queries, and which must not be cached in the form class, because we can't 
restart the server just in order to show changes in those lists.


* similarly, whether a field is required or not might depend on the value 
of another field, or in another related model.


We CAN use __init__ to either replace the fields, or poke values into 
fields and their widgets, but it's not clean: it's excessive code and the 
excess code is entirely Django internals that are subject to change. For 
example, we currently have to know that we need to change the widget's 
values as well:


  self['title'].required = False
  self['title'].widget.is_required = False

whereas if that was handled by a property setter, we'd only have to do 
this:


  self['title'].required = False

and the amount of code in Field is the same, but much of the code from 
__init__ is separated out into individual setters. I hope that you will 
agree that this is cleaner, more reusable design, which allows fields to 
respond intelligently to changes to their properties after creation.


(If you don't agree, is it because you're in the "everything should be 
declarative" camp? I have issues with that position: mainly that sometimes 
some things cannot be known until run time; so we should support doing 
everything at run time, even if we have some magic sugar for doing it 
declaratively as well, when that's possible).


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Test runner with search

2012-06-14 Thread Chris Wilson
Hi all,

I've made some improvements (in my view) to the DjangoTestSuiteRunner. I 
got tired of having to remember my test class names and of typing so much:

  ./manage.py test binder.BinderTest.test_can_create_users

This new version searches for tests with the given name in all 
INSTALLED_APPS, so you can just type this:

  ./manage.py test test_can_create_users

You can find the source code here:

  https://github.com/aptivate/intranet-binder/blob/master/testing.py

Is this something that might be interesting to add to Django core?

Cheers, Chris.
-- 
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cambridge City FC, Milton Road, Cambridge CB4 1UY

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Test runner with search

2012-06-14 Thread Chris Wilson
Hi Andrew,

On Thu, 14 Jun 2012, Andrew Godwin wrote:

> +1 from me as well - the test runner you linked to looks reasonably 
> sane, only a few small things that I'm questioning (like why types is 
> imported inside the for loop).

I prefer to keep the imports as close to the code that uses them as 
possible, to make it easier to move code around and reduce "action at a 
distance". That's just my personal preference and of course if it's merged 
into Django then it can use the Django style.

Cheers, Chris.
-- 
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cambridge City FC, Milton Road, Cambridge CB4 1UY

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Test runner with search

2012-06-18 Thread Chris Wilson
Hi all,

On Mon, 18 Jun 2012, Anssi Kääriäinen wrote:
> On 16 kesä, 09:29, Russell Keith-Magee 
> >
> > Well, sure -- easier is better -- but is this actually going to be
> > easier? Are you saying that you've never written two test classes with
> > the same test name?

I don't think I have, but I don't mind either. The cost of accidentally 
running an additional unnecessary test (which should pass) is very low, 
about 1 second, compared to remembering the class name of the test. Also 
it's a good incentive to give your tests unique names :) 

> > Just looking at code I've got right now -- I've
> > got a dozen tests named "test_bad_date" scattered throughout my test
> > suite. How do I disambiguate between them?

You can still run tests the old way if you want to. The only difference is 
that if you do something that was previously invalid, i.e. specify a 
simple label that's not a package, it will search for tests instead of 
aborting.

> > We already have a problem with namespace collapsing -- if you have two 
> > test classes with the same name in an app, you can't differentiate 
> > between them. I'd rather not compound the problem by removing class 
> > names from the discovery process as well.

I'd rather not be forced to add yet another component to the test label 
when running it from the commnd line. Running tests should be as quick and 
easy as possible.

> I guess this should work by giving a message like this on test name 
> clash: "ERROR: Multiple tests found: ". The 
> user could then copy-paste the correct test and run the test by full 
> identifier.

I might deliberately want to run multiple tests in some cases. I was 
thinking of adding wildcard search support too. It shouldn't be an error, 
it should be explicitly supported.

> I have two specific needs for this feature:
>   1. When a test errors, I want to have as-easy-as-possible way to
> rerun that test. This could be achieved by just adding a line
> containing the full test identifier
> ("queries.Queries1Tests.test_something") in the error output. We
> currently don't have that, instead you need to collect pieces from the
> test output.

That's true, and annoying, but also requires using the mouse to rerun 
tests. That's where the Bash completion extension would be useful, I 
guess.

> I guess Jannis is right - this and the "record errors / replay" feature 
> belong to third party test runner. When we know the API is correct, then 
> adding these to core would make sense to me. But, as we can easily test 
> these outside core, lets start there.

Fine by me. How about a contrib test runner? And does anyone have time to 
write some tests for a test runner?

Cheers, Chris.
-- 
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cambridge City FC, Milton Road, Cambridge CB4 1UY

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: I think adding a "first" method to the QuerySet be useful.

2012-07-03 Thread Chris Wilson

Hi all,

On Tue, 3 Jul 2012, Łukasz Rekucki wrote:

On 3 July 2012 06:27, Maxime Haineault  wrote:

One of the common pitfall I come across way to often with novices is
something like this:

def getFirstUser():
return User.objects.all()[0]

It looks innocuous and often wont raise any exceptions in dev because 
you develop and test with data, so they slip easily in production.


def getFirstUser():
return User.objects.all().first()

If the QuerySet is empty it should just return None instead of raising an
exception.

Other methods like .last() could probably also be useful, but getting 
the first element of a QuerySet is the pitfall I see the most often.


Good point. I was doing this the other day, and couldn't come up with a 
nice way.



E) Use __bool__ which fetches only one batch (depends on DB):

   qs = User.objects.all()
   user = None if not qs else qs[0]


Batch could be quite large, so this could still be inefficient.


F) Use the iterator protocol

   user = next(iter(User.objects.all()), None) # you can add LIMIT or
not, see Alex's comment


This is probably the most Pythonic. But none of these make the intent 
really clear. I love code that's readable and obvious, and first() does it 
for me. +1.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Proposal: use SQLAlchemy Core for query generation

2012-07-03 Thread Chris Wilson

Hi Reinout,

On Tue, 3 Jul 2012, Reinout van Rees wrote:


On 30-06-12 16:22, Luke Plant wrote:

Also, in Django to date we've eschewed external dependencies. That has
been partly due to the poor and confusing state of Python packaging,
which is hopefully improving.

[snip]

Our current recommended installation method is via pip [2], and we would
be doing our users a disservice to recommend otherwise.


Slight warning: everyone recommends pip, but watch out with windows. Pip 
doesn't support binary eggs (which easy_install does), so all those python 
database drivers will need to be compiled.


Good point, thanks.

And virtualenv/pip defaults to a nicely isolated virtualenv, so the 
clicky-clicky-install python database drivers (and other binary eggs) 
won't get used.


So: pip sounds great right until you need binaries on windows.


The virtualenv inherits system-wide packages. So you can install whatever 
binary packages you need system-wide, either using easy_install or an MSI, 
and pip won't try (and fail) to install them again unless they don't meet 
the version requirements of the pip manifest.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



save() method could return the object

2012-10-12 Thread Chris Wilson

Hi all,

If the save() method returned the object itself, then we could chain it 
like this:


old_status = Status(last_contact=None).save()

Instead of having to do this:

old_status = Status(last_contact=None)
old_status.save()

It's a trivial one-line change to the Model class and I don't think it 
would break backwards compatibility, as I doubt that anyone is relying on 
the save() method returning None. Should I submit a patch?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: A.objects.getdefault

2012-10-12 Thread Chris Wilson

Hi all,

On Thu, 11 Oct 2012, Daniel Moisset wrote:


obj, = SomeModel.objects.filter(foo='bar') or [None]


Daniel's solution is elegant, but far from clear or clean.

I'm strongly in favour of a simple, obvious way to do the common thing, 
which is to return None if the object doesn't exist, instead of throwing 
an exception. My preferred method names would be .nget(), .get_or_none() 
or .first().


May I invoke the Zen of Python to add more than +1 to votes for this 
change? (not that my vote counts anyway, of course :)


+1  Beautiful is better than ugly.
+1  Explicit is better than implicit.
+1  Simple is better than complex.
+1  Flat is better than nested.
+1  Readability counts.
-1  Special cases aren't special enough to break the rules.
+1  Although practicality beats purity.
+1  There should be one-- and preferably only one --obvious way to do
it.
+1  Now is better than never. :)
+1  If the implementation is hard to explain, it's a bad idea. (the
alternative one-liner proposed by Daniel is hard to explain)
+1  If the implementation is easy to explain, it may be a good idea.

+9  Total

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: save() method could return the object

2012-10-12 Thread Chris Wilson

On Fri, 12 Oct 2012, Marijonas Petrauskas wrote:


There already exists create method that does exactly what you need:obj = 
SomeModel.objects.create(name='foo', age=42)


OK, thanks, that appears to be completely undocumented.

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Switch to database-level autocommit

2013-03-05 Thread Chris Wilson

Hi Lennart,

On Tue, 5 Mar 2013, Lennart Regebro wrote:

I do agree that 99.9% of the sites committing on the end of the request 
is the right thing to do, and do think that this should be the default, 
although I realize that debate is already lost.


In a perfect academic world where there are never any errors, that 
approach is probably academically correct. In my opinion it makes correct 
error handling extremely difficult, because database errors (unique 
violations, concurrent modifications, etc) are thrown far away from the 
code that knows how to deal with them (the code that caused them in the 
first place).


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-18 Thread Chris Wilson

Hi Julian,

On Mon, 18 Mar 2013, julianb wrote:


imagine the following use case:

You build an online store where you have sorted products into several 
categories and maybe associated an occasion. Now you want to build URLs. So the 
URL schema that all of the
store's owners agree on is:

//
//
//

Look simple.

Because product slugs should generally be different to category slugs and 
occasions, you expect no clashes here. And you don't want to bloat the URLs by 
putting product/ and
category/ in there. And you also do not want to resort making it look cryptic by 
having just /p// and so on.
Then it comes to Django. How do you do that?


I put the type in the URL to make it obvious. That doesn't harm SEO and 
makes your URL routing clearer. For example:


/product//
/category//
/occasion//

But that's getting into django-users territory.

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: reconsider re-opening ticket 901

2013-05-13 Thread Chris Wilson

Hi all,

On Sun, 12 May 2013, Shai Berger wrote:

those should probably be updated (on remote objects!) too, or else 
some serious inconsistencies may be created (when the _id field 
changes, that is).


aa= A.objects.get(...)
bb= aa.b
(some other operation, changing aa.b_id in the database)
aa.refresh()

Now -- unless we do something about it -- bb.a is still aa, but aa.b is not bb
and not even equal to it; thus, "bb.a.b == bb" is false, which currently
cannot happen without serious meddling if I'm not mistaken.


Hibernate (as an implementation of the Java Persistence API, JPA) tries to 
do things like this by implementing a global cache of objects loaded from 
the database, so that there's only one copy of A and B in memory, which 
all references point to.


This makes it possible for A, when its link to B changes to point to a 
different object B', to find all the instances of B in memory (because 
there's only one, and it's in the object cache) and update them to no 
longer point back to A.


However it does this at enormous cost in flexibility, sanity and error 
recovery. I strongly recommend against going down this route:

http://blog.old.aptivate.org/2010/11/26/hibernate-ejb-and-the-unique-constraint/

It's always possible to have an out-of-date model object in memory with 
Django, with or without refresh() (which just allows us to update it 
manually). I suggest we let that sleeping dog lie.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Perception of attitude in tickets

2013-05-13 Thread Chris Wilson

Hi all,

On Mon, 13 May 2013, Russell Keith-Magee wrote:

This isn't political equivocating. Its a genuine call to the community 
to tell us how we can make things better.


If I may make a suggestion to be considered by the community:

The status WONTFIX sounds awfully rude to me. It's like saying "That's a 
pony and you can't have one, ever." It implies a terminal finality which 
actually isn't meant in some cases, because it is possible (as we've seen) 
and even sometimes recommended by a core developer, for a sufficiently 
determined person to push for change on the mailing list and make it 
happen.


Perhaps there's a case for a status like "DISCUSSION" or "NEEDINFO" when a 
feature might be accepted if sufficient evidence for it comes to light, 
but that evidence isn't there yet.


I'd like to feel that reload() and first() are cases where a core 
developer might choose to close a ticket with this new status instead of 
WONTFIX. I would also hope that the core developers might be gentler with 
newbies like me instead of scaring us off with a gruff one-word reply of 
WONTFIX.


I think there is value in people "voting" for a feature on the ticket if 
they don't feel up to arguing the case on the mailing list (which is a 
trial by fire, and not for the faint hearted). Whoever is brave enough to 
take up the issue on the mailing list can point to the number of votes on 
the ticket as evidence for a desire for the feature, and hence its 
usefulness. And voting on the ticket instead of here saves a lot of "me 
too" noise on the mailing list.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Perception of attitude in tickets

2013-05-13 Thread Chris Wilson

Hi Luke,

On Mon, 13 May 2013, Luke Sneeringer wrote:

On May 13, 2013, at 10:06 AM, AK  wrote:

WONTFIX does sound rude to me, as well. Perhaps 'onholdindefinite' can 
be used instead, the effective meaning is the same, just the term 
itself is more polite. It seems that nobody looking at it would think 
"I'll just wait for a while and surely it'll get fixed.". Instead, 
anyone needing it would think "If it is to be fixed, it's clear someone 
has to make a case for it." -ak


Note: "won't fix" is used on many bug systems, not just Django. So, in 
choosing to go for a euphemism, you're also (unintentionally) 
obfuscating, because you're subtly communicating that it's somehow 
different than the standard "won't fix".


I think it *IS* different to the standard "won't fix". The standard one 
implies "won't fix EVER" to me, whereas I think the way Django uses this 
status is "won't fix until someone persuades on the mailing list." Which 
could be expressed better than that, I'm sure. I did suggest a couple of 
other options: NEEDINFO and DISCUSSION. I could suggest some more: 
ITSAPONY? JUSTIFY?


I think we're lucky to have a community where it's possible to persuade 
the BDFLs to change their minds, and unlucky to have an unfriendly-looking 
bug tracker which dissuades people from trying.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Perception of attitude in tickets

2013-05-13 Thread Chris Wilson

On Mon, 13 May 2013, Kirby, Chaim [BSD] - PED wrote:

WONTFIX has a long history in software development. It also does 
(correctly) state intentionality that 'onholdindefinite' lacks. The 
intention of WONTFIX is "yes, this is possibly valid, but in the state 
this ticket is written it is being closed". Using anything else that 
doesn't close the ticket just lets it hang out as cruft.   


I'm OK with it being a closed state, if that helps. I'd just prefer a 
slightly friendlier name for it.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Travis support (again)

2013-05-18 Thread Chris Wilson

Hi all,

On Thu, 7 Mar 2013, Jacob Kaplan-Moss wrote:


On Mon, Feb 25, 2013 at 4:48 PM, Florian Apolloner
 wrote:

So all in all I think we could enable travis,


I agree, +1 from me.


One issue standing in the way: Travis will try to build any branch without 
a .travis.yml file as though it was a Ruby project. This will fail. It 
may generate email notifications too, as they are enabled by default.





I think we need to add a .travis.yml file to *every* branch that we might 
ever commit to (including packaging and stable/1.5.x) to tell Travis not 
to build it.


Does anyone have a particular objection to adding this one file to old 
branches, with minimal contents, e.g:


# see https://github.com/travis-ci/travis-ci/issues/902
branches:
  only:
- master
- travisci

I guess this would only cause problems if anyone's building an old branch, 
or their own fork of django, in Travis. In which case they'll have to 
resolve the merge conflict, and again whenever we change the .travis.yml 
file.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: test discovery

2013-05-18 Thread Chris Wilson

Hi all,

On Fri, 10 May 2013, Carl Meyer wrote:

I merged this patch tonight. Thanks to everyone who contributed! Now 
let's see how the CI servers feel about it...


I think Travis is unhappy about something in this commit. Any ideas?

==
ERROR: test_file_path 
(test_runner.test_discover_runner.DiscoverRunnerTest)

--
Traceback (most recent call last):
  File 
"/home/travis/build/aptivate/django/tests/test_runner/test_discover_runner.py", 
line 65, in test_file_path

["test_discovery_sample/"],
  File "/home/travis/build/aptivate/django/django/test/runner.py", line 
63, in build_suite

tests = self.test_loader.loadTestsFromName(label)
  File "/usr/lib/python2.7/unittest/loader.py", line 91, in 
loadTestsFromName

module = __import__('.'.join(parts_copy))
ImportError: Import by filename is not supported.



Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: test discovery

2013-05-18 Thread Chris Wilson

On Sat, 18 May 2013, Chris Wilson wrote:


I think Travis is unhappy about something in this commit. Any ideas?

==
ERROR: test_file_path (test_runner.test_discover_runner.DiscoverRunnerTest)
--
Traceback (most recent call last):
 File 
"/home/travis/build/aptivate/django/tests/test_runner/test_discover_runner.py", 
line 65, in test_file_path

   ["test_discovery_sample/"],
 File "/home/travis/build/aptivate/django/django/test/runner.py", line 63, 
in build_suite

   tests = self.test_loader.loadTestsFromName(label)
 File "/usr/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
   module = __import__('.'.join(parts_copy))
ImportError: Import by filename is not supported.

<https://next.travis-ci.org/aptivate/django/jobs/7275360>


It seems to be checking if a file exists, using a relative path, in 
django/test/runner.py:65:


if not os.path.exists(label_as_path):
tests = self.test_loader.loadTestsFromName(label)

And that will behave differently depending if you run your tests from 
inside the tests directory like this:


./runtests.py -v2 --settings=test_postgres_nogis test_runner

Or from the parent directory, as Travis currently does, like this:

python -Wall tests/runtests.py --selenium --verbosity 2 \
--settings=django_settings

I can work around it in Travis by changing working directory before 
running the tests, but is it worth fixing this in the test runner, perhaps 
using an absolute path based on __file__?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: test discovery

2013-05-18 Thread Chris Wilson

Hi all,

Another odd behaviour of the new test runner. This runs the tests, but 
fails to add the test app to INSTALLED_APPS, so they all fail because 
their tables are not created:


PYTHONPATH=.. python -Wall runtests.py --selenium --verbosity 2 \
--settings=tests.travis_configs.test_postgres_nogis \
tests.transactions.tests.AtomicInsideTransactionTests

Changing the spec to remove the initial "tests." results in the app being 
loaded and the tests run:


PYTHONPATH=.. python -Wall runtests.py --selenium --verbosity 2 \
--settings=tests.travis_configs.test_postgres_nogis \
transactions.tests.AtomicInsideTransactionTests

Is this expected behaviour? It's rather counter-intuitive to me.

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: test discovery

2013-05-18 Thread Chris Wilson

Hi Carl,

On Sat, 18 May 2013, Carl Meyer wrote:

I don't think this should be fixed in the test runner itself; in 
general, file-path test labels _should_ be interpreted as relative to 
wherever you are running the tests from.


But it should be fixed in the 
test_runner.test_discover_runner.DiscoverRunnerTest.test_file_path test 
- that test apparently needs to isolate itself better by setting the CWD 
for the duration of the test, or something similar. Mind filing a bug? I 
should be able to take a look soon.


Done, thanks: https://code.djangoproject.com/ticket/20449

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Ticket #7220: Last_login in django.contrib.auth should have null=True

2013-06-01 Thread Chris Wilson

Hi all,

Regarding ticket #7220 , this 
is bugging me at the moment. I share a database with a non-django app, and 
need to insert test data, and having to set last_login to something is 
annoying.


I understand @SmileyChris' point that there are no migrations (yet) in 
core, so this is a bit difficult to fix. However I have a proposal:


I tried to fix this myself by overriding last_login in our custom user 
class, and got this error:


FieldError: Local field 'last_login' in class 'IschoolUser' clashes
with field of similar name from base class 'AbstractUser'

I propose that we allow overriding fields, with a safety mechanism: 
it's required to add an "override=True" attribute to the field. Users who 
want to change an existing field must add this attribute to the new 
field in their subclass, and manage any migrations or database 
changes themselves. That removes the responsibility from Django to manage 
it, which currently it can't.


What do you say? If this sounds like a good idea I can work up a patch. 
That would enable a sensible workaround for ticket #7220 (override the 
field and add south to your project) which isn't currently possible.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Composite fields -- (ir)regular status report #3

2013-07-22 Thread Chris Wilson

On Mon, 22 Jul 2013, Michal Petrucha wrote:


I have some awesome news today. At long last I managed to finally get
the refactor of ForeignKey to pass the entire test suite.


That is awesome news! Well done Michal! I am really looking forward to 
having this feature available in Django.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Custom Chainable QuerySets (#20625)

2013-07-22 Thread Chris Wilson

Hi Aymeric,

On Mon, 22 Jul 2013, Aymeric Augustin wrote:

The idea of `QuerySet .as_manager()` sounded sane to me at 
first—especially in the light of the failure of various other 
attempts—but it creates a circular dependency between `Manager` and 
`QuerySet`. Creating the `Manager` class now requires the `QuerySet` 
class, and `QuerySet` needs a reference to `Manager` for `as_manager()`. 
Is it possible to untangle this by writing a 
`Manager.from_queryset(queryset_class)` method instead?


Possibly. Is the dependency at import time? If so, you could import 
manager inside the QuerySet.as_manager() method.



QuerySet shouldn't even know what a Manager is!


I think that's very true. How about this?


  class MyQuerySet(models.QuerySet):
      def published(self):
      return self.filter(published_date__lte=now())

  class MyModel(models.Model):
      published_date = models.DateTimeField()

      objects = CustomManager(MyQuerySet)


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.




Re: Deprecation a little harsh?

2013-08-12 Thread Chris Wilson

Hi all,

On Mon, 12 Aug 2013, Jacob Kaplan-Moss wrote:

If you'd like to help push us closer to where *you* think the right 
place balance is, the best thing to do is to watch our development 
process and speak up in the moment. These sorts of general, post-facto 
observations don't give us a ton that we can really act on, and 
ultimately are going to come down to "feeling" arguments. I'd like to 
avoid those because they're kinda counter productive. If you want to do 
something about it, help us on future decisions; second-guessing the 
past doesn't really get us anywhere.


I would love to see support extended for a bit longer after deprecation. 
Maybe not in every case, but where it doesn't hurt too much. It certainly 
seems that people are eager to rip things out the very second that the 
deprecation waiting period expires and they are "allowed" to.


We have apps in production running Django 1.3. There won't be any security 
fixes. If there's a critical vulnerability, we may have to do a lot of 
unpaid work to either backport the fix, or upgrade the site to Django 1.6, 
which will undoubtedly break a lot of stuff.


I'm not asking anyone to do my job for me (I hope) but it would be really 
nice to have something like 3 years of support for core infrastructure 
like Django, that's really painful to upgrade, and even more painful to 
replace. It would certainly help me to sleep better at night.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Django ORM support for NoSql databases

2013-12-17 Thread Chris Wilson

Hi all,

On Tue, 17 Dec 2013, Tom Evans wrote:

On Tue, Dec 17, 2013 at 3:35 PM, parisrocks 
 wrote:

Waiting for official Django ORM support for NoSql databases.


MongoDB (and the vast majority of NoSQL databases) are not relational 
databases, they are document oriented databases, they store documents of 
an unspecified type and allow you to retrieve documents by id, or 
perhaps other attributes of the document.


At no point with MongoDB can you have relationships between two 
different documents that can be followed or queried. Therefore, I'm 
quite confused what you want from an "ORM" interface to a NoSQL 
database. Each document has no relationships, there is nothing for an 
"ORM" to do...


I think it would be more helpful to say that it IS possible to store the 
ID of a document in a field of another, of course, and that means that 
documents CAN have relationships:


{"id": "1", "name": "Pluto", "owner": "2"}
{"id": "2", "name": "Mickey"}

And some NoSQL databases do not provide any facilities for following that 
link automatically, as part of a single query, so you would need to make a 
separate query to retrieve the owner of an object. Other databases (such 
as CouchDB) do support it.


Even SQL does not always make it easy or convenient to follow these links, 
especially if one wants to retrieve associated records from multiple 
tables without creating a combinatorial explosion. NoSQL databases can 
actually do better than SQL here.


There is value in providing storage backends for NoSQL databases in 
Django. Several already exist, for example:


* https://code.djangoproject.com/wiki/NoSqlSupport
* http://www.allbuttonspressed.com/projects/django-nonrel

The main issues and solutions are listed on the first of those pages.

There is no particular reason NOT to do it, unless one wants Django's ORM 
layer to remain fundamentally tied to SQL. I can't see a good reason to do 
that, but maybe others do. In the mean time, this question will continue 
to be asked until a position is officially documented, even if it's "we 
don't care about NoSQL, just leave us alone."


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1312171639330.6923%40lap-x201.fen.aptivate.org.
For more options, visit https://groups.google.com/groups/opt_out.


Django doesn't log much at high verbosity levels

2014-02-03 Thread Chris Wilson

Hi all,

I'd expect that Django would log something like:

* At INFO level, at least one line per request (similar to runserver 
output)

* At DEBUG level, enough information to recreate the request

However, it doesn't appear to:

* There are only two logger.debug statements, both logging database SQL 
commands

* There are no logger.info statements at all
* Turning on DEBUG logging for everything seems to log only SQL statements 
and errors

* The django.request logger only seems to be used for errors and warnings

I'd like to add more info and debug logging to django.request as stated 
above. Is that likely to be accepted?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1402031219080.19498%40lap-x201.fen.aptivate.org.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Django doesn't log much at high verbosity levels

2014-02-04 Thread Chris Wilson

Hi Russell,

On Tue, 4 Feb 2014, Russell Keith-Magee wrote:


A definite "Maybe". :-)

For my money, this is probably something that is handled better by the web 
server. Apache, for example, logs one line per
request, and can be configured to dump full GET and POST contents if you need 
that information. 

Aside from reproducing functionality available elsewhere, my concern would be 
performance. Adding a single logging call isn't
much, but you're adding it into a critical path, so the impact could be large 
(especially if you're logging the full contents of
POST and FILES data). 

So - I'm probably wavering between a +0 and -0, depending on the actual impact 
your proposed change has.

At the very least, it's worth opening a ticket -- even if the idea is 
ultimately rejected, a ticket allows us to archive the
decision making process (since I doubt you're the first, or will be the last 
person to think of this idea).


Thanks, I've created ticket https://code.djangoproject.com/ticket/21949, 
with some more information on my reasons for proposing this.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1402041018340.19498%40lap-x201.fen.aptivate.org.
For more options, visit https://groups.google.com/groups/opt_out.


#19182 backport request

2014-02-14 Thread Chris Wilson

Hi all,

I just ran into bug #19182 :

ModelAdmin filtering didn't work when list_filter includes a ('fieldname', 
ClassName) tuple, but throws a SuspiciousOperation exception instead. The 
ChangeList would display fine, but as soon as you tried to use it, it 
would throw a SuspiciousOperation exception, because it hadn't correctly 
added 'fieldname' to the list of valid filters.


This bug is already fixed in trunk, but I don't think it's been backported 
to 1.6.x. At least, I don't see the fix in the lookup_allowed() method at

.

Since the documentation for 1.6 says that this should work

but it actually doesn't in 1.6, please would you consider backporting this 
fix to the 1.6 stable branch?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1402141247410.19498%40lap-x201.fen.aptivate.org.
For more options, visit https://groups.google.com/groups/opt_out.


Re: #19182 backport request

2014-02-14 Thread Chris Wilson

Hi all,

On Fri, 14 Feb 2014, Florian Apolloner wrote:

as Tim noted on the ticket, we are not going to backport this as per our 
policy.


In that case will you accept patches to remove description of the
broken feature from the documentation for 1.4, 1.5 and 1.6?

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1402141540230.19498%40lap-x201.fen.aptivate.org.
For more options, visit https://groups.google.com/groups/opt_out.


Re: #19182 backport request

2014-02-14 Thread Chris Wilson

Hi Tim,

On Fri, 14 Feb 2014, Tim Graham wrote:

I sympathize with your frustration. On the other hand as we have over 
500 bugs in Trac, and I'm not sure starting to accept documentation 
requests for them is the best use of our resources. I'm not sure the 
feature is really completely broken enough to warrant removing it from 
the docs?


I'm afraid that the documented feature is totally broken. It cannot work, 
unless you override ModelAdmin.lookup_allowed() in your own admin and add 
the fields manually, filter parameters to the returned set.


I can submit a documentation patch if necessary, so someone would just 
have to approve it.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1402141746570.19498%40lap-x201.fen.aptivate.org.
For more options, visit https://groups.google.com/groups/opt_out.


Re: [GSOC] Improving the Test-Suite

2014-02-15 Thread Chris Wilson

Hi all,

On Sat, 15 Feb 2014, Russell Keith-Magee wrote:

One of the improvements I see is classification of test cases. 
Classifying them into categories (read multiple-categories), would make 
it easier for users/developers/maintainers to run them. Basis of 
classification,etc is what I am still thinking on. But surely 
classification will help in deciding which all test cases to run. For 
example - just running third-party app test cases, or just run my test 
cases, or those which check part ABC of my project, or just those with 
priority set to important.

[...]
I would envisage that this would be a declarative process - in code, 
marking a specific test as a "system" test or an "integration" test (or 
whatever other categories we develop).


It just occurred to me that most classification systems are completely 
arbitrary and therefore not very useful. What's a "system" test and how 
would I know whether I need to run it?


But some ideas that I can think of that might be useful are:

* Automatically building test coverage maps for each test, and reversing 
them, so we can see which tests touch the line(s) of code that we just 
modified, and rerun them easily. A good smoke test to run while modifying 
part of Django.


* Categorising by imports: run all tests that import django.db or 
django.core.http for example. Not perfect, some tests may touch facilities 
without needing to actually import them, but it would be quick and cheap.


* Profile and speed up the test suite, so that we can run all tests more 
quickly, especially with databases like postgres where it takes an hour to 
run them all.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1402151512310.19498%40lap-x201.fen.aptivate.org.
For more options, visit https://groups.google.com/groups/opt_out.


Re: should we include images in Django documentation?

2014-02-22 Thread Chris Wilson

Hi all,

On Sat, 22 Feb 2014, Aymeric Augustin wrote:

There's already a few images and I can't see any reason against adding 
more where appropriate.


There's just one small hurdle. Last time I checked, the intersection of 
"editors producing good-looking diagrams" and "open-source diagram 
editors" was empty. And we don't want ASCII diagrams.


I'm not sure what kind of diagrams we'd be looking for, but I've created 
some diagrams with OpenOffice Draw that I'm quite happy with:


https://raw.github.com/aptivate/inaspmaterials/master/src/Network_Management/Unit_5_The_Bandwidth_Challenge/images/congestion-queueing.png
https://raw.github.com/aptivate/inaspmaterials/master/src/Network_Management/Unit_6_Solving_Network_Problems/images/total-bandwidth-illustration.png
https://raw.github.com/aptivate/inaspmaterials/master/src/Network_Management/Unit_11_Technical_Measures/images/hfsc-queueing.png

All of these have .ODG sources in the same directory.


I would recommend to use a tool that produces good looking results and
commit:

- the source file (in your tool's default format)
- a SVG export (for the HTML version of the docs)
- a PDF export (for the PDF version of the docs)


OpenOffice Draw does export to SVG and PDF. We might also want a .PNG 
fallback for browsers that don't support SVG (well).


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1402221852540.5481%40lap-x201.lap.aptivate.org.
For more options, visit https://groups.google.com/groups/opt_out.


Saving forms without validation, and formsets

2014-03-18 Thread Chris Wilson

Hi all,

(Apologies if this belongs on django-users instead. To clarify, I'm 
interested in what appear to be potentially missing or useful features in 
Django, and whether it's worth implementing them and submitting pull 
requests, or creating separate reusable modules to implement them. I'm 
also not asking for anyone to fix these issues for me.)


We're building an application which has some large forms including 
repeating formsets and file upload fields. It's intended for use in 
developing countries with slow and unreliable Internet connections. Thus 
it's important for users to be able to save the form regularly, and to not 
lose uploaded files even if there's a form validation error.


We encountered a number of issues which feel like they might either be:

* bugs in Django, or
* deliberate limitations of Django core, that could be useful as
  a third-party module.

And I'd like your feedback about where you feel these issues lie. I'm 
hoping that we're not the only ones to encounter them!


1. GCBV and Vanilla Views do a great job for simple forms, but they leave 
out embedded formsets entirely. (For example our large form has repeating 
sections for employment history, education, etc.) It feels like it would 
be great to have formsets handled "more automatically" - instantiating, 
validating and saving them just like we do for forms. A lot of our code is 
shared between all our formsets, and potentially reusable.


2. Adding instances to a formset on the client side using Javascript. 
There is a django-formset-js package on PyPI, but it requires special 
attributes to be added to form elements that would have been difficult 
with crispy forms (which we're also using) and it feels almost like this 
functionality ought to be in Django core (maybe not today, but one day?)


3. We couldn't change the "extra" of a formset without redefining the 
formset class. We wanted to do this because the required repeating 
sections (employment history etc.) must not have a blank form in them if 
they already have some valid forms, otherwise the blank forms prevent the 
form being submitted because HTML5 client-side validation fails. So we had 
to do all this:


def get_context_data(self, **kwargs):
...
for name, formset_class in self.formsets.iteritems():
# doesn't exist yet, so we can't call its queryset() method
queryset = formset_class.model._default_manager.filter(
**{formset_class.fk.name: application})
extra = 0 if queryset.exists() else 1

# need to reconstruct the formset class to change extra?
formset_class = inlineformset_factory(
Application,
formset_class.model,
formset_class.form,
formset_class,
extra=extra,
can_delete=formset_class.can_delete,
fields=formset_class.form.base_fields.keys(),
max_num=formset_class.max_num,
)

4. We needed to be able to save the form without validation. To do this, 
we had to make all fields non-required on our model, and make them 
required on the form instead. Since nearly all fields are required, we'd 
have to redefine them all, making the ModelForm a bit pointless, but 
instead we were naughty and looped over the fields setting their required 
values to True, and also updating the widget (which gets a copy of this 
attribute on constructions). Naughty but DRY. I feel that a shortcut to 
list required fields of a ModelForm in its Meta class would be useful 
(overriding the blank and null status of the underlying Model fields).


def _save_data_without_validation(self, form):
# construct a new form with the required value turned off on all
# fields, and use that to save the instance.
all_optional_form = self.get_form(
data=self.request.POST,
files=self.request.FILES,
instance=form.instance
)
for field in all_optional_form:
field.field.required = False
all_optional_form.full_clean()

5. We have to be able to save a new instance of the model even if form 
validation fails. So both the form_valid and form_invalid methods of the 
CreateView return redirects to the UpdateView for the newly saved 
instance. But that makes the form load through GET instead of POST, and 
thus it isn't validated. This is fine for saving the form, but not for 
submitting it, so we have to validate it like this:


def get_form(self, data=None, files=None, **kwargs):
# If we've been redirected here because of a form error, then we 
need

# to validate the form again. To do that, we need to generate a
# simulated "POST data dict" from the model data, and clean the 
form,

# which we wouldn't normally do on a GET request.
# http://stackoverflow.com/a/8996585/648162

request_origin = self.request.GET.get('type')
 

Re: Saving forms without validation, and formsets

2014-03-19 Thread Chris Wilson

Hi Carl,

On Tue, 18 Mar 2014, Carl Meyer wrote:

On 03/18/2014 02:54 PM, Chris Wilson wrote:


1. GCBV and Vanilla Views do a great job for simple forms, but they
leave out embedded formsets entirely. (For example our large form has
repeating sections for employment history, education, etc.) It feels
like it would be great to have formsets handled "more automatically" -
instantiating, validating and saving them just like we do for forms. A
lot of our code is shared between all our formsets, and potentially
reusable.


The approach I've used for this is encapsulating the creation and
handling of the inline formsets entirely within the "parent" form class,
so from the perspective of the view it acts like a normal simple form.


That seems like an excellent idea, thanks! I'm reworking my forms to work 
this way.



This requires overrides of a few methods on the parent form: __init__()
to create the formsets, is_valid() to ensure the formsets are valid too,
has_changed() to see if formsets have changed, and save() to save
formsets too.


There's a few more Form methods that I'm not sure whether I need to 
override, especially if I want to be able to generalise this. I think just 
_get_media and is_multipart should take sub-formsets into account?



If we were going to look at incorporating something into Django, I'd
like to consider this option as an alternative to adding more GCBVs with
inline-formset support. I think it's a nicer abstraction (because in a
real sense those inlines are "part of" the parent form), and I don't
think we want to add to the already-extensive list of GCBVs and mixin
classes.


I agree. Then it should also work for Vanilla Views with no changes.


I think either approach would be workable as a third-party project, too.


I think it would be nice for forms to be able to treat embedded formsets 
as a special (complex) type of 'field', so that as_table, as_p etc. would 
render the formsets as well, and for ModelForm to be able to generate 
these automatically for models with ForeignKeys that link back to the 
ModelForm's Model.



2. Adding instances to a formset on the client side using Javascript.
There is a django-formset-js package on PyPI, but it requires special
attributes to be added to form elements that would have been difficult
with crispy forms (which we're also using) and it feels almost like this
functionality ought to be in Django core (maybe not today, but one day?)


I've used this: http://plugins.jquery.com/django-superformset/


Thanks for the tip, I didn't know about that one.


3. We couldn't change the "extra" of a formset without redefining the
formset class. We wanted to do this because the required repeating
sections (employment history etc.) must not have a blank form in them if
they already have some valid forms, otherwise the blank forms prevent
the form being submitted because HTML5 client-side validation fails. So
we had to do all this:


I don't understand this. The 'extra' attribute of a formset class can be
tweaked dynamically as a normal attribute (although this may not be
documented); it doesn't require recreating the entire formset class.


I'm not sure that's possible, because inline_formset defines a new class. 
That class has a class attribute called 'extra' containing the value of 
'extra', but changing it would change it for all instances of the class, 
so it's not thread-safe. And when the class is instantiated, the value is 
immediately used to create the initial empty forms, and not used after 
that, so changing it after instantiating the form has no effect. Thus, it 
seems that we have to subclass BaseFormSet anyway to get into the 
constructor and change the value of extra before the initial forms are 
created.


I snipped your last three items, regarding saving invalid forms. I think 
this is an unusual use case, and I'm not sure support for it belongs in 
core. It would be interesting to experiment with something to make 
filefields less painful when validation fails, but that can be a 
third-party project first I think.


It may be unusual, but I'm not the only one to want this. At least two 
people participated on this SO question:

http://stackoverflow.com/questions/8993749/transform-an-unbound-form-to-a-bound-one

From there I got my current approach, which is to use model_to_dict to 
create a fake 'data' to make the unbound form bound, so that I can 
validate it. However this is ugly, and it doesn't work with formsets 
because it doesn't account for the management form, which complains about 
having been tampered with because its data is missing. And there's no 
form_to_dict equivalent of model_to_dict, so I'd have to hack something 
up, all to pretend to Django that I've got an unbound form, just so I can 
validate its con

Re: Saving forms without validation, and formsets

2014-03-19 Thread Chris Wilson

Hi all,

As part of our test process we have code that automatically fills in forms 
with dummy values to make them valid. It fills in all required fields.


This works well except for formsets. BaseModelFormSet.add_fields() adds a 
PK field to the form after it's created. It marks this field as not 
required:


form.fields[self._pk_field.name] = ModelChoiceField(qs, initial=pk_value, 
required=False, widget=widget)


However it will blow up if editing an existing instance and this field 
isn't present, so in a sense it is required.


I know it's a very minor issue and I can easily work around it (however 
ugly that is), but would it logically make sense for this field to be 
required if pk is not None?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.OSX.2.00.1403191703550.13253%40chris-macbook.lan.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Shifting to Py.Test and Improving the Test Suite

2014-04-06 Thread Chris Wilson

Hi Andrew,

I'm not a Django core contributor but just a user and occasional patcher. 
I submit some comments on this proposal in the hope that they will be 
useful.


On Sun, 6 Apr 2014, Andrew Pashkin wrote:


Some Pytest advocacy:
1) Pytest has convenient tests collection options - you can just specify folder 
to run all tests in it. It is also possible to
filter tests by regex, and select specific ones.


I do use pytest and it has a lot of good points. Running all tests in a 
directory or a specific file is one of them, as is the better test 
selection logic.



class MyTestCase(TestCase):
    def test_something(self):
   expected_content = ...
   response = self.client(...)
       self.assertEqual(expected_content, response.content)  

Pytest:

def test_something(client):
expected_content = ...
response = self.client(...)
assert expected_content == response.content


But this is not one of them, at least not for me. I strongly dislike this 
style of assertions:



Pytest style is cleaner and complies PEP8. Assertions with assert are more 
readable.


* The difference between "self.assertEquals(a, b)" and "assert a == b" is 
minimal for me.


* It hides a LOT of magic in how pytest goes back and evaluates the 
assertion's arguments again if the assertion fails. Sometimes it gets 
different results (if one of the tested methods has side effects) and then 
the assertion message makes no sense, hides the problem, or pytest breaks.


* It makes it much harder to write custom assertions and get meaningful 
display on error.


It is also possible to group tests, by placing them into class without 
subclassing something.


Grouping tests with decorators allows tests to be members of multiple 
suites, it's true. However this is not something that I've ever needed.


And losing the power of inheritance to add methods to my test classes, by 
not putting tests into classes, is enough to kill this style completely 
for me. I write a LOT of my own assertions:


  https://github.com/aptivate/django-harness/tree/master/django_harness


4) In Pytest it is possible to define setup functions for modules, classes and 
specific tests.


I can do this by adding mixins to test classes. Works fine for me, and 
less magical.


5) Pytest also has fixture mechanism, which allows to use additional 
resources in tests by specifying fixture names as an arguments (like 
client) - it is good alternative to using setUp's and subclassing 
something.


We have named fixtures in Django test classes already, so I don't see that 
as much of an advantage. We can't easily add fixtures in mixins at the 
moment. I've not needed that so far.



6) pytest-xdist plugin provides parallel execution of tests.
7) Pytest has features for writing parametrized tests.

So Pytest is kind of cool and it is easy to migrate to it.


I don't think it's necessary or wise to rewrite the test suite to use 
pytest-style assertions. But better integration of Django and Pytest, and 
the ability to write tests with Pytest-style assertions if you like, would 
certainly have my support (fwiw).


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1404061751310.7500%40lap-x201.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Shifting to Py.Test and Improving the Test Suite

2014-04-06 Thread Chris Wilson

Hi Andrew,

On Sun, 6 Apr 2014, Andrew Pashkin wrote:

* It makes it much harder to write custom assertions and get meaningful 
display on error.


Can you give an examples for cases with messages/breakings and for custom 
assertions?


I don't have an example of breakage to hand, I ripped out the offending 
code in disgust and replaced it with standard assertions that worked. If I 
find another example I'll be sure to let you know before I do the same 
again.


Regarding custom assertions, how would you write assert_not_redirected 
from 
https://github.com/aptivate/django-harness/blob/master/django_harness/fast_dispatch.py

as a PyTest assertion? Consider the following tests:

def not_redirected(response):
return (response.status_code != 302)


class RedirectViewTests(FastDispatchMixin, TestCase):
def test_redirect_view_with_custom_assertion(self):
response = self.fast_dispatch('my-redirecting-view')
self.assert_not_redirected(response)

def test_redirect_view_with_pytest_assertion(self):
response = self.fast_dispatch('my-redirecting-view')
assert not_redirected(response)

assert_not_redirected is from FastDispatchMixin in 
https://github.com/aptivate/django-harness/blob/master/django_harness/fast_dispatch.py, 
not_redirected is a simple assertion I wrote as a demo, since I don't use 
pytest much myself for this reason. Now compare the output (reformatted 
for readability):


 
RedirectViewTests.test_redirect_view_with_custom_assertion 


main/tests/tests.py:71: in test_redirect_view_with_custom_assertion

  self.assert_not_redirected(response)
.ve/src/django-harness/django_harness/fast_dispatch.py:114: in 
assert_not_redirected

  msg_prefix)
E   AssertionError: 302 != 200 : unexpectedly redirected to 
http://www.google.com/


 
RedirectViewTests.test_redirect_view_with_pytest_assertion 


main/tests/tests.py:75: in test_redirect_view_with_pytest_assertion

  assert not_redirected(response)
E   AssertionError: assert 
not_redirected(0x493da10>)


I.e. my assertion outputs:

AssertionError: 302 != 200 : unexpectedly redirected to 
http://www.google.com/


and the simple Python assertion outputs this:


AssertionError: assert
not_redirected()


How would you generate a useful error message, bearing in mind that you 
can't really access response._headers['location'][1] without first having 
checked that the response is a redirect, which isn't the normal case?


Regarding magic in Pytest - yeah - there is some =)) But if think 
deeper, inheritance and self is also magic, and Pytest just uses its own 
magic (dependency injection) that is differs from OO magic =)


OO "magic" is well known and understood by most Python developers. 
Pytest's magic is (a) largely undocumented (afaik) and (b) relies on 
understanding of Python bytecode which is definitely not common knowledge.


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1404061926030.7500%40lap-x201.
For more options, visit https://groups.google.com/d/optout.


Re: [GSOC] Shifting to Py.Test and Improving the Test Suite

2014-04-07 Thread Chris Wilson

Hi Andrew,

On Sun, 6 Apr 2014, Chris Wilson wrote:

On Sun, 6 Apr 2014, Andrew Pashkin wrote:

* It makes it much harder to write custom assertions and get meaningful 
display on error.


Can you give an examples for cases with messages/breakings and for custom 
assertions?


I don't have an example of breakage to hand, I ripped out the offending code 
in disgust and replaced it with standard assertions that worked. If I find 
another example I'll be sure to let you know before I do the same again.


OK, here is one.

chris@lap-x201:~/projects/2014/webassets$ .ve/bin/tox -e py27 -- 
tests.test_filters:TestPyScss.test_search_path

GLOB sdist-make: /home/chris/projects/2014/webassets/setup.py
py27 inst-nodeps: 
/home/chris/projects/2014/webassets/.tox/dist/webassets-0.10.dev.zip

py27 runtests: PYTHONHASHSEED='894115171'
py27 runtests: commands[0] | nosetests 
tests.test_filters:TestPyScss.test_search_path

F
==
FAIL: tests.test_filters.TestPyScss.test_search_path
--
Traceback (most recent call last):
  File 
"/home/chris/projects/2014/webassets/.tox/py27/local/lib/python2.7/site-packages/nose/case.py", 
line 197, in runTest

self.test(*self.arg)
  File "/home/chris/projects/2014/webassets/tests/test_filters.py", line 
969, in test_search_path
assert self.get('out.css') == 'h1 {\n  color: #ff;\n}\na {\n 
color: #ff8000;\n}\n'

AssertionError

--
Ran 1 test in 0.055s

FAILED (failures=1)

"AssertionError". Gee, thanks. That helps a lot. So if the string wasn't 
what we expected, what WAS it? How is this anywhere NEAR as useful as 
self.assertEqual()?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1404071049580.7500%40lap-x201.
For more options, visit https://groups.google.com/d/optout.


Re: Building a library of SQL functions into Django

2014-06-18 Thread Chris Wilson

Hi Carl,

On Wed, 18 Jun 2014, Carl Meyer wrote:

On 06/18/2014 02:59 PM, Aymeric Augustin wrote:

2014-06-18 19:18 GMT+02:00 Carl Meyer mailto:c...@oddbird.net>>:

If you need a Function in your project (whether provided by Django or 
by a third-party library), and the Function doesn't natively support 
the database backend you're using, you can simply subclass it, add the 
appropriate as_vendor method, and use your subclass.


I was thinking about third-party database backends. How are they 
expected to ship customised implementations of Functions included in 
Django?


Yes, I wasn't thinking about this case in my first mail, and I agree
that this is a serious problem for the pure as_vendor() approach; it's
not acceptable IMO to expect third-party database backends to
monkeypatch their as_vendor() methods onto the built-in Function classes.

I think database backends should have a hook to override the SQL
implementation of any given Function. I don't think this needs to imply
pushing the default implementation of all Functions down into the
database backend (that just makes life unnecessarily difficult for
third-party Functions).


Why does it even need a hook or monkey-patching? Why not pass the 
expression tree to the SQL backend (as the most natural format for 
programmatically parsing), for it to render out in backend-specific SQL as 
it sees fit? Perhaps relying in inherited base methods in most cases, and 
overriding a few methods where that backend needs to render a particular 
feature differently?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1406182253240.4005%40lap-x201.
For more options, visit https://groups.google.com/d/optout.


Re: Building a library of SQL functions into Django

2014-06-18 Thread Chris Wilson

Hi Carl,

On Wed, 18 Jun 2014, Carl Meyer wrote:

I think database backends should have a hook to override the SQL 
implementation of any given Function. I don't think this needs to 
imply pushing the default implementation of all Functions down into 
the database backend (that just makes life unnecessarily difficult for 
third-party Functions).


Why does it even need a hook or monkey-patching? Why not pass the 
expression tree to the SQL backend (as the most natural format for 
programmatically parsing), for it to render out in backend-specific SQL 
as it sees fit? Perhaps relying in inherited base methods in most 
cases, and overriding a few methods where that backend needs to render 
a particular feature differently?


Yes - that's going all the way to the other end of the spectrum, making 
life easier for third-party database backends at the expense of 
third-party Functions. How, in your plan, is a third-party Function 
(which won't be recognized by any database backend's SQL emitter) 
supposed to provide "just works" support for all built-in database 
backends, presuming it needs different SQL for different backends?


Good point, thanks :)

If neither the backend knows about the function, nor the function about 
the backend, then a third-party module would have to register support for 
both (e.g. "I know how to convert a Case() Function into HSQLDB SQL"). 
Perhaps we could have a registry where third-party modules could register 
such support.


Lacking that, perhaps it's just a bug? E.g. "Our Case() Function works on 
Postgres, MySQL and MSSQL, but not on Oracle because we haven't got 
around to implementing the patch in the Oracle backend to support it, 
patches welcome." A backend could easily detect a Function that it doesn't 
understand in the expression tree, give it a chance to render itself, and 
if that fails then it could raise a warning or an error?


Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Citylife House, Sturton Street, Cambridge, CB1 2QF, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/alpine.DEB.2.02.1406182336590.4005%40lap-x201.
For more options, visit https://groups.google.com/d/optout.