Saving forms without validation, and formsets
Hi all,
(Apologies if this belongs on django-users instead. To clarify, I'm
interested in what appear to be potentially missing or useful features in
Django, and whether it's worth implementing them and submitting pull
requests, or creating separate reusable modules to implement them. I'm
also not asking for anyone to fix these issues for me.)
We're building an application which has some large forms including
repeating formsets and file upload fields. It's intended for use in
developing countries with slow and unreliable Internet connections. Thus
it's important for users to be able to save the form regularly, and to not
lose uploaded files even if there's a form validation error.
We encountered a number of issues which feel like they might either be:
* bugs in Django, or
* deliberate limitations of Django core, that could be useful as
a third-party module.
And I'd like your feedback about where you feel these issues lie. I'm
hoping that we're not the only ones to encounter them!
1. GCBV and Vanilla Views do a great job for simple forms, but they leave
out embedded formsets entirely. (For example our large form has repeating
sections for employment history, education, etc.) It feels like it would
be great to have formsets handled "more automatically" - instantiating,
validating and saving them just like we do for forms. A lot of our code is
shared between all our formsets, and potentially reusable.
2. Adding instances to a formset on the client side using Javascript.
There is a django-formset-js package on PyPI, but it requires special
attributes to be added to form elements that would have been difficult
with crispy forms (which we're also using) and it feels almost like this
functionality ought to be in Django core (maybe not today, but one day?)
3. We couldn't change the "extra" of a formset without redefining the
formset class. We wanted to do this because the required repeating
sections (employment history etc.) must not have a blank form in them if
they already have some valid forms, otherwise the blank forms prevent the
form being submitted because HTML5 client-side validation fails. So we had
to do all this:
def get_context_data(self, **kwargs):
...
for name, formset_class in self.formsets.iteritems():
# doesn't exist yet, so we can't call its queryset() method
queryset = formset_class.model._default_manager.filter(
**{formset_class.fk.name: application})
extra = 0 if queryset.exists() else 1
# need to reconstruct the formset class to change extra?
formset_class = inlineformset_factory(
Application,
formset_class.model,
formset_class.form,
formset_class,
extra=extra,
can_delete=formset_class.can_delete,
fields=formset_class.form.base_fields.keys(),
max_num=formset_class.max_num,
)
4. We needed to be able to save the form without validation. To do this,
we had to make all fields non-required on our model, and make them
required on the form instead. Since nearly all fields are required, we'd
have to redefine them all, making the ModelForm a bit pointless, but
instead we were naughty and looped over the fields setting their required
values to True, and also updating the widget (which gets a copy of this
attribute on constructions). Naughty but DRY. I feel that a shortcut to
list required fields of a ModelForm in its Meta class would be useful
(overriding the blank and null status of the underlying Model fields).
def _save_data_without_validation(self, form):
# construct a new form with the required value turned off on all
# fields, and use that to save the instance.
all_optional_form = self.get_form(
data=self.request.POST,
files=self.request.FILES,
instance=form.instance
)
for field in all_optional_form:
field.field.required = False
all_optional_form.full_clean()
5. We have to be able to save a new instance of the model even if form
validation fails. So both the form_valid and form_invalid methods of the
CreateView return redirects to the UpdateView for the newly saved
instance. But that makes the form load through GET instead of POST, and
thus it isn't validated. This is fine for saving the form, but not for
submitting it, so we have to validate it like this:
def get_form(self, data=None, files=None, **kwargs):
# If we've been redirected here because of a form error, then we
need
# to validate the form again. To do that, we need to generate a
# simulated "POST data dict" from the model data, and clean the
form,
# which we wouldn't normally do on a GET request.
# http://stackoverflow.com/a/8996585/648162
request_origin = self.request.GET.get('type')
Re: Saving forms without validation, and formsets
Hi Chris,
I've definitely faced similar issues. A few scattered thoughts below:
On 03/18/2014 02:54 PM, Chris Wilson wrote:
[snip]
> 1. GCBV and Vanilla Views do a great job for simple forms, but they
> leave out embedded formsets entirely. (For example our large form has
> repeating sections for employment history, education, etc.) It feels
> like it would be great to have formsets handled "more automatically" -
> instantiating, validating and saving them just like we do for forms. A
> lot of our code is shared between all our formsets, and potentially
> reusable.
The approach I've used for this is encapsulating the creation and
handling of the inline formsets entirely within the "parent" form class,
so from the perspective of the view it acts like a normal simple form.
This requires overrides of a few methods on the parent form: __init__()
to create the formsets, is_valid() to ensure the formsets are valid too,
has_changed() to see if formsets have changed, and save() to save
formsets too.
If we were going to look at incorporating something into Django, I'd
like to consider this option as an alternative to adding more GCBVs with
inline-formset support. I think it's a nicer abstraction (because in a
real sense those inlines are "part of" the parent form), and I don't
think we want to add to the already-extensive list of GCBVs and mixin
classes.
I think either approach would be workable as a third-party project, too.
> 2. Adding instances to a formset on the client side using Javascript.
> There is a django-formset-js package on PyPI, but it requires special
> attributes to be added to form elements that would have been difficult
> with crispy forms (which we're also using) and it feels almost like this
> functionality ought to be in Django core (maybe not today, but one day?)
I've used this: http://plugins.jquery.com/django-superformset/
I am skeptical of adding things to Django core that can be implemented
mostly or entirely on the front-end. Django is a server-side framework.
I think changes to the forms library that make it easier to implement
such client-side libraries is a better place to start.
> 3. We couldn't change the "extra" of a formset without redefining the
> formset class. We wanted to do this because the required repeating
> sections (employment history etc.) must not have a blank form in them if
> they already have some valid forms, otherwise the blank forms prevent
> the form being submitted because HTML5 client-side validation fails. So
> we had to do all this:
>
> def get_context_data(self, **kwargs):
> ...
> for name, formset_class in self.formsets.iteritems():
> # doesn't exist yet, so we can't call its queryset() method
> queryset = formset_class.model._default_manager.filter(
> **{formset_class.fk.name: application})
> extra = 0 if queryset.exists() else 1
>
> # need to reconstruct the formset class to change extra?
> formset_class = inlineformset_factory(
> Application,
> formset_class.model,
> formset_class.form,
> formset_class,
> extra=extra,
> can_delete=formset_class.can_delete,
> fields=formset_class.form.base_fields.keys(),
> max_num=formset_class.max_num,
> )
I don't understand this. The 'extra' attribute of a formset class can be
tweaked dynamically as a normal attribute (although this may not be
documented); it doesn't require recreating the entire formset class.
I snipped your last three items, regarding saving invalid forms. I think
this is an unusual use case, and I'm not sure support for it belongs in
core. It would be interesting to experiment with something to make
filefields less painful when validation fails, but that can be a
third-party project first I think.
Carl
signature.asc
Description: OpenPGP digital signature
Re: [GSOC] Introduction and task proposal
Hi all, As promised, I have been working on the formalizing Meta task. I apologize to not have posted in the last days, but I have come back with my proposal. I want to post this now to the community, in order to gain feedback and re-iterate. Formalizing Meta Enabling users to build custom stores that work well with Django https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit# Kind regards, Daniel On Tuesday, March 11, 2014 10:48:43 PM UTC, Daniel Pyrathon wrote: > > Hi Russel, > > Sorry for getting back now (I did not have notifications set up!). Thank > you very much for the suggestions, I will have a look at models/options.py > right away. > > Regards, > Daniel Pyrathon > > On Thursday, March 6, 2014 12:03:04 AM UTC, Russell Keith-Magee wrote: >> >> Hi Daniel, >> >> On Wed, Mar 5, 2014 at 11:48 PM, Daniel Pyrathon wrote: >> >>> Hi, >>> >>> My name is Daniel Pyrathon. I am currently a third year BSc student in >>> Computer Science at the University of Plymouth. >>> >>> I love programming and I have always been active in the Open Source >>> community (especially Python). In the past years I have written lots of >>> Python, Javascript, Ruby, Java, and I am currently using C++ for many >>> university projects. I have attended the last 3 EuroPython conferences and >>> I have been a staff member of the conference for the last 2 years. >>> >>> I am currently looking for a way to contribute to Django. Working on >>> Django would increase my knowledge of the framework as well as let me share >>> my own experience. >>> >>> Reading the ideas list I found 2 of them that are very interesting for >>> me, and so the reason behind this post is not only to present myself but >>> also to discuss their feasibility. >>> >>> Formalizing the Meta object >>> >>> This task is very challenging because it digs in the internals of >>> Django. I feel that I could learn a lot from it because I am very committed >>> to refactoring and write most of my code in TDD. I have also experience >>> with backwards compatibility. >>> >>> Do you have any resources (code) I should read to get up to date and to >>> understand better how it is currently implemented? >>> >> >> Unfortunately not; at least, not other than just trying to untangle the >> mess that is django/db/models/options.py and the things that depend on it >> (ModelForms and Admin in particular). This project really is the very model >> of untangling a ball of string. The reason it's listed as a potential >> project is specifically *because* there are no resources we can point you >> at. >> >> If, after looking at the code, you feel it's a little "thin" for 12 >> weeks, one way to bulk it out is to build a proof of concept alternate >> implementation of Meta. Aside from the "it would be good to document this" >> aspect, the practical reason for wanting to formalise Meta is that it would >> allow people to build duck-typed implementations of Meta. If you can build >> an alternate implementation of the Meta interface, then you should be able >> to deploy your "duck" into a Django project and build a ModelForm, or >> browse it in Admin. >> >> So, for example, you could build a wrapper around a NoSQL store, or >> around an LDAP or email store, that *looked* like a Django model from the >> outside. This means you could view your NoSQL data, or LDAP records, or >> emails in Admin without needing to go through a SQL database. >> >> The aim of proof of concept wouldn't be to commit something to Django's >> core - it would be to build an external, standalone proof-of-concept, >> demonstrating that your documentation was complete and correct. Depending >> on what backend you choose, it might turn into a fully viable project on >> it's own. >> >> >>> Improved error reporting >>> >>> The idea of making people’s lives better by improving error messages is >>> fundamental. There would be a lot to discuss: what type of imports would we >>> want to mask? I have read BetterErrorMessages and would be happy to get >>> started soon. My idea behind this task would be to expand on this ticket: >>> what would be great is to add a web console with live REPL support, similar >>> to what Werkzeug debugger does. This could be a great starting point and >>> would lead to a better use of Django. >>> >> >> This is an interesting idea; however, I see two problems: >> >> 1) It would involve reinventing the wheel. Werkzeug exists, and does its >> job well; a GSoC project to "duplicate Werkzeug" doesn't strike me as a >> good use of GSoC resources. However, a project to integrate Werkzeug's live >> debugging capabilities into Django might be more viable. >> >> 2) Security implications. Unfortunately, more than one site has been >> launched with debug=True accidentally left on; all you need to do then is >> stimulate a server error, and you have REPL shell access to the server. >> This strikes me as a rema
Re: [GSOC] Introduction and task proposal
2) Security implications. Unfortunately, more than one site has been > launched with debug=True accidentally left on; all you need to do then is > stimulate a server error, and you have REPL shell access to the server. > This strikes me as a remarkably effective foot-gun :-) Before you get too > involved in the implementation, I'd want to know the security issues have > been locked down. > Curious, how do you get REPL shell access to the server with DEBUG=True with a vanilla Django deployment? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/363c7b06-5a62-453d-9253-68bcb24b4398%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
