Re: Model-level validation
James - these developers come from all over. They represent the random sample of people who end of working on or inheriting legacy django projects. Probably more Rails than anything else, but a few who probably worked with Hibernate or Node frameworks. For better or worse, it's who's using django. The reason I don't want to use serializers or forms in celery tasks is because validation should happen every time a model attribute is changed. This pattern would mean more imports and boilerplate scattered around my codebase any time I want to update a status attribute on a model. Pure boilerplate vs. just calling `full_clean()` on save. full_clean() is even a method already present on Model! Why import a *third-party library* just so a model instance can call it's own method?!? I think you (and Django at large) are conflating Validation and conversion/serialization/santization/input filtering, I think of them as separate concepts. Validation should happen everywhere, because bugs can happen anywhere. Input filtering is a concern of the external interfaces. Other frameworks treat them separately and I find it to be a far more robust and flexible pattern. On Friday, October 7, 2022 at 11:28:58 PM UTC-7 James Bennett wrote: > On Fri, Oct 7, 2022 at 6:21 PM Aaron Smith wrote: > >> Mariusz - fair enough, I will consider my point made and apologies if it >> came off too strong. FWIW it's not just my opinion, it's shared by every >> developer (dozens) I've had this conversation with up until now. It's a >> stark contrast that makes me wonder how aware the core developers / old >> timers are of the broader user base's experience. > > > I would wonder how many of these developers you've talked to are used to > working in Python. > > The main standalone ORM package people use in Python is SQLAlchemy, which > *also* does not do validation in the ORM layer. The last time I worked with > Flask, the standard practice was to use Marshmallow to write serializers, > and these days the popular async frameworks like Starlite and FastAPI have > you write Pydantic models. In either case they fill the role of, say, a DRF > serializer -- they do the validation, and data type conversion at the > application boundaries, so that the ORM doesn't have to. > > It's true that when you start branching out into other languages you'll > encounter ORMs which have validation built-in, like Entity Framework or > Hibernate, but you'll also more often encounter that in statically-typed > languages where the data conversion step has already been handled for you. > It's also not always clear that the ORM is the right place for validation, > since often the rules being enforced are ones that aren't actually enforced > at the DB level by constraints. > > Either way, I think I've made the case for why Django doesn't and > shouldn't do this. You seem to have a strong reluctance to use either > Django forms (in a "vanilla" Django project) or DRF serializers (in a more > "API" project) to validate data from sources other than direct > user-initiated HTTP request, but I don't really get that -- the validation > utilities are there, and if you're not willing to use them that still is > not Django's problem to solve -- after all, someone else might be equally > set in their conviction that all the existing validation layers are the > wrong way to do things, and demand we add yet another one, and I doubt > you'd be supportive of that. > > So I think Django should continue to be Django, and validation should > continue to be a layer independent of the ORM (which, as I originally > noted, it *has* to be in a web framework, since not every use case for > validation will end up touching the database). For that reason I'd be very > strongly against ever adding even an optional default enforcement of > model-level data validation. > >> -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/888f34cb-1f21-4036-ac16-79be14c74541n%40googlegroups.com.
Re: Model-level validation
On Sat, Oct 8, 2022 at 8:44 AM Aaron Smith wrote:
> The reason I don't want to use serializers or forms in celery tasks is
> because validation should happen every time a model attribute is changed.
> This pattern would mean more imports and boilerplate scattered around my
> codebase any time I want to update a status attribute on a model.
>
This feels to me like an important symptom -- regardless of ORM design
patterns (which I'll get to in a moment), data mutation should occur only
in well-defined, controlled ways. Having it "scattered around [the]
codebase" is somewhat worrying. In an Active Record ORM like Django, very
commonly this is just methods on the model itself exposing the desired
logical operations ("resolve this ticket", "publish this article", etc.),
while in a lot of Data Mapper ORM setups it would be in a "business logic"
or "domain" object or some sort of Repository pattern or CQRS setup or...
well, lots of options, but either way it would be strongly discouraged to
be directly mutating data from lots of places around the codebase.
I think you (and Django at large) are conflating Validation and
> conversion/serialization/santization/input filtering, I think of them as
> separate concepts.
>
I still think there's a static-versus-dynamic thing going on here where the
type conversion is seen as an afterthought in statically-typed frameworks
simply because they're statically typed (and a similar phenomenon occurs in
Python with Pydantic).
But the basic point -- that a web framework will have to do validation
that's completely independent of the persistence layer -- stands. There are
simply too many use cases for
forms/submissions/payloads/whatever-you-call-them that *don't* get
persisted to the DB, as I mentioned in my original message. Once that's
established, it's inevitable that a validation -- not just conversion --
layer is needed that isn't tied to the persistence layer. And once that
exists, it's reasonable to ask whether having *another* one in the
persistence layer is wasteful/redundant.
Anyway, I've explained this about as thoroughly as I can now. I think the
real underlying problem here is an unsupported generalization from liking a
particular pattern to deciding that pattern is objectively the only
correct/acceptable one. There are lots of acceptable patterns for how to
design and build and use ORMs and surrounding code. Django has settled on
one in particular, and isn't the only Python framework or ORM to have
settled on it. That doesn't mean it's wrong or bad, just that it's
different from the pattern you want or are used to.
And so I am still very strongly against trying to push a
model-layer-validation approach in Django, even optionally. Django's
pattern is fine, and works well/makes sense for a web framework, for the
reasons I've gone over multiple times now. I would suggest once again that
you adapt to it, because fighting against a framework is never pleasant.
Or, failing that, I'd suggest that you look into switching to something
better suited to your preferences.
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAL13Cg9NBVH-enng5C_4KaSQeRUsWLe5%2BEnkomsgCKXGj2UkiA%40mail.gmail.com.
Re: Model-level validation
> > And so I am still very strongly against trying to push a > model-layer-validation approach in Django, even optionally. > It already exists, though. `full_clean()` is a method on Model. CharFields on the model *already* have a notion of allowed choices. Validators are *already* an option on model fields. *Models already do their own validation.* But they require additional implementations of classes further up the stack to actually be triggered. While validation not being a concern of the model layer is a valid design choice (not one that I would prefer, but one that I could live with) the root of the problem here is that the concept of validation is spread across *multiple* layers in a way that's extremely misleading for people coming to Django from other ORMs. If I can specify `validators` on a field definition, I should be able to expect that it's actually run without extra steps. Surely we can agree that *something* should happen here? The status quo is confusing, a footgun and a gotcha. If it's not Model's concern, then get it out of Model. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f136472a-326a-41c2-8ccd-a8024f79b6f4n%40googlegroups.com.
Re: Model-level validation
On Sat, Oct 8, 2022 at 8:32 PM Aaron Smith wrote: > Surely we can agree that *something* should happen here? The status quo >> is confusing, a footgun and a gotcha. If it's not Model's concern, then get >> it out of Model. >> > I've already said that I wish model-level validation hadn't been added to Django. Unfortunately it's something that's been in long enough now that it would be very difficult to deprecate and remove. But that's not an argument, to my mind, for expanding it. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAL13Cg8ZvG6%3DtCNEEKV-4rQURzZ_GFSuPRypkoOOqGULFC9pZw%40mail.gmail.com.
