Hi Pascal,
I know this is a very late reply, but there have been some things going
round my head that I thought I should get down.
*First,* some positive changes I think we can make from your suggestions:
1. I think in our release notes, for breaking changes that might need
some justification, we should link to the tickets and/or django-devs
discussions behind them. Adding full reasoning in the notes themselves
would likely bloat them far too much, but a link could be helpful for
anything which is not obviously necessary.
2. We should change the wording in our API stability docs
<https://docs.djangoproject.com/en/stable/misc/api-stability/>, which
does indeed initially sound like a promise never to break working code
that follows docs:
Django promises API stability and forwards-compatibility since
version 1.0.
We do have this line later on:
If, for some reason, an API declared stable must be removed or
replaced, it will be declared deprecated but will remain in the API
for at least two feature releases. Warnings will be issued when the
deprecated method is called.
But we do make changes at a rate faster than that document would make
you think.
I suggest that as a contrast to the existing text on that page, we also
point out that we are continually improving Django, and have a
"(eventually) one way to do it" policy regarding how we plan APIs, which
means we will remove old things, while always providing better
alternatives. I'm happy to write a patch for this.
*Second,* there were things I wanted to say that might be useful to you
(Pascal) in understanding why you are not likely to get very far with
your current approach.
Your exaggerated tone is not likely to help you. You talk about Django's
compatibility breakages as "/ruining the day of thousands of anonymous
plugin/website maintainers all over the world/". When we make these kind
of changes, we always do it with very clear and detailed upgrade notes,
and almost always with several releases emitting deprecation warnings,
as per our policy. And no-one is forced to upgrade immediately. Those
who value stability above other things can use LTS, which gives them a
*3 year release cadence,* not 8 months, which is far from excessively
fast compared to other projects.
My involvement in Django these days is primarily not as a core
contributor, but as a user and library maintainer, and I've also
contributed many patches to different 3rd party Django projects to fix
them for Django upgrades. The impression that you give of Django being a
continuously shifting sand is very far from my experience. The core
features and structure of Django have remained remarkably stable over
its history. A few import changes here and there are very easy to fix
(and, more importantly, noisy and easy to diagnose), and other changes
are usually straightforward. Normally upgrades to libraries I maintain
to support the latest Django version require very little work, sometimes
zero, and often I'm able to support 4 or 5 Django versions without a
problem. Using something like django-compat-patcher, or some of the
other similar libraries, would add more complexity than the few lines of
code I need otherwise.
Even larger, more complex Django projects like Mezzanine, which use
quite a few hacks and internals, are often able to span across two LTS
versions.
With all this in mind, it's difficult to see how we are making so many
people's lives miserable.
Instead of debating in abstract, or answering your points one by one, I
thought a few illustrations might help you see the benefits of Django's
approach. The first is the longest, it might be of historical interest
to others.
1. A 14 year old Django project
I started work on my first Django project in September 2005
<https://lukeplant.me.uk/blog/posts/a-django-website-that-took-a-lot-more-than-20-minutes/>,
and it went live in March 2006. It has been running continuously since
then - perhaps by now it can claim to be one of the oldest continuously
running Django sites? I'd be interested to know if anyone can beat it...
Here's a snippet from one of my models.py nearly 14 years ago:
class Message(meta.Model):
fromUser = meta.ForeignKey(User,
verbose_name="from user",
related_name="messageSent"
)
toUser = meta.ForeignKey(User,
verbose_name="to user",
related_name="messageReceived")
time = meta.DateTimeField("At")
text = meta.TextField("Message")
box = meta.PositiveSmallIntegerField("Message box",
choices = MESSAGE_BOXES)
def __repr__(self):
#return str(self.id)
return "[" + str(self.id) + "] to " + repr(self.get_toUser()) + " from
" + repr(self.get_fromUser())
class META:
admin = meta.Admin(
list_display = ('toUser', 'fromUser', 'time')
)
ordering = ('-time',)
Wow, just look at that! ORM stuff under a `meta` namespace. Admin
options stored under the model META inner class. These funky `get_XXX()`
accessors for related fields. (Not to mention my naming conventions and
code formatting back then...) While I have memories of some of the big
changes in syntax that happened pre 1.0, I had no memory that we use to
do most of these things. Why? Because this project no longer does it
this way. My brain thankfully reclaimed the space taken up by these old
ways of doing things. If we developed Django according to the backwards
compatibility philosophy you are suggesting, that would not have
happened. *Instead, my code would be just as bad/quirky today as was did
back then*, for two reasons:
1. Without being forced to upgrade, I probably wouldn't have done.
2. If Django devs had needed to keep supporting old ways of doing things
for long periods of time, two things could have happened:
* they would have been very conservative about adding new things,
because it adds complexity to Django itself, and because of the
principle of "One Way To Do It". So the new improved ways often
wouldn't even exist.
* or, they would have just piled on the new things while leaving the
old. Quite quickly you get to a point where the quantity of cruft,
and the need to support all the different legacy ways, means that
adding new things becomes very difficult, and development would have
stagnated and quality dropped. To use your illustration, you can't
build a castle on a weak foundation.
When I work on this old project I find:
1. *It is virtually indistinguishable from brand new Django code.*
2. *The code quality has improved over time, and the project has never
been in better shape than now.
*
I have none of that "Oh no I've got to fix that old project, how did we
use to do this stuff again?" feeling. This project is a breeze to work
with, and feels completely modern. If I needed to pass it on to someone
else, I would be less worried now about doing so than I ever have been.
How many other 14 year old projects can you say such things about?
I could multiply examples. Take URLs. Here's the old code:
from django.conf.urls.defaults import *
urlpatterns = patterns('cciw.apps.cciw.views',
(r'^thisyear/$', 'camps.thisyear'),
(r'^camps/$', 'camps.index'),
(r'^camps/(?P<year>\d{4})/?$', 'camps.index'),
(r'^camps/(?P<year>\d{4})/(?P<number>\d+)/?$', 'camps.detail'),
(r'^sites/$', 'sites.index'),
(r'^sites/(?P<name>.*)/$', 'sites.detail'),
(r'', 'htmlchunk.find')
)
Versus the current code:
from django.urls import path
from cciw.cciwmain.views import camps as camp_views
from cciw.cciwmain.views import sites as sites_views
urlpatterns = [
# Camps
path('thisyear/', camp_views.thisyear, name="cciw-cciwmain-thisyear"),
path('camps/', camp_views.index, name="cciw-cciwmain-camps_index"),
path('camps/<yyyy:year>/', camp_views.index,
name="cciw-cciwmain-camps_year_index"),
path('camps/<yyyy:year>/<slug:slug>/', camp_views.detail,
name="cciw-cciwmain-camps_detail"),
# Sites
path('sites/', sites_views.index, name="cciw-cciwmain-sites_index"),
path('sites/<slug:slug>/', sites_views.detail,
name="cciw-cciwmain-sites_detail"),
# ...
]
There are many improvements, for example:
* Star imports have been removed. There are lots of good reasons why
every Python code linter complains about these. Or would you rather
still have them?
* In my editor I can put my cursor on a view function like `index`, do
"jump to definition" and it does the right thing, because it is
normal Python. Do you think dotted strings were better?
* My list of URLs is just a list, and can be manipulated with normal
list methods and functions (which I often need to do). Do you think
wrapping everything in `patterns()` would be an improvement? For the
sake of an extensibility we might just use someday (though we didn't
in well over a decade)?
* The addition of `path` is a great help, and has really cleaned up
URL handling a ton, and even more in other areas of my project. I
wasn't forced to make this change, but it's a great improvement.
*However, this is the kind of big addition that the Django
developers only considered and were able to do because the Django
codebase was not a pile of backwards compatibility hacks and cruft
already. *And yes, doing so meant we lost some /undocumented/
classes which were implementation details, which you also seem to be
complaining about.
Everything is simpler and more Pythonic - while at the same time the
fundamental structure and abstractions are extremely similar, and every
migration has been straightforward. It is unfortunate that we didn't
have a crystal ball in order to optimize these migrations a bit more, or
indeed get it right from the start. But that's life, we don't have
crystal balls.
All of the changes with URLs have had thought through reasons. Often the
changes fix issues that newbies consistently trip over (based on
feedback from people who do a lot training of new Django users) and
security issues we've had.
2. Wordpress
You mentioned PHP and Wordpress, and I would agree that their approach
is quite different. But it also comes with huge costs. Thankfully I
already wrote a blog post that covers pretty much everything I want to
say, with quite a few comparisons to Django:
https://lukeplant.me.uk/blog/posts/wordpress-4.7.2-post-mortem/
A zero day vulnerability in which your web site gets defaced and
possibly fully hacked is what I think of when I hear the phrase "my day
was ruined", and that is the kind of vulnerability we've so far been
able to avoid in Django.
Critical security vulnerabilities like the above are also just the tip
of the iceberg - the same issues that cause them also cause all kind of
bugginess which hurt you in much more every-day ways, but these costs
add up.
3. AngularJS
I'm surprised you let AngularJS off the hook so lightly. One of my
clients has a project with 25,000 lines of AngularJS code. Having looked
at the detailed upgrade instructions, in the context of our application
the first step essentially amounts to "first re-write your entire app"
(to use components everywhere, which we are not). There are entire books
<https://www.upgradingangularjs.com/> devoted to this upgrade, it is a
massive undertaking that requires a deep understanding of both AngularJS
and Angular. The team is considering ditching everything and starting
with a new framework. This is the kind of situation that leaves
businesses in very bad situations, having to choose between *either
*being able to deliver new features for the next six months to a year
*or* rewriting and paying off the technical debt.
For my past and current clients, and all my side projects, I have never
faced anything remotely closely to that choice with Django. Sure, we've
had to schedule in some time for upgrades, especially if we've had a lot
of dependencies, but doing the upgrades, and fixing most of the
deprecation warnings too, for small to medium sized projects, usually
has been work on the scale of hours, sometimes days — maybe a week of
development work if we were ahead of the curve in terms of upgrading and
had to submit patches to lots of 3rd party dependencies. But never
months — never anything close to "senior management needs to get
involved with this decision".
*To sum up,* Django has so far avoided the huge upgrade costs of
something like AngularJS. We've never reached the stage where we've
thought "It's too awful, we'll have to start again", and never had to
write that blog post - exactly the kind of news which might indeed ruin
your day as a web developer or project manager. This might have been
luck at choosing the right abstractions while AngularJS had some bad
luck, but whatever the explanation, it's a great thing. At the same time
we've avoided the horrors of a codebase like Wordpress which has just
kept going with layer upon layer of cruft. Old Django projects (that
have been maintained), and the Django code base itself, remain a
pleasure to work with. That's also why we're able to attract excellent
developers, like our current Django Fellows, so that Django continues to
improve.
And we're not done yet - Django appears to have quite a bit of life left
it in still! I'm convinced that all of this has been due to it's
approach to improvement, gradual backwards compatibility breakages, and
not tolerating previous bad design
<https://lukeplant.me.uk/blog/posts/why-escape-on-input-is-a-bad-idea/#a-lesson-for-pythonistas>,
rather than despite it.
Hope that helps,
Luke
--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/b907b24d-29d9-8994-5d1a-154ca62e4a7b%40cantab.net.
