A nice HTML rendering of this proposal is available at: http://lukasz.langa.pl/2/upgrading-choices-machinery-django/
========================================== Upgrading the choices machinery for Django ========================================== Specifying choices for form fields and models currently does not do much justice to the `DRY <https://docs.djangoproject.com/en/1.4/misc/design-philosophies/#don-t-repeat-yourself-dry>`_ philosophy Django is famous for. Everybody seems to either have their own way of working around it or live with the suboptimal tuple-based pairs. This Django enhancement proposal presents a comprehensive solution based on an existing implementation, explaining reasons behind API decisions and their implications on the framework in general. Current status -------------- The current way of specifying choices for a field is as follows:: GENDER_CHOICES = ( (0, 'male'), (1, 'female'), (2, 'not specified'), ) class User(models.Model): gender = models.IntegerField(choices=GENDER_CHOICES) When I then want to implement behaviour which depends on the ``User.gender`` value, I have a couple of possibilities. I've seen all in production code which makes it all the more scary. Way 1 ~~~~~ Use ``get_FIELD_display``:: GENDER_CHOICES = ( (0, 'male'), (1, 'female'), (2, 'not specified'), ) class User(models.Model): gender = models.IntegerField(choices=GENDER_CHOICES) def greet(self): gender = self.get_gender_display() if gender == 'male': return 'Hi, boy.' elif gender == 'female': return 'Hello, girl.' else: return 'Hey there, user!' This will fail once you start translating the choice descriptions or if you rename them and is generally brittle. So we have to improve on it by using the numeric identifiers. Way 2 ~~~~~ Use numeric IDs:: GENDER_CHOICES = ( (0, _('male')), (1, _('female')), (2, _('not specified')), ) class User(models.Model): gender = models.IntegerField(choices=GENDER_CHOICES, default=2) def greet(self): if self.gender == 0: return 'Hi, boy.' elif self.gender == 1: return 'Hello, girl.' else: return 'Hey there, user!' This is just as bad because once the identifiers change, it's not trivial to grep for existing usage, let alone 3rd party usage. This looks less wrong when using ``CharFields`` instead of ``IntegerFields`` but the problem stays the same. So we have to improve on it by explicitly naming the options. Way 3 ~~~~~ Explicit choice values:: GENDER_MALE = 0 GENDER_FEMALE = 1 GENDER_NOT_SPECIFIED = 2 GENDER_CHOICES = ( (GENDER_MALE, _('male')), (GENDER_FEMALE, _('female')), (GENDER_NOT_SPECIFIED, _('not specified')), ) class User(models.Model): gender = models.IntegerField(choices=GENDER_CHOICES, default=GENDER_NOT_SPECIFIED) def greet(self): if self.gender == GENDER_MALE: return 'Hi, boy.' elif self.gender == GENDER_NOT_SPECIFIED: return 'Hello, girl.' else: return 'Hey there, user!' This is a saner way but starts getting overly verbose and redundant. You can improve encapsulation by moving the choices into the ``User`` class but that on the other hand beats reusability. The real problem however is that there is no `One Obvious Way To Do It <http://www.python.org/dev/peps/pep-0020/>`_ and newcomers are likely to choose poorly. tl;dr or The Gist of It ----------------------- My proposal suggests embracing a solution that is already implemented and tested with 100% statement coverage. It's easily installable:: pip install dj.choices and lets you write the above example as:: from dj.choices import Choices, Choice class Gender(Choices): male = Choice("male") female = Choice("female") not_specified = Choice("not specified") class User(models.Model): gender = models.IntegerField(choices=Gender(), default=Gender.not_specified.id) def greet(self): gender = Gender.from_id(self.gender) if gender == Gender.male: return 'Hi, boy.' elif gender == Gender.female: return 'Hello, girl.' else: return 'Hey there, user!' or using the provided ``ChoiceField`` (fully compatible with ``IntegerFields``):: from dj.choices import Choices, Choice from dj.choices.fields import ChoiceField class Gender(Choices): male = Choice("male") female = Choice("female") not_specified = Choice("not specified") class User(models.Model): gender = ChoiceField(choices=Gender, default=Gender.not_specified) def greet(self): if self.gender == Gender.male: return 'Hi, boy.' elif self.gender == Gender.female: return 'Hello, girl.' else: return 'Hey there, user!' BTW, the reason choices need names as arguments is so they can be picked up by ``makemessages`` and translated. But it's much more than that so read on. The Choices class ----------------- By default the ``Gender`` class has its choices enumerated similarly to how Django models order their fields. This can be seen by instantiating it:: >>> Gender() [(1, u'male'), (2, u'female'), (3, u'not specified')] By default an item contains the numeric ID and the localized description. The format can be customized, for instance for ``CharField`` usage:: >>> Gender(item=lambda c: (c.name, c.desc)) [(u'male', u'male'), (u'female', u'female'), (u'not_specified', u'not specified')] But that will probably make more sense with a foreign translation:: >>> from django.utils.translation import activate >>> activate('pl') >>> Gender(item=lambda c: (c.name, c.desc)) [(u'male', u'm\u0119\u017cczyzna'), (u'female', u'kobieta'), (u'not_specified', u'nie podano')] It sometimes makes sense to provide only a subset of the defined choices:: >>> Gender(filter=('female', 'not_specified'), item=lambda c: (c.name, c.desc)) [(u'female', u'kobieta'), (u'not_specified', u'nie podano')] A single Choice --------------- Every ``Choice`` is an object:: >>> Gender.female <Choice: female (id: 2, name: female)> It contains a bunch of attributes, e.g. its name:: >>> Gender.female.name u'female' which probably makes more sense if accessed from a database model (in the following example, using a ``ChoiceField``):: >>> user.gender.name u'female' Other attributes include the numeric ID, the localized and the raw description (the latter is the string as present before the translation):: >>> Gender.female.id 2 >>> Gender.female.desc u'kobieta' >>> Gender.female.raw 'female' Within a Python process, choices can be compared using identity comparison:: >>> u.gender <Choice: male (id: 1, name: male)> >>> u.gender is Gender.male True Across processes the serializable value (either ``id`` or ``name``) should be used. Then a choice can be retrieved using a class-level getter:: >>> Gender.from_id(3) <Choice: not specified (id: 3, name: not_specified)> >>> Gender.from_name('male') <Choice: male (id: 1, name: male)> Grouping choices ---------------- One of the problems with specifying choice lists is their weak extensibility. For instance, an application defines a group of possible choices like this:: >>> class License(Choices): ... gpl = Choice("GPL") ... bsd = Choice("BSD") ... proprietary = Choice("Proprietary") ... >>> License() [(1, u'GPL'), (2, u'BSD'), (3, u'Proprietary')] All is well until the application goes live and after a while the developer wants to include LGPL. The natural choice would be to add it after gpl but when we do that, the indexing would break. On the other hand, adding the new entry at the end of the definition looks ugly and makes the resulting combo boxes in the UI sorted in a counter-intuitive way. Grouping lets us solve this problem by explicitly defining the structure within a class of choices:: >>> class License(Choices): ... COPYLEFT = Choices.Group(0) ... gpl = Choice("GPL") ... ... PUBLIC_DOMAIN = Choices.Group(100) ... bsd = Choice("BSD") ... ... OSS = Choices.Group(200) ... apache2 = Choice("Apache 2") ... ... COMMERCIAL = Choices.Group(300) ... proprietary = Choice("Proprietary") ... >>> License() [(1, u'GPL'), (101, u'BSD'), (201, u'Apache 2'), (301, u'Proprietary')] This enables the developer to include more licenses of each group later on:: >>> class License(Choices): ... COPYLEFT = Choices.Group(0) ... gpl_any = Choice("GPL, any") ... gpl2 = Choice("GPL 2") ... gpl3 = Choice("GPL 3") ... lgpl = Choice("LGPL") ... agpl = Choice("Affero GPL") ... ... PUBLIC_DOMAIN = Choices.Group(100) ... bsd = Choice("BSD") ... public_domain = Choice("Public domain") ... ... OSS = Choices.Group(200) ... apache2 = Choice("Apache 2") ... mozilla = Choice("MPL") ... ... COMMERCIAL = Choices.Group(300) ... proprietary = Choice("Proprietary") ... >>> License() [(1, u'GPL, any'), (2, u'GPL 2'), (3, u'GPL 3'), (4, u'LGPL'), (5, u'Affero GPL'), (101, u'BSD'), (102, u'Public domain'), (201, u'Apache 2'), (202, u'MPL'), (301, u'Proprietary')] The behaviour in the example above was as follows: - the developer renamed the GPL choice but its meaning and ID remained stable - BSD, Apache and proprietary choices have their IDs unchanged - the resulting class is self-descriptive, readable and extensible The explicitly specified groups can be used as other means of filtering, etc.:: >>> License.COPYLEFT <ChoiceGroup: COPYLEFT (id: 0)> >>> License.gpl2 in License.COPYLEFT.choices True >>> [(c.id, c.desc) for c in License.COPYLEFT.choices] [(1, u'GPL, any'), (2, u'GPL 2'), (3, u'GPL 3'), (4, u'LGPL'), (5, u'Affero GPL')] Pushing polymorphism to the limit - extra attributes ---------------------------------------------------- Let's see our original example once again:: from dj.choices import Choices, Choice from dj.choices.fields import ChoiceField class Gender(Choices): male = Choice("male") female = Choice("female") not_specified = Choice("not specified") class User(models.Model): gender = ChoiceField(choices=Gender, default=Gender.not_specified) def greet(self): if self.gender == Gender.male: return 'Hi, boy.' elif self.gender == Gender.female: return 'Hello, girl.' else: return 'Hey there, user!' If you treat DRY really seriously, you'll notice that actually separation between the choices and the greetings supported by each of them may be a violation of the "Every distinct concept and piece of data should live in one, and only one, place" rule. You might want to move this information up:: from dj.choices import Choices, Choice from dj.choices.fields import ChoiceField class Gender(Choices): male = Choice("male").extra( hello='Hi, boy.') female = Choice("female").extra( hello='Hello, girl.') not_specified = Choice("not specified").extra( hello='Hey there, user!') class User(models.Model): gender = ChoiceField(choices=Gender, default=Gender.not_specified) def greet(self): return self.gender.hello As you see, the ``User.greet()`` method is now virtually gone. Moreover, it already supports whatever new choice you will want to introduce in the future. This way the choices class starts to be the canonical place for the concept it describes. Getting rid of chains of ``if`` - ``elif`` statements is now just a nice side effect. .. note:: I'm aware this is advanced functionality. This is not a solution for every case but when it is needed, it's priceless. Advantages of merging this solution ----------------------------------- 1. Having a single source of information, whether it is a list of languages, genders or states in the USA, is DRY incarnate. 2. If necessary, this source can later be filter and reformatted upon instatiation. 3. Using ``ChoiceFields`` or explicit ``from_id()`` class methods on choices enables cleaner user code with less redundancy and dependency on hard-coded values. 4. Upgrading the framework's choices to use class-based choices will increase its DRYness and enable better future extensibility. 5. Bikeshedding on the subject will hopefully stop. Disadvantages ------------- 1. A new concept in the framework increases the vocabulary necessary to understand what's going on. 2. Because of backwards compatibility in order to provide a *One Obvious Way To Do It* we actually add another way to do it. We can however endorse it as the recommended way. Performance considerations -------------------------- Creation of the proper ``Choices`` subclass uses a metaclass to figure out choice order, group membership, etc. This is done once when the module containing the class is loaded. After instantiation the resulting object is little more than a raw list of pairs. Using ``ChoiceFields`` instead of raw ``IntegerFields`` introduces automatic choice unboxing and moves the field ID checks up the stack introducing negligible performance costs. I don't personally believe in microbenchmarks so didn't try any but can do so if requested. FAQ --- Is it used anywhere already? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Yes, it is. It grew out of a couple of projects I did over the years, for instance `allplay.pl <http://allplay.pl/>`_ or `spiralear.com <http://spiralear.com/en/>`_. Various versions of the library are also used by my former and current employers. It has 100% statement coverage in unit tests. Has anyone evaluated it yet? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I have shown this implementation to various people during PyCon US 2012 and it gathered some enthusiasm. Jannis Leidel, Carl Meyer and Julien Phalip seemed interested in the idea at the very least. Why not use an existing Django enum implementation? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The most popular are: - `django-choices <http://pypi.python.org/pypi/django-choices>`_ by Jason Webb. The home page link is dead, the implementation lacks grouping support, internationalization support and automatic integer ID incrementation. Also, I find the reversed syntax a bit unnatural. There are some tests. - `django-enum <pypi.python.org/pypi/django-enum>`_ by Jacob Smullyan. There is no documentation nor tests, the API is based on a syntax similar to namedtuples with a single string of space-separated choices. Doesn't support groups nor internationalization. Why not use a generic enum implementation? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The most popular are: - `enum <http://pypi.python.org/pypi/enum>`_ by Ben Finney (also a rejected `PEP <http://www.python.org/dev/peps/pep-0354/>`_) - uses the "object instantiation" approach which I find less readable and extensible. - `flufl.enum <https://launchpad.net/flufl.enum>`_ by Barry Warsaw - uses the familiar "class definition" approach but doesn't support internationalization and grouping. Naturally, none of the generic implementations provide shortcuts for Django forms and models. Why not use namedtuples? ~~~~~~~~~~~~~~~~~~~~~~~~ This doesn't solve anything because a ``namedtuple`` defines a type that is later populated with data on a per-instance basis. Defining a type first and then instantiating it is already clumsy and redundant. Do you have a field like ``ChoiceField`` but holding characters? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Not yet but this is planned. I don't like having to write ``Choice()`` all the time. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can use a trick:: class Gender(Choices): _ = Choices.Choice male = _("male") female = _("female") not_specified = _("not specified") Current documentation for the project uses that because outside of Django core this is necessary for ``makemessages`` to pick up the string for translation. Once merged this will not be necessary so **this is not a part of the proposal**. You're free to use whatever you wish, like importing ``Choice`` as ``C``, etc. -- Best regards, Łukasz Langa Senior Systems Architecture Engineer IT Infrastructure Department Grupa Allegro Sp. z o.o. http://lukasz.langa.pl/ +48 791 080 144 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
