Hi, I'm David Ma, a first-year Science student from the University of British Columbia. I'm a enthusiastic Django developer and have four years Python programming experience. I've read through the posts about replacing the current autoreloader and would like to work on this task during GSoC. I paraphrase the discussions as follows:
Currently, the default autoreloader contains a lot of problems, most of which are caused by the inefficient methods to look for file changes (#27685). Notably, it fails to work with many python libraries including Jinja (#25624). In addition, if a syntax error is in the settings file, the autoreloader would stop working. We want to get rid of this by tracking the status of an entire folder and using subprocesses to reload the development server. On going discussions have deemed that the best way to solve this is to replace the current autoreloader with a new one implemented with Watchman (https://groups.google.com/forum/#!topic/django-developers/voXNIDdDcpU/discussion). Since inotify is a low-level library and cannot be applied to platforms including Windows and Macintosh OS, the best library to replace Watchdog is Watchman, a library that supports both of these three platforms (for Windows it's still in the alpha stage.) It runs as a standalone process and has a python wrapper library that is incompatible with Python 3. Therefore, one of the goals is to figure out how to interact with this library. There are also complaints on Django not tracking non-python files or tracking files that it is not supposed to watch. For the new version, we'd expect it to work with cached templates (#25791) and be customizable. So my goals would be: 1. Replace the old auto-reloader with the Watchman version 2. Make sure that the new auto-reloader can work with Python3 and on other platforms 3. Make it compatible with other Python libraries 4. Allow Django users to choose the files that should be watched 5. Integrate the Watchman package and its Python wrapper into DjangoI For Phase 1, I will write a prototype that can replace the current autoreloader. In the second phase, I'll come up with another one capable of handling cached files and excluding certain types of files based on the configuration, and the final version will be a cross-platform autoreloader that works with most of the Python libraries. Here's my development schedule: - May 30 – June 8: Surveying the functions implemented in autoreloader.py, researching the dependency of the functions included, and constructing the development environment. - June 8 – June 17: Play with the Watchman library; design a way to interact with Watchman (probably as a standalone process); write wrappers for Watchman. Make a list of files that is not monitored by default (e.g. sqlite files or files under the media directory) and make sure it works with settings.py. - June 17 – June 25: Implement the auto-reloader from scratch; make sure the new reloader can work with Django properly (particularly can survive a syntax error and would not be affected by cached files.) - July 26 – June 30: Phase 1 evaluations - July 1 – July 8: Add appropriate benchmarks and check whether if the new implementation meets the expectation. Test whether if the new implementation can detect the addition of a file in the trivial cases and work with applications like git. - July 8 – July 15: Figure out how to monitor cached files in Django; add an option that enables Watchman to ignore all cached files. Check whether if this solution works with cached templates. - July 15 – July 24: Designate options to control the files monitored by Watchman with a standard include / exclude mechanism; write a function to read the relevant portion in the configuration file. - July 24 – July 28: Phase 2 evaluations - August 2 – August 15: Test compatibility with other libraries including Jinja2 and Jython; revise the code if necessary. Test the autoreloader on Windows and Macintosh OS. - August 15 – August 21: Integrate the Watchman library and its wrappers into Django (might need to submit pull requests to the pywatchman project.) - August 21 – August 29: Final evaluations Thanks for reading my proposal. If you have any questions or concerns, please do not hesitate to comment. Any advice and suggestions would be greatly appreciated! -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/455343df-6401-45eb-8b08-45e51ef7896b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
