Re: structural & functional review of django documentation

[Tim Allen]

HI Doug, I can relate to what you are saying, I had a similar experience 
when trying to find a reference for the generic FormView. Issuing a P.R. to 
improve it is on my BLOTTD (big list of things to do). I was interviewing a 
candidate last week who knows I'm a big fan of Django, and wanted to throw 
a curveball, so I asked what they thought the biggest shortcoming of Django 
is. Without hesitation, they mentioned that their current team (which is 
just starting to learn Django) had trouble with the documentation. When I 
followed up, they mentioned it all seemed to be by example, rather than 
reference, and both are useful.

It is a big fish to fry, as Tim G mentioned, but I think it can be improved 
incrementally. I've used the Django Documentation for several years, and 
this is the first I've heard of the "topics", "reference" and "how-to".

As someone who used PHP for years, one thing I always felt they got right 
was documentation. That may just be the way my mind works, however.

This is a long way of saying, I'm interesting in helping out.

Regards,

Tim

On Sunday, December 27, 2015 at 9:06:46 PM UTC-5, Doug Epling wrote:
>
> Tim --
>
> Thank you so much for the link to that blog post.  I am going away now to 
> think about that data a little, but I'll be back.
>
> I was not aware of the "topics", "reference", and "how-to" assortment.  
> But I have scanned the Table of Contents, and I find almost nothing here 
> addressing documentation 
> .  
>
> I am afraid I have caused enough trouble for now.  I need a little time to 
> digest that survey data, but I will be back.
>
> When I look at Django docs, I want to be informed as a Python programmer, 
> perhaps even a novice one.  As far as I can tell, there is no such thing in 
> Django as Forms -- forms or form either for this matter.  There is a Form 
> class, and it can instantiate any number of objects.  But THIS KIND OF 
> THING (caps for emphasis on this example) is the kernel of my frustration. 
>
> Again, many thanks Tim, your last post is very helpful to me.
>
> On Friday, December 18, 2015 at 7:02:56 PM UTC-5, Doug Epling wrote:
>>
>> I filed bug report 
>> #25952 but apparently it 
>> was in the wrong place.  And I referenced this post 
>> ,
>>  
>> but I was thinking it was this group ... I wonder how that happened?
>>
>> So I am hereby suggesting that the road map for the v. 2.0 release 
>> include revamped documentation.  
>>
>> It should begin as soon as possible with the somewhat standard best 
>> practice of collecting "find what you were looking for" or "was this page 
>> helpful" or "rate this page on its organization, clarity, brevity, etc." 
>> data on every single existing page.  
>>
>> It might also be helpful to evaluate how different audiences access the 
>> docs.  Tutorials are great -- module and class libraries, not so much.
>>
>> With resulting user feedback along with expert categorization of 
>> documentation use cases, as with any writing exercise, there must be an 
>> outline.  The existing outline might be a good place to start.
>>
>> Oh, and those pesky deadlines, when is v. 2.0 slated for release?
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/025ed576-59e1-45cf-9cbe-be11ee56115d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: structural & functional review of django documentation

[Aymeric Augustin]

On 27 déc. 2015, at 23:42, Tim Graham  wrote:
> In my view, Django's docs haven't strayed from the "topics", "reference", and 
> "how-to" division that we've had since 1.0 or so.

I’ve been around for some time and, to be honest, I still have a hard time 
using this classification efficiently.

> Maybe a "how the docs are organized" section on the index page would help 
> communicate that and make it more intuitive where to look for something?

This would help documentation writers at least as much as documentation 
readers. By providing guidelines for what kind of content should go where, it 
could help limit overlap between each type of document.

-- 
Aymeric.




-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/DD6E5C59-85B4-482C-AC8D-03FCB3ECCB17%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

URL dispatching framework: feedback requested

[Marten Kenbeek]

Hi everyone,

This past week I've made some great progress in rewriting the URL 
dispatcher framework and iterating on my implementation. A big part of my 
effort to refactor my original code was to increase the performance of 
reverse() to a level similar to the legacy dispatcher, and to decouple the 
various parts of the code. I think I have now achieved both goals, so I'd 
like to get some feedback on the result. 

The current code can be found at https://github.com/django/django/pull/5578.

I will be cleaning up the code and shuffling around some of it. There's 
still a lot to be done, but the high-level design and public API are pretty 
much finished and ready for review. 

The API consists of 4 parts, most of which are extendible or replaceable: a 
Dispatcher, a set of Resolvers, Constraints and the URL configuration. 

The main entry point for users is the Dispatcher class. The dispatcher is 
responsible for resolving namespaces and reversing URLs, and handles some 
of the utility functions available to users (some more may be moved here, 
such as is_valid_path() or translate_url()). It is a thin wrapper around 
the root Resolver to allow a single entry point for both reversing and 
resolving URLs. It currently provides the following public API:

   - Dispatcher.resolve(path, request=None) -> Resolve path to a 
   ResolverMatch, or raise Resolver404. 
   - Dispatcher.resolve_namespace(viewname, current_app=None) -> Resolve 
   the namespaces in viewname, taking current_app into account. Returns 
   resolved lookup in a list.
   - Dispatcher.reverse(lookup, *args, **kwargs) -> Reverse lookup, 
   consuming *args and **kwargs. Returns a full URL path or raises 
   NoReverseMatch. 
   - Dispatcher.resolve_error_handler(view_type) -> Get error handler for 
   status code view_type from current URLConf. Fall back to default error 
   handlers. 
   - Dispatcher.ready -> (bool) Whether the dispatcher is fully 
   initialized. Used to warn about reverse() where reverse_lazy() must be 
   used. 

I'm currently looking into possible thread-safety issues with 
Dispatcher.load_namespace(). There are some parts of Django that depend on 
private API's of Dispatcher and other parts of the dispatching framework. 
To maximize extensibility, I'll look if these can use public API's where 
appropriate, or gracefully fail if a swapped-in implementation doesn't 
provide the same private API. One example is admindocs, which uses the 
Dispatcher._is_callback() function for introspection. 

If a developer wishes to completely replace the dispatcher framework, this 
would be the place to do it. This will most likely be possible by setting 
request.dispatcher to a compatible Dispatcher class. 

The BaseResolver class currently has two implementations: Resolver and 
ResolverEndpoint. The resolver's form a tree structure, where each resolver 
endpoint is a leaf node that represents a view; it's job is to resolve a path 
and request to a ResolverMatch. Users will mostly use this through 
Dispatcher.resolve(), rather than using it directly. Its public API 
consists of two functions:

   - BaseResolver.resolve(path, request=None) -> Return a ResolverMatch or 
   raise Resolver404.
   - BaseResolver.describe() -> Return a human-readable description of the 
   pattern used to match the path. This is used in error messages. 

There is a slightly more extensive API that allows a resolver to "play 
nice" with Django's resolver implementations. This allows a developer to 
replace a single layer of resolvers to implement custom logic/lookups. For 
example, you can implement a resolver that uses the first hierarchical part 
of the path as a dict lookup, rather than iterating each pattern. To make 
this possible, a resolver should accept the following arguments in its 
constructor:

   - BaseResolver.__init__(pattern, decorators=None) -> pattern is a 
   URLPattern instance (explained below). decorators is a list of 
   decorators that should be applied to each view that's a "descendant" of 
   this resolver. This list is passed down so the fully decorated view can be 
   cached. 

I'm still looking how exactly we'd allow a developer to hook in a custom 
resolver, any ideas are welcome. 

Constraints are the building blocks of the current dispatcher framework. A 
Constraint can (partially) match a path and request, and extract arguments 
from them. It can also reconstruct a partial URL from a set of arguments. 
Current implementations are a RegexPattern, LocalizedRegexPattern, LocalePrefix 
and ScriptPrefix. This is the main extension point for developers. I 
envision that over time, Django will include more constraints into core for 
common use-cases. One can for example implement a DomainConstraint or 
MethodConstraint 
to match a domain name or request method in the URL, or implement a set of 
constraints based on the parse library for better performance than the 
built-in regex-based constraints. A Constraint currently has the followin

Re: structural & functional review of django documentation

[Samuel Bishop]

Broadly speaking, I think the 'optimal' goal is not going to be one that 
changes our existing documentation structure in any disruptive way.
 
I think the general concept would be covered by either creating a "fourth 
division". So we would go from "topics", "reference", and "how-to", to 
"topics", "reference", "how-to", and "implementation"/"internals"/"APIs"/etc
Or enhancing the content in the reference section so that in addition to 
our existing handwritten documentation, we expose the 'api-docs' tree as an 
addendum to the reference section.

Sphinx gives us a lot of power to work with for trying either approach. I 
cant remember which project I saw it on, but I recall seeing some sphinx 
powered projects that had Django style hand written documentation, and each 
heading (module, class, function, etc) in their hand written documentation 
had an automatically generated link to the matcihng "implementation docs" 
that took you to that segment of the sphinx autodoc output. It worked quite 
well from what I recall. 
I'm pretty busy for a few days, but if no one else does this first and 
shares some results, I'll definitely take a look at the output from 
sphinx's autodoc tools when run over the Django repo and post some 
findings. 
Since there is certainly the big question of "is the information we have 
useful"? 
The docstrings and sundry that sphinx uses for autodoc generation may need 
work before it measures up to the quality standards of our existing hand 
written documentation. 

On Monday, 28 December 2015 06:42:49 UTC+8, Tim Graham wrote:
>
> Adding a survey link is not difficult. We conducted a community survey [1] 
> earlier this year with one question related to documentation, "What parts 
> of the Django documentation do you find most useful?" What questions to ask 
> and how to turn the answers into actionable items is the part I'm not sure 
> about and maybe you could advise on.
>
> In my view, Django's docs haven't strayed from the "topics", "reference", 
> and "how-to" division that we've had since 1.0 or so. Are you aware of this 
> grouping? Maybe a "how the docs are organized" section on the index page 
> would help communicate that and make it more intuitive where to look for 
> something?
>
> I'll admit I'm skeptical of a wholesale reorganization to this structure 
> for several reasons:
> 1. It'll be confusing for existing users who are familiar with the current 
> section.
> 2. It'll make it more difficult or impossible to backport documentation 
> enhancements to the stable version of the docs (assuming we don't also 
> reorganize them with same structure)
> 3. It'll create an opportunity for broken links (obviously we could 
> mitigate this to some extent by adding redirects to the new locations).
>
> It seems to me you were pretty close to finding what you were looking for 
> at https://docs.djangoproject.com/en/1.9/ref/forms/ (first bullet, I 
> think), but I didn't understand what you meant by the page being "the Joy 
> of Cooking with Django."
>
> [1] https://www.djangoproject.com/weblog/2015/may/07/community-survey/
>
> On Sunday, December 27, 2015 at 2:47:35 PM UTC-5, Doug Epling wrote:
>>
>> Again, I am sorry if my comments have ruffled anyone's feathers.  I am 
>> not going to argue.  My sole intent is a positive one.  And, indeed, I am 
>> humbled by the ongoing work of this community over a period of time that I, 
>> until now, have not been involved.
>>
>> I beleive, it is my impression, that between Django 1.1 and now, on the 
>> verge of its second major version, there has been a tremendous amount of 
>> Python software develpment.  And the internal commenting as well as the 
>> public documentation has trailed along ad hoc.
>>
>> It can be said without legitimate reproach that any system whether it is 
>> thermodynamics or a system of communication, such as our documentation, 
>> will naturally tend toward entropy unless something actively intervenes. 
>>  And we have developed a fairly complex system compared to, say, werksgeud. 
>>
>> That patchwork approach has disrupted a flow of utility for users in both 
>> public documentation and internal commenting.  If this is true, Django has 
>> strayed from principles of its foundation.  And our motto: "The framework 
>> for perfectionists with deadlines."; holds true only until fininding 
>> oneself lost in the documentation.
>>
>> Tim is exactly right; this is with no doubt a non-trivial issue.  Is 
>> Django capable of tackling non-trivial issues?  If not I am in the wrong 
>> place (a challenge to Django, relax, it's not personal) because I believe 
>> Django should be setting the standard.  And this issue will not be resolved 
>> by an ad hoc approach; meaning our traditional methodology of a problem 
>> ticket reporting process is not amenable.  This calls for something else if 
>> it calls for anything.
>>
>> However, Wim has a good idea!  Some exploratory research is a very 
>> reasonable first step toward an o

Re: structural & functional review of django documentation

[Tim Graham]

There are some automatically added "[source]" links (see [1] for an 
example), however, other links don't work (e.g. [2]) because we use the 
convenience import path in the documentation. It would be a good task to 
see if this could be fixed in Sphinx.

There is some room to add some "contributor facing" documentation to 
"internals/" to describe the design of Django's components (e.g. an 
accepted ticket for doing so for migrations [3]).

By the way, here's Jacob's series of blog posts that describes the 
structure we have: https://jacobian.org/writing/great-documentation/

[1] 
https://docs.djangoproject.com/en/dev/ref/exceptions/#django.core.exceptions.ObjectDoesNotExist
[2] 
https://docs.djangoproject.com/en/dev/ref/applications/#django.apps.AppConfig
[3] https://code.djangoproject.com/ticket/24989

On Monday, December 28, 2015 at 12:40:13 PM UTC-5, Samuel Bishop wrote:
>
> Broadly speaking, I think the 'optimal' goal is not going to be one that 
> changes our existing documentation structure in any disruptive way.
>  
> I think the general concept would be covered by either creating a "fourth 
> division". So we would go from "topics", "reference", and "how-to", to 
> "topics", "reference", "how-to", and "implementation"/"internals"/"APIs"/etc
> Or enhancing the content in the reference section so that in addition to 
> our existing handwritten documentation, we expose the 'api-docs' tree as an 
> addendum to the reference section.
>
> Sphinx gives us a lot of power to work with for trying either approach. I 
> cant remember which project I saw it on, but I recall seeing some sphinx 
> powered projects that had Django style hand written documentation, and each 
> heading (module, class, function, etc) in their hand written documentation 
> had an automatically generated link to the matcihng "implementation docs" 
> that took you to that segment of the sphinx autodoc output. It worked quite 
> well from what I recall. 
> I'm pretty busy for a few days, but if no one else does this first and 
> shares some results, I'll definitely take a look at the output from 
> sphinx's autodoc tools when run over the Django repo and post some 
> findings. 
> Since there is certainly the big question of "is the information we have 
> useful"? 
> The docstrings and sundry that sphinx uses for autodoc generation may need 
> work before it measures up to the quality standards of our existing hand 
> written documentation. 
>
> On Monday, 28 December 2015 06:42:49 UTC+8, Tim Graham wrote:
>>
>> Adding a survey link is not difficult. We conducted a community survey 
>> [1] earlier this year with one question related to documentation, "What 
>> parts of the Django documentation do you find most useful?" What questions 
>> to ask and how to turn the answers into actionable items is the part I'm 
>> not sure about and maybe you could advise on.
>>
>> In my view, Django's docs haven't strayed from the "topics", "reference", 
>> and "how-to" division that we've had since 1.0 or so. Are you aware of this 
>> grouping? Maybe a "how the docs are organized" section on the index page 
>> would help communicate that and make it more intuitive where to look for 
>> something?
>>
>> I'll admit I'm skeptical of a wholesale reorganization to this structure 
>> for several reasons:
>> 1. It'll be confusing for existing users who are familiar with the 
>> current section.
>> 2. It'll make it more difficult or impossible to backport documentation 
>> enhancements to the stable version of the docs (assuming we don't also 
>> reorganize them with same structure)
>> 3. It'll create an opportunity for broken links (obviously we could 
>> mitigate this to some extent by adding redirects to the new locations).
>>
>> It seems to me you were pretty close to finding what you were looking for 
>> at https://docs.djangoproject.com/en/1.9/ref/forms/ (first bullet, I 
>> think), but I didn't understand what you meant by the page being "the Joy 
>> of Cooking with Django."
>>
>> [1] https://www.djangoproject.com/weblog/2015/may/07/community-survey/
>>
>> On Sunday, December 27, 2015 at 2:47:35 PM UTC-5, Doug Epling wrote:
>>>
>>> Again, I am sorry if my comments have ruffled anyone's feathers.  I am 
>>> not going to argue.  My sole intent is a positive one.  And, indeed, I am 
>>> humbled by the ongoing work of this community over a period of time that I, 
>>> until now, have not been involved.
>>>
>>> I beleive, it is my impression, that between Django 1.1 and now, on the 
>>> verge of its second major version, there has been a tremendous amount of 
>>> Python software develpment.  And the internal commenting as well as the 
>>> public documentation has trailed along ad hoc.
>>>
>>> It can be said without legitimate reproach that any system whether it is 
>>> thermodynamics or a system of communication, such as our documentation, 
>>> will naturally tend toward entropy unless something actively intervenes. 
>>>  And we have developed a fairly complex

Re: Help needed with the MySQL max index length problem for Django 1.10

[Collin Anderson]

Hi All,

I finally looked at this more today. I started working on the INDEX
(col1(191)) solution from #18392, but unfortunately I don't think we can
use that solution in this case because it's a UNIQUE index. (I still think
it's best solution for non-unique indexes.)

I think these are our options (in my humble order of preference), none of
them ideal:
1. Make username max_length=191 or some other nice number less 191.
2. Revert completely back to username max_length=30
3. Tell mysql folks they can only use the insecure[1] version of utf8
unless do some really complex reconfiguration (see "Hardest" solution in
the article [2]) to get utfmb4 support. Based on the fact that they're
using mysql and not PostgreSQL in the first place, many mysql users are
probably are unable to make the needed changes.

The good news is that mysql 5.7.7 (which, will _start_ to get a lot of real
use in 16.04) changes some defaults[3] to make the "hardest" solution
easier (just need to set ROW_FORMAT=DYNAMIC). MariaDB hasn't changed the
default [4]. I hope they do before RHEL/CentOS 8. Even then it's a long
wait before most people have it.

Collin

[1] https://youtu.be/qFfjJ8pOrWY?t=2534
[2] https://serversforhackers.com/mysql-utf8-and-indexing
[3] http://bugs.mysql.com/bug.php?id=68453#c430229
[4]
https://mariadb.com/kb/en/mariadb/xtradbinnodb-server-system-variables/#innodb_large_prefix



On Mon, Dec 21, 2015 at 11:32 AM, Tim Graham  wrote:

> I merged the often requested increase of User.username max_length to 254
> characters [1] a few weeks ago, however, the ticket was reopened pointing
> out this issue:
>
>
> "This patch breaks on MySQL installations using the utf8mb4 charset,
> because it breaks the index length limit - it comes out at a maximum of 254
> * 4 = 1016 bytes whilst by default InnoDB can only index 767 bytes. I found
> this because I am using utf8mb4 in django-mysql's tests.
>
> Django encourages using the utf8 charset (3 bytes per character - cannot
> store emojis), although there has been some discussion for moving to
> utf8mb4 in #18392 . One can
> index 254 character utf8mb4 fields in MySQL by using a couple settings as
> described in that ticket, Django could enforce those, or the field could be
> changed to just 191 characters instead which is the maximum indexable (767
> // 4)."
>
> Do we have any MySQL enthusiasts willing to champion a patch (or at least
> a decision design about the best way to proceed) for #18392 [2] to resolve
> this?
>
> [1] https://code.djangoproject.com/ticket/20846
> [2] https://code.djangoproject.com/ticket/18392
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ceb190b5-5218-446c-a6a0-0aea75e7fd6b%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFO84S4kBMPkr149Qs7LFNMv4awaApMiNiyZs9NzixRALOsCwQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Help needed with the MySQL max index length problem for Django 1.10

[Tim Graham]

Ugh, I guess I'm in favor of max_length=191. It'll just be awkward to 
explain that one in the docs.

On Monday, December 28, 2015 at 3:27:23 PM UTC-5, Collin Anderson wrote:
>
> Hi All,
>
> I finally looked at this more today. I started working on the INDEX 
> (col1(191)) solution from #18392, but unfortunately I don't think we can 
> use that solution in this case because it's a UNIQUE index. (I still think 
> it's best solution for non-unique indexes.)
>
> I think these are our options (in my humble order of preference), none of 
> them ideal:
> 1. Make username max_length=191 or some other nice number less 191.
> 2. Revert completely back to username max_length=30
> 3. Tell mysql folks they can only use the insecure[1] version of utf8 
> unless do some really complex reconfiguration (see "Hardest" solution in 
> the article [2]) to get utfmb4 support. Based on the fact that they're 
> using mysql and not PostgreSQL in the first place, many mysql users are 
> probably are unable to make the needed changes.
>
> The good news is that mysql 5.7.7 (which, will _start_ to get a lot of 
> real use in 16.04) changes some defaults[3] to make the "hardest" solution 
> easier (just need to set ROW_FORMAT=DYNAMIC). MariaDB hasn't changed the 
> default [4]. I hope they do before RHEL/CentOS 8. Even then it's a long 
> wait before most people have it.
>
> Collin
>
> [1] https://youtu.be/qFfjJ8pOrWY?t=2534
> [2] https://serversforhackers.com/mysql-utf8-and-indexing
> [3] http://bugs.mysql.com/bug.php?id=68453#c430229
> [4] 
> https://mariadb.com/kb/en/mariadb/xtradbinnodb-server-system-variables/#innodb_large_prefix
>
>
>
> On Mon, Dec 21, 2015 at 11:32 AM, Tim Graham  > wrote:
>
>> I merged the often requested increase of User.username max_length to 254 
>> characters [1] a few weeks ago, however, the ticket was reopened pointing 
>> out this issue:
>>
>>
>> "This patch breaks on MySQL installations using the utf8mb4 charset, 
>> because it breaks the index length limit - it comes out at a maximum of 254 
>> * 4 = 1016 bytes whilst by default InnoDB can only index 767 bytes. I found 
>> this because I am using utf8mb4 in django-mysql's tests.
>>
>> Django encourages using the utf8 charset (3 bytes per character - cannot 
>> store emojis), although there has been some discussion for moving to 
>> utf8mb4 in #18392 . One can 
>> index 254 character utf8mb4 fields in MySQL by using a couple settings as 
>> described in that ticket, Django could enforce those, or the field could be 
>> changed to just 191 characters instead which is the maximum indexable (767 
>> // 4)."
>>
>> Do we have any MySQL enthusiasts willing to champion a patch (or at least 
>> a decision design about the best way to proceed) for #18392 [2] to resolve 
>> this? 
>>
>> [1] https://code.djangoproject.com/ticket/20846
>> [2] https://code.djangoproject.com/ticket/18392
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/ceb190b5-5218-446c-a6a0-0aea75e7fd6b%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/78e114b4-9595-484a-bc33-f87635a2e7c0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: delegating our static file serving

[Tim Graham]

I'd like to work together with Dave to develop a proof of concept that 
integrates whitenoise into Django. I spent about an hour looking through 
whitenoise and our own static file serving code, and I think integrating 
whitenoise will yield a simpler user experience with about the same amount 
of code as have now.

Essentially, we'd recommend adding something like this to existing wsgi.py 
files (it would be in the default startproject template)

from whitenoise.django import DjangoWhiteNoise
application = DjangoWhiteNoise(application)
application.add_files(settings.MEDIA_ROOT, prefix=settings.MEDIA_URL)

which would have the benefit of working out of the box in production too, I 
think. Of course, you could disable that based on settings.DEBUG or some 
other toggle.

We could then deprecate:

* django/contrib/staticfiles/views.py
* django/contrib/staticfiles/management/commands/runserver.py
* django/contrib/staticfiles/handlers.py
* django/views/static.py

Any objections to doing further investigation in this area?

On Saturday, June 20, 2015 at 8:09:11 AM UTC-4, David Evans wrote:
>
> On Friday, 5 December 2014 19:14:29 UTC, Carl Meyer wrote:
>>
>> On 12/04/2014 10:33 PM, Collin Anderson wrote: 
>> > Hi All, 
>> > 
>> > I'm pretty interested in getting secure and _somewhat_ efficient static 
>> > file serving in Django. 
>> > 
>> > Quick history: 
>> > 2005 - Jacob commits #428: a "static pages" view.  Note that this view 
>> > should only be used for testing!" 
>> > 2010 - Jannis adds staticfiles. Serving via django is considered 
>> "grossly 
>> > inefficient and probably insecure". 
>> > 2011 - Graham Dumpleton adds wsgi.file_wrapper to Gunicorn. 
>> > 2012 - Aymeric adds StreamingHttpResponse and now files are read in 
>> chunks 
>> > rather than reading the entire file into memory. (No longer grossly 
>> > inefficient IMHO.) 
>> > 
>> > I propose: 
>> > - Deprecate the "show_indexes" parameter of static.serve() (unless 
>> people 
>> > actually use it). 
>> > - Have people report security issues to secu...@djangoproject.com 
>> (like 
>> > always) 
>> > - Audit the code and possibly add more security checks and tests. 
>> > - add wsgi.file_wrapper support to responses (5-line proof of concept: 
>> > https://github.com/django/django/pull/3650 ) 
>> > - support serving static files in production, but still recommend 
>> > nginx/apache or a cdn for performance. 
>> > - make serving static files in production an opt-in, but put the view 
>> in 
>> > project_template/project_name/urls.py 
>> > 
>> > I think it's a huge win for low-traffic sites or sites in the "just 
>> trying 
>> > to deploy and get something live" phase. You can always optimize later 
>> by 
>> > serving via nginx or cdn. 
>> > We already have the views, api, and logic around for finding and 
>> serving 
>> > the correct files. 
>> > We can be just as efficient and secure as static/dj-static without 
>> needing 
>> > to make people install and configure wsgi middleware to the 
>> application. 
>> > We could have staticfiles classes implement more complicated features 
>> like 
>> > giving cache recommendations, and serving pre-gzipped files. 
>> > 
>> > Is this a good idea? I realize it's not totally thought through. I'm 
>> fine 
>> > with waiting until 1.9 if needed. 
>>
>> I also think this is a good plan. It certainly makes sense to look at 
>> "static" and "whitenoise" for ideas and compare their code to ours to 
>> see where we could be more efficient or secure, but it's much less churn 
>> for Django users if we simply improve our existing code rather than pull 
>> in something wholly new. 
>>
>> Carl 
>>
>>
>>  
> Sorry to revive an old thread here, but I just wanted to add that v2.0 of 
> WhiteNoise now supports serving development files, providing the same 
> behaviour as runserver currently does in DEBUG mode. (There were enough 
> people wanting to do their development using gunicorn rather than runserver 
> to make this worthwhile.)
>
> This means that WhiteNoise is now a one-stop-shop for static file handling 
> in Django. If there's still an appetite for integrating it, or something 
> equivalent, into core I'd be happy to help out.
>
> Dave
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a2136cbb-2dfb-4269-8bb9-64a55f8c7b6a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: structural & functional review of django documentation

[Daniele Procida]

On Mon, Dec 28, 2015, Samuel Bishop  wrote:

>I think the general concept would be covered by either creating a "fourth 
>division". So we would go from "topics", "reference", and "how-to", to 
>"topics", "reference", "how-to", and "implementation"/"internals"/"APIs"/etc
>Or enhancing the content in the reference section so that in addition to 
>our existing handwritten documentation, we expose the 'api-docs' tree as an 
>addendum to the reference section.

The main existing sections are:

* tutorials (/intro)

Tutorials take the new user by the hand through a series of steps. The 
important thing isn't to explain all the steps, but to achieve something useful 
with a minimum of effort. 

After every step of a tutorial, the user should have something that works, even 
if they barely understand what is happening (and it's not necessary for them to 
understand, that can come later. What matters is that they are successful).

* how-to guides (/howto)

How-to guides are recipes that take the user through steps in key subjects. 
They are more advanced than tutorials and assume a lot more about what the user 
already knows than tutorials do, and unlike documents in the tutorial they can 
stand alone.  

* discussion and explanation (/topic)

Aimed at explaining (at a fairly high level) rather than doing.

* reference (/ref)

Technical reference for APIs, key models and so on. It doesn't need to explain 
so much as describe and instruct.


I don't know who came up with this structure in Django, but whoever did got it 
absolutely right.

The structure doesn't confuse teaching with explaining, and understands why 
tutorials should be concerned with concrete and particular rather than abstract 
and general matters. It understands the difference between explaning how to 
achieve something and explaining how something works. And so on.

I think that the structure of the documentation as much as its content is what 
makes it so good, but also that it's structure is obvious - so I am surprised 
that not everyone finds it so obvious. If that's the case then I agree it 
should be made more explicit.

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20151229005216.254922486%40mail.wservices.ch.
For more options, visit https://groups.google.com/d/optout.

Re: structural & functional review of django documentation

[Eric Holscher]

On Monday, December 28, 2015 at 4:52:36 PM UTC-8, Daniele Procida wrote:
>
> On Mon, Dec 28, 2015, Samuel Bishop > 
> wrote:
> The main existing sections are: 
>
> * tutorials (/intro) 
>
> Tutorials take the new user by the hand through a series of steps. The 
> important thing isn't to explain all the steps, but to achieve something 
> useful with a minimum of effort. 
>
> After every step of a tutorial, the user should have something that works, 
> even if they barely understand what is happening (and it's not necessary 
> for them to understand, that can come later. What matters is that they are 
> successful). 
>
> * how-to guides (/howto) 
>
> How-to guides are recipes that take the user through steps in key 
> subjects. They are more advanced than tutorials and assume a lot more about 
> what the user already knows than tutorials do, and unlike documents in the 
> tutorial they can stand alone.   
>
> * discussion and explanation (/topic) 
>
> Aimed at explaining (at a fairly high level) rather than doing. 
>
> * reference (/ref) 
>
> Technical reference for APIs, key models and so on. It doesn't need to 
> explain so much as describe and instruct. 
>
>  
I think the above post does a good job of describing the layout, and 
something similar should be included in the docs. Without having read 
Jacob's posts on the subject, there is nothing in the official docs that 
gives the reader an understanding that this is how things are laid out, as 
far as I know.

I think the underlying structure makes sense, and it seems that mostly 
people are just upset about the lack of a pure auto-generated code 
reference. I believe historically that this has been excluded explicitly, 
not because of lack of technology. There is no use of autodoc in the Django 
tree, even where it might make sense.

At Django Under the Hood this year, I had a few conversations with folks 
about rethinking and explicitly defining these policies. I think it makes a 
lot of sense to write down the logic and structure behind these decisions 
in a DEP, and explain the layout to doc users in a few places in the 
documentation explicitly.

Cheers,
Eric 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0daa98a8-6714-4f76-adb2-ea5ced43d361%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.