[Python-Dev] Bundling decimal.py with Django
Apologies if this is a little off-topic for python-dev, but it seemed like the best place to ask and get the attention of those needed. I am championing a patch to improve Django's support for numeric types by using Decimals with numeric columns and floats with double precision columns, rather than the current rather unpredictable behaviour (which depends on the particular python version and database backend being used). It has been proposed[1] to include decimal.py from the Python 2.5 distribution with Django, to ensure consistent and compatible behaviour between Python 2.3 and Python >= 2.4[2]. Jacob Kaplan-Moss wrote in response to this proposal """ In order for this to be accepted, we need to know: * What is the license for the decimal module, and is said license compatible with the BSD license that Django uses? * Are the authors of the decimal module OK with us distributing it with Django? Our policy for including third party libraries requires an answer to both those questions. We won't include anything if it's not legal to do so, and we won't include anything that the original author doesn't want included. """ I'm addressing the second of Jacob's points here; would it be acceptable if the decimal module was distributed with Django? Andrew [1] http://groups.google.com/group/django-developers/msg/230892a9f9a71472 [2] Python's _threading_local.py module is already included with Django for the same compatibility reasons. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
On 6/28/05, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > > AFAICT, the only unresolved issue outstanding is a compromise or > Pronouncement regarding the atime/ctime/mtime members' datatype. This is > assuming, of course, that making the "empty path" be os.curdir doesn't > receive any objections, and that nobody strongly prefers 'path.fromcwd()' > over 'path.cwd()' as the alternate constructor name. > > Apart from these fairly minor issues, there is a very short to-do list, > small enough to do an implementation patch in an evening or > two. Documentation might take a similar amount of time after that; mostly > it'll be copy-paste from the existing os.path docs, though. While we'ew discussing outstanding issues: In a related discussion of the path module on c.l.py, Thomas Heller pointed out that the path module doesn't correctly handle unicode paths: | I have never used the path module before, although I've heard good | things about it. But, it seems to have problems with unicode pathnames, | at least on windows: | | C:\>mkdir späm | | C:\späm>py24 | Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 | Type "help", "copyright", "credits" or "license" for more information. | >>> from path import path | >>> path.getcwd() | | Traceback (most recent call last): | File "", line 1, in ? | File "C:\TSS5\components\_Pythonlib\path.py", line 97, in getcwd | return path(os.getcwd()) | UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 5: ordinal not in range(128) He suggests a possible fix in his message: http://mail.python.org/pipermail/python-list/2005-June/287372.html http://groups-beta.google.com/group/comp.lang.python/msg/b3795a2a0c52b93f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Triple-quoted strings and indentation
In general, I find triple-quoted strings to be very handy, particularly for standalone scripts. However, the fact that they have to be written in the left-hand column to avoid leading whitespace really grates, particularly when they're nested within a block or two -- it's a wart: try: options, args = getopt.getopt(sys.argv[1:], "cf:s") except getopt.GetoptError: print """Usage: dostuff Options: -c - blah blah -f - do stuff with file "filename" -s - more blah""" sys.exit(1) This really makes the code hard to read, as the indentation is all mixed up (visually). I have written a patch that changes the way triple-quoted strings are scanned so that leading whitespace is ignored in much the same way that pep 257 handles it for docstrings. Largely this was for a learning experience in hacking the parser, but it would be very nice IMO if something of this sort could be implemented in a future version of Python. To this end, I have sketched out a draft PEP (which was itself a good learning exercise in thinking out the issues of such a change). Should I post it here for discussion? Andrew ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Triple-quoted strings and indentation
On 7/6/05, Terry Reedy <[EMAIL PROTECTED]> wrote:
>
> Doc strings, first meant for the code reader, need to be where they are.
> They also come before the code itself, so don't interfere.
Doc strings are really not an issue, due to the conventions for
processing whitespace in them (and also the fact that their primary
use is for the reader of the code, even before any automated
processing).
> > -- it's a wart:
>
> That is rather extreme, and is definitely an opinion.
My opinion, certainly. However, I think the fact that workarounds for
the leading whitespace issue are needed (PEP 257, textwrap.dedent("""\
)) points to it being more than that. But of
course that is also my opinion :-)
> And what if I want the leading whitespace left just the way I carefully put
> it?
You can still put leading whitespace as you want it -- there would
just be a slightly different convention to follow -- I'll post what I
wrote up so you can see the whole proposal: better than me repeating
it all in bits.
> And what of existing code dependent on literals being as literal as
> they currently are?
There would be some breakage, certainly -- though I believe it would
be quite little.
> I think the soonest this could be considered is Python
> 3.0.
Quite likely so.
Andrew
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Triple-quoted strings and indentation
Here's the draft PEP I wrote up: Abstract Triple-quoted string (TQS henceforth) literals in Python preserve the formatting of the literal string including newlines and whitespace. When a programmer desires no leading whitespace for the lines in a TQS, he must align all lines but the first in the first column, which differs from the syntactic indentation when a TQS occurs within an indented block. This PEP addresses this issue. Motivation TQS's are generally used in two distinct manners: as multiline text used by the program (typically command-line usage information displayed to the user) and as docstrings. Here's a hypothetical but fairly typical example of a TQS as a multiline string: if not interactive_mode: if not parse_command_line(): print """usage: UTIL [OPTION] [FILE]... try `util -h' for more information.""" sys.exit(1) Here the second line of the TQS begins in the first column, which at a glance appears to occur after the close of both "if" blocks. This results in a discrepancy between how the code is parsed and how the user initially sees it, forcing the user to jump the mental hurdle in realising that the call to sys.exit() is actually within the second "if" block. Docstrings on the other hand are usually indented to be more readable, which causes them to have extraneous leading whitespace on most lines. To counteract the problem, PEP 257 [1] specifies a standard algorithm for trimming this whitespace. In the end, the programmer is left with a dilemma: either to align the lines of his TQS to the first column, and sacrifice readability; or to indent it to be readable, but have to deal with unwanted whitespace. This PEP proposes that TQS's should have a certain amount of leading whitespace trimmed by the parser, thus avoiding the drawbacks of the current behaviour. Specification Leading whitespace in TQS's will be dealt with in a similar manner to that proposed in PEP 257: "... strip a uniform amount of indentation from the second and further lines of the [string], equal to the minimum indentation of all non-blank lines after the first line. Any indentation in the first line of the [string] (i.e., up to the first newline) is insignificant and removed. Relative indentation of later lines in the [string] is retained." Note that a line within the TQS that is entirely blank or consists only whitespace will not count toward the minimum indent, and will be retained as a blank line (possibly with some trailing whitespace). There are several significant differences between this proposal and PEP 257's docstring parsing algorithm: * This proposal considers all lines to end at the next newline in the source code (whether escaped or not); PEP 257's algorithm only considers lines to end at the next (necessarily unescaped) newline in the parsed string. * Only literal whitespace is counted; an escape such as \x20 will not be counted as indentation. * Tabs are not converted to spaces. * Blank lines at the beginning and end of the TQS will *not* be stripped. * Leading whitespace on the first line is preserved, as is trailing whitespace on all lines. Rationale I considered several different ways of determining the amount of whitespace to be stripped, including: 1. Determined by the column (after allowing for expanded tabs) of the triple-quote: myverylongvariablename = """\ This line is indented, But this line is not. Note the trailing newline: """ + Easily allows all lines to be indented. - Easily leads to problems due to re-alignment of all but first line when mixed tabs and spaces are used. - Forces programmers to use a particular level of indentation for continuing TQS's. - Unclear whether the lines should align with the triple- quote or immediately after it. - Not backward compatible with most non-docstrings. 2. Determined by the indent level of the second line of the string: myverylongvariablename = """\ This line is not indented (and has no leading newline), But this one is. Note the trailing newline: """ + Allows for flexible alignment of lines. + Mixed tabs and spaces should be fine (as long as they're consisten
Re: [Python-Dev] Triple-quoted strings and indentation
On 7/6/05, Nick Coghlan <[EMAIL PROTECTED]> wrote: > > However, while I prefer what you describe to Python's current > behaviour, I am not yet convinced the backward compatibility pain is > worth it. Adding yet-another-kind-of-string-literal (when we already > have bytestrings on the horizon) is also unappealing. First off, thanks for your feedback--you raise some very good points that I have addressed insufficiently or not at all. I personally feel that backward compatibility issues provide the strongest argument against accepting the proposal (but obviously I find the rest of it favourable :-). It may not be particularly clear (that's why it's a draft) that I am not proposing another kind of string literal, but rather a change of rules for an existing one. > Your readability examples are good, but the first two involve strings > that should probably be module level constants (as Terry described) > and the test case involving expected output is probably better handled > using doctest, which has its own mechanism for handling indentation. I think the question of whether an inline string should be made a module-level constant (or moved to a separate file entirely) relates in general to higher-level considerations of readability and program structure (similar, for example, to those that would prompt one to refactor a function). IOW, the answer to that question would be the same with or without this proposal. In any case, I chose the examples (from different parts of the standard library) more because they illustrated different classes of usage for TQS's than because they were the best illustrations of readability improvement--perhaps something I should address. > Raw and unicode string literals need to be mentioned in the PEP. Which > literals would the reformatting apply to? All 3? Only standard and > unicode, leaving raw strings alone? The proposal would apply to all 4 :-) -- normal, raw, unicode, and raw unicode. > You should research the reasons why PEP 295 [1] was rejected, and > describe in the new PEP how it differs from PEP 295 (unfortunately PEP > 295 was not updated with the rationale for rejection, but this post > [2] looks like Guido putting the final nail in its coffin). THANK YOU! In my research for this, I didn't come across PEP 295 at all -- perhaps due to the fact that it uses the term "multiline strings" exclusively, which is not how I would describe them at all. I will certainly address this in my next draft. Cheers, Andrew. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Triple-quoted strings and indentation
On 7/7/05, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > I don't think so. It smells too much of DWIM, which is very unpythonic. EIBTI. In what way? The scheme described is explicit, and consistently applied to all triple-quoted strings[*] -- although the rules are different to the current behaviour. On the other hand, my written proposal may not be clear or explicit, something I will attempt to remedy over the next few days. [*] Whether it should apply also to normal strings with escaped newlines is not something I have yet considered. Andrew ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Triple-quoted strings and indentation
On 7/7/05, Terry Reedy <[EMAIL PROTECTED]> wrote: > > I believe there were some current alternatives and concerns already > expressed that have not been included yet that maybe should be. Yes; Nick pointed me to one, and I'll be looking at that and the related discussions before redrafting; I'll also have a further look for other similar proposals. > Some of your examples look worse than needed by putting the first line > after the triple quote instead of escaping the first newline like you did > elsewhere. In general, I wanted to preserve as much as possible the way that the string was originally written (as these examples were taken and adapted from the standard library source). In the example with the embedded python code, I felt it was significantly clearer if the initial newline was escaped. > Having separate rules for doc strings and other tq strings would be a > nuisance. I totally agree -- and if the proposal as written gives that impression then I'll correct it. What I was trying to say about docstrings was that the change would have no effect on the result after processing them with docutils or anything else that follows PEP 257 -- which is very significant in terms of backward compatibility, as docstrings are AFAICT the leading use of TQS's (by a large margin). Cheers, Andrew. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Triple-quoted strings and indentation
On 7/11/05, Josiah Carlson <[EMAIL PROTECTED]> wrote: > > You are wrong. Current string literals are explicit. They are what you > type. No they are not: >>> "I typed \x41, but got this!" 'I typed A, but got this!' What we have are not explicit string literals but *explicit rules*, forming part of the language definition and given in the documentation that certain sequences in string literals are preprocessed by the compiler. People have learnt these rules and apply them unconsciously when reading the source--but that can apply to any rule. For example, there's another explicit rule, that an "r" prefixed before the string literal uses a different set of rules without \-escape sequences: >>> r"I typed \x41, but got this!" 'I typed \\x41, but got this!' The point is that processing \-escape sequences is just as implicit as my proposal: but tradition, and custom, and documentation make it *seem* explicit of itself. IOW, arguing that my proposal is "implicit" or "DWIM" is neither relevant nor valid--but arguing either that it is confusing to a long-term Python user, or that it is not fully backward compatible is valid, and these (and other) arguments against should be weighed up against those in favour. (Even some fairly recent and major changes to Python have been accepted despite having these two particular arguments against them, such as unified classes/types and nested scopes). > When you have to start differentiating, or consider differentiating, how > preprocessing occurs based on the existance or non-existance of escaped > newlines, you should realize that this has a serious "Do what I mean" > stink (as Guido has already stated, more politely). What I am considering differentiating on here is a feature of Python that is (at least) awkward and (at most) has a "serious stink" -- the ability to escape newlines in a single-quoted [' or "] string with a backslash, which has inconsistent or confusing behaviour: >>> "This is a normal string\ ... with an escaped newline." 'This is a normal stringwith an escaped newline.' >>> r"This is a raw string\ ... with an escaped newline." 'This is a raw string\\\nwith an escaped newline.' This is not an issue with TQS's because they can naturally (i.e. without escapes) span multiple lines. Since your main objections above are much the same as Guido's, I'll respond to his in this message also: On 7/11/05, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > The scheme may be explicitly spelled out in the language reference, > but it is implicit to the human reader -- what you see is no longer > what you get. See discussion of explicitness vs. implicitness above. > I recommend that you give it up. You ain't gonna convince me. Very likely. But given the number of times that similar proposals have been put forth in the past, it is reasonable to expect that they will be brought up again in the future by others, if this is rejected--and in that case, these other can simply be pointed to a thorough (but rejected) PEP that discusses the proposal and variants and reasons for rejection. And so--while I still hope that you can be convinced (there's precedent ;-), I think a good, thorough PEP will be of benefit even if rejected. And, of course, such a PEP is bound to be more convincing than a hasty ill-considered one. So I am rewriting my previous draft accordingly, and will submit it as a PEP when it's done. Cheers, Andrew. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove str.find in 3.0?
On 8/31/05, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> [Hye-Shik Chang]
> > What would be a result for rpartition(s, '?') ?
> > ('', '', 'http://www.python.org')
> > or
> > ('http://www.python.org', '', '')
>
> The former. The invariants for rpartition() are a mirror image of those
> for partition().
Just to put my spoke in the wheel, I find the difference in the
ordering of return values for partition() and rpartition() confusing:
head, sep, remainder = partition(s)
remainder, sep, head = rpartition(s)
My first expectation for rpartition() was that it would return exactly
the same values as partition(), but just work from the end of the
string.
IOW, I expected "www.python.org".partition("python") to return exactly
the same as "www.python.org".rpartition("python")
To try out partition(), I wrote a quick version of split() using
partition, and using partition() was obvious and easy:
def mysplit(s, sep):
l = []
while s:
part, _, s = s.partition(sep)
l.append(part)
return l
I tripped up when trying to make an rsplit() (I'm using Python 2.3),
because the return values were in "reverse order"; I had expected the
only change to be using rpartition() instead of partition().
For a second example: one of the "fixed stdlib" examples that Raymond
posted actually uses rpartition and partition in two consecutive lines
-- I found this example not immediately obvious for the above reason:
def run_cgi(self):
"""Execute a CGI script."""
dir, rest = self.cgi_info
rest, _, query = rest.rpartition('?')
script, _, rest = rest.partition('/')
scriptname = dir + '/' + script
scriptfile = self.translate_path(scriptname)
if not os.path.exists(scriptfile):
Anyway, I'm definitely +1 on partition(), but -1 on rpartition()
returning in "reverse order".
Andrew
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove str.find in 3.0?
On 8/31/05, Delaney, Timothy (Tim) <[EMAIL PROTECTED]> wrote:
> Andrew Durdin wrote:
>
> > Just to put my spoke in the wheel, I find the difference in the
> > ordering of return values for partition() and rpartition() confusing:
> >
> > head, sep, remainder = partition(s)
> > remainder, sep, head = rpartition(s)
>
> This is the confusion - you've got the terminology wrong.
>
> before, sep, after = s.partition('?')
> ('http://www.python.org', '', '')
>
> before, sep, after = s.rpartition('?')
> ('', '', 'http://www.python.org')
That's still confusing (to me), though -- when the string is being
processed, what comes before the separator is the stuff at the end of
the string, and what comes after is the bit at the beginning of the
string. It's not the terminology that's confusing me, though I find
it hard to describe exactly what is. Maybe it's just me -- does anyone
else have the same confusion?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove str.find in 3.0?
On 8/31/05, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > Hm. The example is poorly chosen because it's an end case. The > invariant for both is (I'd hope!) > > "".join(s.partition()) == s == "".join(s.rpartition()) > (Just think of it as rpartition() stopping at the last occurrence, > rather than searching from the right. :-) Ah, that makes a difference. I could see that there was a different way of looking at the function, I just couldn't see what it was... Now I understand the way it's been done. Cheers, Andrew. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simplify the file-like-object interface
On 9/6/05, Antoine Pitrou <[EMAIL PROTECTED]> wrote:
>
> One could use "class decorators". For example if you want to define the
> method foo() in a file-like class, you could use code like:
I like the sound of this. Suppose there were a function textstream()
that decorated a file-like object (supporting read() and write()), so
as to add all of __iter__(), next(), readline(), readlines(), and
writeline() that it did not already implement. Then you could wrap any
file-like object easily to give it convenient text-handling:
mytextsocketstream = textstream(mysocketstream)
for line in mytextsocketstream:
print line
Another area where I think this approach can help is with the
text/binary file distinction. file() could always open files as
binary, and there could be a convenience function textfile(name, mode)
which would simply return textstream(file(name, mode)). This would
remove the need for "a" or "b" in the mode parameter, and make it
easier to keep text- and binary-file access separate in one's mind:
tf = textfile("log.txt", "w")
tf.writelines(loglist)
bf = file("img.jpg", "r")
process_image_file(bf)
Finally, I think this could nicely tie into the print
statement/function/method debate, as the print() method could be
defined by the textstream() wrapper, allowing any text stream to
support the convenient print() behaviour, but not requiring it for
binary streams, for which such a function makes little sense anyway.
And then textstream() could be used (in the sys module, preferably) to
ensure that sys.stdout and sys.stderr both support print(). So you'd
have the builtin:
# Actual signature depending on which variant proposal is taken up
def print(*args, **kwargs):
sys.stdout.print(*args, **kwargs)
And using print() could be a simple as:
print("Hello, World")
sys.stdout.print("This is normal output")
sys.stderr.print("This is an error message")
I have a vague idea that a wrapper like this (or another similar
wrapper) could provide for convenient, transparent handling of Unicode
text files also, but I don't know Unicode well enough to be certain.
Andrew.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Replacement for print in Python 3.0
On 9/6/05, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > My hypothesis is that there are actually only two use cases that > matter enough to be supported directly: > > (a) quickly print a bunch of items with spaces in between them and a > trailing newline > > (b) print one or more items with precise control over each character Doesn't the write() method of file-like objects already cover (b), except that it only takes a single argument? If the need to print multiple arguments without any separator is so common, could the write() method be extended to take multiple arguments and just write them all out? It'd certainly be backward compatible with old code that called the write() method... Andrew. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C coding experiment
On 9/1/05, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > The goal is to determine whether the setobject.c implementation would be > improved by recoding the set_lookkey() function to optimize key > insertion order using Brent's variation of Algorithm D (See Knuth vol. > III, section 6.4, page 525). Brent's variation depends on the next probe position for a key being derivable just from the key and its current position. The use of perturbation in set_lookkey() prevents that, as we cannot say, given a key at a certain position, where the next probe location for that key would have been, without doing a complete lookup of that key (obviously too expensive). It would be interesting to see whether Brent's variation or perturbation give better expected overall performance -- the latter may well prove better in most cases, as it should reduce the number of probes needed for both successful and unsuccessful lookups, while Brent's variation only speeds up successful lookups. Well, this sort of question is what the experiment is meant to resolve, no? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
