Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
Greg Ewing wrote: > M.-A. Lemburg wrote: > >> Believe me: you have to if you want to do more >> advanced calculus related to pricing and risk >> analysis of derivatives. > > When you do things like that, you're treating > money as though it were a continuous quantity. > This is an approximation, so you can tolerate > having small inaccuracies in the result. > > But when adding up actual, real amounts of > money, where the result must be exact, using > binary fractions is a very bad idea. Agreed. Those are different use cases, though. >> This is not the same: if you round both value >> and then compare, you test a different interval >> than by taking the difference and applying >> a tolerance value comparison: > > That's exactly my point. Chopping decimal > places is not a substitute for doing tolerance > testing properly. As I said: those two are different ways of doing comparisons. If you are eventually rounding to say 2 decimal places in the end of the calculation, you won't want to find yourself presenting the user 1.12 and 1.13 as equal values :-) >> Most typical uses of round() don't use the >> optional argument, true, but I still fail >> to see what returning an integer instead of >> a float would buy you. > > It saves you a function call in the vast > majority of cases, where an int is what > you ultimately want. -1 on that. Just saving a single function call isn't really enough to break all sorts of applications out there that use round() in calculations (with or without argument). Most such calculations do work with floats, so having round() return an int would just add another coercion to a float for those use-cases. Seriously, rounding is a complicated business. round() just implements one way of doing it. Perhaps a module providing different rounding models would be of use ?! The decimal module already has a few of those implemented, so it could benefit from such a module as well. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
Greg Ewing wrote: > Raymond Hettinger wrote: > >> -1 on an extra built-in just to save the time for function call > > The time isn't the main issue. The main issue > is that almost all the use cases for round() > involve doing an int() on it afterwards. At > least nobody has put forward an argument to > the contrary yet. I'll try to give a valid argument of the contrary. I think you are only considering part of the usefulness of round(). In statistics and science math equations, rounding isn't used to get an integer, but instead used to give an answer that is meaningful within the margin of error of the data being used. Consider an example where you are combining data that had different number of significant digits. Keeping all the digits of your answer gives a false since of accuracy. The extra digits are meaningless because the margin of error is greater than any of the digits that exceed the minimum number of significant digits in your data. So you need to remove the extraneous digits by rounding. Then this answer can be further used as data, and sense the number of significant digits is maintained, the margin of error can be calculated accurately. [ NOTE: While floating point may contribute a significant error to value that are very large or very small, in most cases it is a very small amount in relation to the precision of the calculation and so the error can be managed without too much trouble. In cases where it is a significant factor, another internal representation should probably be considered. I only mention it here because someone would point it out (again) anyway. The accuracy of floating point is not the issue being discussed in this thread. ] It makes since for round have an argument to specify the number of digits of precision to round to and also for it to return floating point numbers because rounding results to a float of n digits of precision is a necessary operation. If you have the default round() case of 0 (and negative) digits of precision return integers, you then have a function that may sometimes returns integers, and sometimes returns floats. This can be problematic if the values are further used in division operations. Which will result in many cases of.. float(round(x)) in order to convert integer results back to floats. See pep-0238: Changing the division operator. http://www.python.org/dev/peps/pep-0238/ While it is true we often will use round along with converting to an integer, it is also true that functions that are predictable and return a single type are easier to use and learn. So the question of which is better is a matter of point of view in this respect. And in regard to using round() combined with division operators in floating point math, it is an issue of least surprises. > Sure you can define your own function to do > this. But that's still a considerable burden > when you add up the effort over all the > programs that use int(round()). I think the only time it would be a burden is if a single program uses int(round()) many times, otherwise it is only a very small additional amount to type for each program. In the case of single program using it many times, a helper function is a very practical solution. Cheers, Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly ...
James Y Knight <[EMAIL PROTECTED]> wrote: > > I'd be happy to see floats lose their __int__ method entirely, > replaced by an explicit truncate function. Come back Algol - all is forgiven :-) Yes, indeed. I have favoured that view for 35 years - anything that can lose information quietly should be explicit. [EMAIL PROTECTED] (Christian Tanzer) wrote: > Greg Ewing <[EMAIL PROTECTED]> wrote: > > > What's the feeling about this? If, e.g. int() > > were changed in Py3k to round instead of truncate, > > would it cause anyone substantial pain? > > Gratuitous breakage! > > I shudder at the thought of checking hundreds of int-calls to see if > they'd still be correct under such a change. My experience of doing that when compilers sometimes did one and sometimes the other is that such breakages are rarer than the conversions to integer that are broken with both rules! And both are rarer than the code that works with either rule. However, a 5% breakage rate is still enough to be of concern. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly ...
Nick Maclaren <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] (Christian Tanzer) wrote: > > Greg Ewing <[EMAIL PROTECTED]> wrote: > > > > > What's the feeling about this? If, e.g. int() > > > were changed in Py3k to round instead of truncate, > > > would it cause anyone substantial pain? > > > > Gratuitous breakage! > > > > I shudder at the thought of checking hundreds of int-calls to see if > > they'd still be correct under such a change. > > My experience of doing that when compilers sometimes did one and > sometimes the other is that such breakages are rarer than the > conversions to integer that are broken with both rules! And both are > rarer than the code that works with either rule. > > However, a 5% breakage rate is still enough to be of concern. I couldn't care less about how many calls would break -- I'd still need to look at each and every one. And I know that quite a number of calls need the truncation semantics (I just don't know which without looking). -- Christian Tanzerhttp://www.c-tanzer.at/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
Ron Adam wrote: > Consider an example where you are combining data that had different > number of significant digits. Keeping all the digits of your answer > gives a false since of accuracy. The extra digits are meaningless > because the margin of error is greater than any of the digits that > exceed the minimum number of significant digits in your data. So you > need to remove the extraneous digits by rounding. Then this answer can > be further used as data, and sense the number of significant digits is > maintained, the margin of error can be calculated accurately. This is a fallacy though - add two numbers together each with an error of +/- 0.05, and the error in the total is +/- 0.1. The approach you propose gives the misleading impression that the error in the total is still only +/- 0.05 (as the answer will contain 1 decimal place). If you really care about error, you need to do it properly (e.g. stddev or adding the error margins). Anyway, it's not proposed to remove the "round to x decimal places" capability entirely - merely remove it from the builtin round() so that the return type can be changed. > It makes since for round have an argument to specify the number of > digits of precision to round to and also for it to return floating point > numbers because rounding results to a float of n digits of precision is > a necessary operation. > > If you have the default round() case of 0 (and negative) digits of > precision return integers, you then have a function that may sometimes > returns integers, and sometimes returns floats. That's not the proposal, as Greg has already said (the fround() mentioned would probably be 'math.fround' rather than a builtin): [Greg Ewing] > No, round() wouldn't have that option at all. If > you wanted it, you would use fround() instead, > which would have the option and return a float > always. > > This would be a Py3k thing, obviously. If done > before then, the new function would have to be > given a different name. [Ron Adam] > This can be problematic > if the values are further used in division operations. Which will > result in many cases of.. float(round(x)) in order to convert integer > results back to floats. Not in Py3k where int/int will return a float as needed. > And in regard to using round() combined with division operators in > floating point math, it is an issue of least surprises. And in Py3k, with a round that always returned an integer there wouldn't be any surprises at all: - division would do the right thing, because true division becomes the only behaviour for '/' - the result of the builtin rounding operation would be usable directly as an index without requiring subsequent coercion to an integer Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
Hi all,
I've been trying to port our software to python 2.5.
unfortunately I'm getting constantly hit by segfaults.
I've boiled it down to the following code:
[EMAIL PROTECTED]:~/bug$ cat t.py
import array
class Indexer(object):
maximumForwardSize = property(array.array.fromstring)
[EMAIL PROTECTED]:~/bug$ python t.py
Segmentation fault
[EMAIL PROTECTED]:~/bug$ python
Python 2.5b3 (trunk:51066, Aug 3 2006, 13:18:16)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
[18733 refs]
[7182 refs]
[EMAIL PROTECTED]:~/bug$ gdb python
GNU gdb 6.4-debian
Copyright 2005 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...Using host libthread_db
library "/lib/tls/i686/cmov/libthread_db.so.1".
(gdb) r t.py
Starting program: /home/rs/exp/bin/python t.py
[Thread debugging using libthread_db enabled]
[New Thread -1210304832 (LWP 21660)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1210304832 (LWP 21660)]
0x0811b68a in visit_decref (op=0xb7db21d8, data=0x0) at
Modules/gcmodule.c:270
270 if (PyObject_IS_GC(op)) {
(gdb) bt
#0 0x0811b68a in visit_decref (op=0xb7db21d8, data=0x0) at
Modules/gcmodule.c:270
#1 0x0814043d in property_traverse (self=0xb7d7b5b4, visit=0x811b654
, arg=0x0) at Objects/descrobject.c:1235
#2 0x0811b753 in subtract_refs (containers=0x8197d88) at
Modules/gcmodule.c:295
#3 0x0811c453 in collect (generation=2) at Modules/gcmodule.c:790
#4 0x0811cf76 in PyGC_Collect () at Modules/gcmodule.c:1264
#5 0x0810d5da in Py_Finalize () at Python/pythonrun.c:377
#6 0x08057556 in Py_Main (argc=2, argv=0xbfb5a6f4) at Modules/main.c:516
#7 0x080565ba in main (argc=2, argv=0xbfb5a6f4) at ./Modules/python.c:23
(gdb)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
Ralf Schmitt schrieb:
> Hi all,
>
> I've been trying to port our software to python 2.5.
> unfortunately I'm getting constantly hit by segfaults.
>
> I've boiled it down to the following code:
>
> [EMAIL PROTECTED]:~/bug$ cat t.py
> import array
>
> class Indexer(object):
> maximumForwardSize = property(array.array.fromstring)
> [EMAIL PROTECTED]:~/bug$ python t.py
> Segmentation fault
Confirmed under Windows. Here's the problem (Objects/descrobject.c, near line
1200,
in property_init(...):
/* if no docstring given and the getter has one, use that one */
if ((doc == NULL || doc == Py_None) && get != NULL &&
PyObject_HasAttrString(get, "__doc__")) {
if (!(get_doc = PyObject_GetAttrString(get, "__doc__")))
return -1;
Py_DECREF(get_doc); /* it is INCREF'd again below */
^^
doc = get_doc;
}
Py_XINCREF(get);
Py_XINCREF(set);
Py_XINCREF(del);
Py_XINCREF(doc);
If the refcount of get_doc drops to zero in the Py_DECREF(), the Py_XINCREF()
shortly after doesn't help ;-).
Thomas
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
> /* if no docstring given and the getter has one, use that one */
> if ((doc == NULL || doc == Py_None) && get != NULL &&
> PyObject_HasAttrString(get, "__doc__")) {
> if (!(get_doc = PyObject_GetAttrString(get, "__doc__")))
> return -1;
> Py_DECREF(get_doc); /* it is INCREF'd again below */
> ^^
> doc = get_doc;
> }
>
> Py_XINCREF(get);
> Py_XINCREF(set);
> Py_XINCREF(del);
> Py_XINCREF(doc);
>
A strange programming style, if you ask me, and I wonder why Coverity doesn't
complain
about it.
Thomas
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
Thomas Heller <[EMAIL PROTECTED]> wrote in
news:[EMAIL PROTECTED]:
>> /* if no docstring given and the getter has one, use that one */
>> if ((doc == NULL || doc == Py_None) && get != NULL &&
>> PyObject_HasAttrString(get, "__doc__")) {
>> if (!(get_doc = PyObject_GetAttrString(get, "__doc__")))
>>return -1;
>> Py_DECREF(get_doc); /* it is INCREF'd again below */
>> ^^
>> doc = get_doc;
>> }
>>
>> Py_XINCREF(get);
>> Py_XINCREF(set);
>> Py_XINCREF(del);
>> Py_XINCREF(doc);
>>
>
> A strange programming style, if you ask me, and I wonder why Coverity
> doesn't complain about it.
>
Does Coverity recognise objects on Python's internal pools as deallocated?
If not it wouldn't complain because all that the Py_DECREF does is link the
block into a pool's freeblock list so any subsequent reference from the
Py_XINCREF is still a reference to allocated memory.
[Off topic:
Microsoft have (or had?) a similarly screwy bit in their ActiveX atl
libraries: a newly created ActiveX object has its reference count
incremented before calling FinalConstruct and then decremented to 0
(using a method which decrements the reference count but doesn't free it)
before being incremented again. If in the meanwhile you increment and
decrement the reference count in another thread then it goes bang.]
The moral is to regard the reference counting rules as law: no matter how
sure you are that you can cheat, don't or you'll regret it.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
Duncan Booth wrote: > > > The moral is to regard the reference counting rules as law: no matter how > sure you are that you can cheat, don't or you'll regret it. > Or someone else will regret it, just like in this case. :) - Ralf ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] unicode hell/mixing str and unicode as dictionary keys
Still trying to port our software. here's another thing I noticed:
d = {}
d[u'm\xe1s'] = 1
d['m\xe1s'] = 1
print d
With python 2.4 I can add those two keys to the dictionary and get:
$ python2.4 t2.py
{u'm\xe1s': 1, 'm\xe1s': 1}
With python 2.5 I get:
$ python2.5 t2.py
Traceback (most recent call last):
File "t2.py", line 3, in
d['m\xe1s'] = 1
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
ordinal not in range(128)
Is this intended behaviour? I guess this might break lots of programs
and the way python 2.4 works looks right to me.
I think it should be possible to mix str/unicode keys in dicts and let
non-ascii strings compare not-equal to any unicode string.
At least it should be documented prominently in the "what's new" document.
- Ralf
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
Ralf Schmitt wrote:
> Still trying to port our software. here's another thing I noticed:
>
> d = {}
> d[u'm\xe1s'] = 1
> d['m\xe1s'] = 1
> print d
>
> With python 2.4 I can add those two keys to the dictionary and get:
> $ python2.4 t2.py
> {u'm\xe1s': 1, 'm\xe1s': 1}
>
> With python 2.5 I get:
>
> $ python2.5 t2.py
> Traceback (most recent call last):
>File "t2.py", line 3, in
> d['m\xe1s'] = 1
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
> ordinal not in range(128)
>
> Is this intended behaviour? I guess this might break lots of programs
> and the way python 2.4 works looks right to me.
> I think it should be possible to mix str/unicode keys in dicts and let
> non-ascii strings compare not-equal to any unicode string.
Also this behaviour makes your programs break randomly, that is, it will
break when the string you add hashes to the same value that the unicode
string has (at least that's what I guess..)
- Ralf
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
Duncan Booth <[EMAIL PROTECTED]> writes: > Does Coverity recognise objects on Python's internal pools as deallocated? Coverity doesn't work on that level; it analyzes source code, and knows about Python's INCREFs and DECREFs. > The moral is to regard the reference counting rules as law: no matter how > sure you are that you can cheat, don't or you'll regret it. This is the truth. Cheers, mwh -- As it seems to me, in Perl you have to be an expert to correctly make a nested data structure like, say, a list of hashes of instances. In Python, you have to be an idiot not to be able to do it, because you just write it down. -- Peter Norvig, comp.lang.functional ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
Ralf Schmitt wrote:
> Ralf Schmitt wrote:
>> Still trying to port our software. here's another thing I noticed:
>>
>> d = {}
>> d[u'm\xe1s'] = 1
>> d['m\xe1s'] = 1
>> print d
>>
>> With python 2.4 I can add those two keys to the dictionary and get:
>> $ python2.4 t2.py
>> {u'm\xe1s': 1, 'm\xe1s': 1}
>>
>> With python 2.5 I get:
>>
>> $ python2.5 t2.py
>> Traceback (most recent call last):
>>File "t2.py", line 3, in
>> d['m\xe1s'] = 1
>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
>> ordinal not in range(128)
>>
>> Is this intended behaviour? I guess this might break lots of programs
>> and the way python 2.4 works looks right to me.
>> I think it should be possible to mix str/unicode keys in dicts and let
>> non-ascii strings compare not-equal to any unicode string.
>
> Also this behaviour makes your programs break randomly, that is, it will
> break when the string you add hashes to the same value that the unicode
> string has (at least that's what I guess..)
This is because Unicode and 8-bit string keys only work
in the same way if and only if they are plain ASCII.
The reason lies in the hash function used by Unicode: it is
crafted to make hash(u) == hash(s) for all ASCII s, such
that s == u.
For non-ASCII strings, there are no guarantees as to the
hash value of the strings or whether they match or not.
This has been like that since Unicode was introduced, so it's
not new in Python 2.5.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Aug 03 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free !
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
On Aug 3, 2006, at 9:51 AM, M.-A. Lemburg wrote:
> Ralf Schmitt wrote:
>> Ralf Schmitt wrote:
>>> Still trying to port our software. here's another thing I noticed:
>>>
>>> d = {}
>>> d[u'm\xe1s'] = 1
>>> d['m\xe1s'] = 1
>>> print d
>>>
>>> With python 2.4 I can add those two keys to the dictionary and get:
>>> $ python2.4 t2.py
>>> {u'm\xe1s': 1, 'm\xe1s': 1}
>>>
>>> With python 2.5 I get:
>>>
>>> $ python2.5 t2.py
>>> Traceback (most recent call last):
>>>File "t2.py", line 3, in
>>> d['m\xe1s'] = 1
>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in
>>> position 1:
>>> ordinal not in range(128)
>>>
>>> Is this intended behaviour? I guess this might break lots of
>>> programs
>>> and the way python 2.4 works looks right to me.
>>> I think it should be possible to mix str/unicode keys in dicts
>>> and let
>>> non-ascii strings compare not-equal to any unicode string.
>>
>> Also this behaviour makes your programs break randomly, that is,
>> it will
>> break when the string you add hashes to the same value that the
>> unicode
>> string has (at least that's what I guess..)
>
> This is because Unicode and 8-bit string keys only work
> in the same way if and only if they are plain ASCII.
>
> The reason lies in the hash function used by Unicode: it is
> crafted to make hash(u) == hash(s) for all ASCII s, such
> that s == u.
>
> For non-ASCII strings, there are no guarantees as to the
> hash value of the strings or whether they match or not.
>
> This has been like that since Unicode was introduced, so it's
> not new in Python 2.5.
What is new is that the exception raised on "u == s" after hash
collision is no longer silently swallowed.
-bob
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
Ralf Schmitt schrieb: > I've been trying to port our software to python 2.5. > unfortunately I'm getting constantly hit by segfaults. I understand it's unfortunate for you, but it is fortunate for us that you have been trying to port your application before 2.5 was released, and reported the bug when you discovered it. So thanks for reporting it. FWIW, the bug that Thomas found was introduced by patch #1434038, and committed as r42916. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
M.-A. Lemburg wrote:
> Ralf Schmitt wrote:
>> Ralf Schmitt wrote:
>>> Still trying to port our software. here's another thing I noticed:
>>>
>>> d = {}
>>> d[u'm\xe1s'] = 1
>>> d['m\xe1s'] = 1
>>> print d
>>>
>>> With python 2.4 I can add those two keys to the dictionary and get:
>>> $ python2.4 t2.py
>>> {u'm\xe1s': 1, 'm\xe1s': 1}
>>>
>>> With python 2.5 I get:
>>>
>>> $ python2.5 t2.py
>>> Traceback (most recent call last):
>>>File "t2.py", line 3, in
>>> d['m\xe1s'] = 1
>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
>>> ordinal not in range(128)
>>>
>>> Is this intended behaviour? I guess this might break lots of programs
>>> and the way python 2.4 works looks right to me.
>>> I think it should be possible to mix str/unicode keys in dicts and let
>>> non-ascii strings compare not-equal to any unicode string.
>> Also this behaviour makes your programs break randomly, that is, it will
>> break when the string you add hashes to the same value that the unicode
>> string has (at least that's what I guess..)
>
> This is because Unicode and 8-bit string keys only work
> in the same way if and only if they are plain ASCII.
This is okay. But in the case where one is not ASCII I would prefer to
be able to compare them (not equal) instead of getting a UnicodeError.
I know it's too late to change this, ...
>
> The reason lies in the hash function used by Unicode: it is
> crafted to make hash(u) == hash(s) for all ASCII s, such
> that s == u.
>
> For non-ASCII strings, there are no guarantees as to the
> hash value of the strings or whether they match or not.
>
> This has been like that since Unicode was introduced, so it's
> not new in Python 2.5.
>
...but in the case of dictionaries this behaviour has changed and in
prior versions of python dictionaries did work as I expected them to.
Now they don't.
When working with unicode strings and (accidently) mixing with str
strings, things might seem to work until the first non-ascii string
is given to some code and one gets that UnicodeDecodeError (e.g. when
comparing them).
If one mixes unicode strings and str strings as keys in a dictionary
things might seem to work far longer until he tries to put in some non
ASCII string with the "wrong" hash value and suddenly things go boom.
I'd rather keep the pre 2.5 behaviour.
- Ralf
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
Greg Ewing wrote: > The time isn't the main issue. The main issue > is that almost all the use cases for round() > involve doing an int() on it afterwards. At > least nobody has put forward an argument to > the contrary yet. Or, more accurately, they *should* involve an int afterwards, but often don't, because people expect round(something) to produce an integer. Looking only at the one-argument round() calls not surrounded by int(), I would bet that most are bugs rather than intentional. The proportion is certainly high enough that I can't just trust it. Ron Adam wrote: > I think you are only considering part of the usefulness of round(). In > statistics and science math equations, rounding isn't used to get an > integer, but instead used to give an answer that is meaningful within > the margin of error of the data being used. Yes, but (1) Only for display (or storage). This rounding should explicitly not be done on intermediate values, as it introduces another source of error. (2) You need to pass the precision, so this isn't the one-argument round(). (3) As a side note, even the two-argument round may not be what you want; it isn't uncommon to have accuracy to a number of significant figures, rather than to an absolute "three places past the decimal point". This gets complicated enough that it is reasonable to use math.fround() or some such instead of a builtin. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
Ralf Schmitt wrote:
Still trying to port our software. here's another thing I noticed:
d = {}
d[u'm\xe1s'] = 1
d['m\xe1s'] = 1
print d
With python 2.5 I get:
$ python2.5 t2.py
Traceback (most recent call last):
File "t2.py", line 3, in
d['m\xe1s'] = 1
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
ordinal not in range(128)
>> This is because Unicode and 8-bit string keys only work
>> in the same way if and only if they are plain ASCII.
>
> This is okay. But in the case where one is not ASCII I would prefer to
> be able to compare them (not equal) instead of getting a UnicodeError.
> I know it's too late to change this, ...
It is too late to change this, since it was always like this ;-)
Seriously, Unicode is doing the right thing here: you should
really always get an exception if you compare apples and
oranges, rather than reverting to comparing the ids of apples
and oranges as fall-back solution.
I believe that Py3k will implement this.
>> The reason lies in the hash function used by Unicode: it is
>> crafted to make hash(u) == hash(s) for all ASCII s, such
>> that s == u.
>>
>> For non-ASCII strings, there are no guarantees as to the
>> hash value of the strings or whether they match or not.
>>
>> This has been like that since Unicode was introduced, so it's
>> not new in Python 2.5.
>>
>
> ...but in the case of dictionaries this behaviour has changed and in
> prior versions of python dictionaries did work as I expected them to.
> Now they don't.
Let's put it this way: Python 2.5 uncovered a bug in your
application that has always been there. It's better to
fix your application than arguing to cover up the bug again.
> When working with unicode strings and (accidently) mixing with str
> strings, things might seem to work until the first non-ascii string
> is given to some code and one gets that UnicodeDecodeError (e.g. when
> comparing them).
>
> If one mixes unicode strings and str strings as keys in a dictionary
> things might seem to work far longer until he tries to put in some non
> ASCII string with the "wrong" hash value and suddenly things go boom.
> I'd rather keep the pre 2.5 behaviour.
It's actually a good preparation for Py3k where 1 == u'abc' will
(likely) also raise an exception.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Aug 03 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free !
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
On Thu, 3 Aug 2006, M.-A. Lemburg wrote: [...] > It's actually a good preparation for Py3k where 1 == u'abc' will > (likely) also raise an exception. I though I'd heard (from Guido here or on the py3k list) that it was only 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still evaluate to False. Did I misunderstand? John ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)
Martin v. Löwis wrote: > Ralf Schmitt schrieb: >> I've been trying to port our software to python 2.5. >> unfortunately I'm getting constantly hit by segfaults. > > I understand it's unfortunate for you, but it is fortunate > for us that you have been trying to port your application > before 2.5 was released, and reported the bug when you > discovered it. So thanks for reporting it. > > FWIW, the bug that Thomas found was introduced by > patch #1434038, and committed as r42916. I already fixed it locally and will commit as soon as trunk is unfrozen. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
John J Lee wrote: > On Thu, 3 Aug 2006, M.-A. Lemburg wrote: > [...] >> It's actually a good preparation for Py3k where 1 == u'abc' will >> (likely) also raise an exception. > > I though I'd heard (from Guido here or on the py3k list) that it was only > 1 < u'abc' that would raise an exception, and that 1 == u'abc' would still > evaluate to False. Did I misunderstand? Could be that I'm wrong. All such mixed-type compares are wrong depending on the scale of brokenness you apply ;-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Aug 03 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
On Thu, 3 Aug 2006, Greg Ewing wrote: > The time isn't the main issue. The main issue > is that almost all the use cases for round() > involve doing an int() on it afterwards. That's my experience as well. In my opinion, the purpose of round() is most commonly described as "to make an integer". So it should yield an integer. As other comments in this discussion imply, whether you want round() to return an integer really depends on your intended audience and their pattern of usage. It seems to me that for round() to produce an integer is about as intuitive as for 1/2 to produce 0.5. That is, if you bring basic mathematical intuition to the table instead of a background in how C behaves, then "1/2" suggests one-half and "rounding" suggests "convert to an integer". If you have a background in programming languages and you understand the value of type consistency, then you would see why dividing integers should yield integers and rounding floats should yield floats. Since Python was indeed changed to make 1/2 evaluate to 0.5, that suggests that it is Pythonic to target the first audience rather than the second. And that implies that it would also be Pythonic to make round() return an int. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly ...
Ka-Ping Yee <[EMAIL PROTECTED]> wrote: > > That's my experience as well. In my opinion, the purpose of round() > is most commonly described as "to make an integer". So it should > yield an integer. Grrk. No, that logic is flawed. There are algorithms where the operation of rounding (or truncation) is needed, but where the value may be larger than can be held in an integer, and that is not an error. If the only rounding or truncation primitive converts to an integer, those algorithms are unimplementable. You need at least one primitive that converts a float to an integer, held as a float. Which is independent of whether THIS particular facility should yield an integer or float! Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly ...
On Aug 3, 2006, at 9:43 PM, Nick Maclaren wrote: Ka-Ping Yee <[EMAIL PROTECTED]> wrote: That's my experience as well. In my opinion, the purpose of round() is most commonly described as "to make an integer". So it should yield an integer. Grrk. No, that logic is flawed. There are algorithms where the operation of rounding (or truncation) is needed, but where the value may be larger than can be held in an integer, and that is not an error. Is that really true for python? Python integers are unbounded in magnitute, they are not the same as int or long in C, therefore any float except exceptional values like NaN can be converted to an integer value. The converse is not true, python integers can contain values that are larger than any float (aka C's double). >>> v = 1e200 >>> int(v) 69733122212510361659474503275455023626482417509503468484 355540755341963384047062518680275124159738824081821357343682784846393850 41047239877871023591066789981811181813306167128854888448L >>> type(v) >>> t = int(v) >>> t ** 2 39466244425020724235032897553743712247233978162062705420 868772363027380308001932133054230558394675289323324880702327952854432161 552216024892912466614409626956153314556116473848998339762109232220813863 099472521374735119038509661875525607726747258646821773646868361139842288 412173261267039669530389442594522433115448347796339690544576171593343952 0020822843337114038314499908946523848704L >>> float(t**2) Traceback (most recent call last): File "", line 1, in OverflowError: long int too large to convert to float Ronald smime.p7s Description: S/MIME cryptographic signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly ...
Ronald Oussoren <[EMAIL PROTECTED]> wrote: > > > There are algorithms where the operation of rounding (or truncation) > > is needed, but where the value may be larger than can be held in an > > integer, and that is not an error. > > Is that really true for python? Python integers are unbounded in > magnitute, they are not the same as int or long in C, therefore any > float except exceptional values like NaN can be converted to an > integer value. The converse is not true, python integers can contain > values that are larger than any float (aka C's double). It depends a great deal on what you mean by a Python integer! Yes, I was assuming the (old) Python model, where it is a C long, but so were many (most?) of the other postings. If you are assuming the (future?) model, where there is a single integer type of unlimited size, then that is true. There is still an efficiency point, in that such algorithms positively don't want a float value like 1.0e300 (or, worse, 1.0e4000) expanded to its full decimal representation as an intermediate step. Whatever. There should still be at least one operation that rounds or truncates a float value, returning a float of the same type, on either functionality or efficiency grounds. I and most programmers of such algorithms don't give a damn which it does, provided that it is clearly documented, at least half-sane and doesn't change with versions of Python. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] unicode hell/mixing str and unicode as dictionary keys
http://mail.python.org/pipermail/python-dev/2006-August/067934.html
M.-A. Lemburg mal at egenix.com
> Ralf Schmitt wrote:
>> Still trying to port our software. here's another thing I noticed:
>> d = {}
>> d[u'm\xe1s'] = 1
>> d['m\xe1s'] = 1
>> print d
(a 2-element dictionary, because they are not equal)
>> With python 2.5 I get: [ a traceback ending in ]
>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
>> ordinal not in range(128)
> Let's put it this way: Python 2.5 uncovered a bug in your
> application that has always been there.
No; he application would only have a bug if he expected those two
objects to compare equal. Trying to stick something hashable into a
dictionary should not raise an Exception just because there is already
a similar key, (regardless of whether or not the other key is equal or
identical).
The only way this error could be the right thing is if you were trying
to suggest that he shouldn't mix unicode and bytestrings at all.
There is a command line switch for that, but it doesn't get much use.
-jJ
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
Jim Jewett wrote:
> http://mail.python.org/pipermail/python-dev/2006-August/067934.html
> M.-A. Lemburg mal at egenix.com
>
>> Ralf Schmitt wrote:
>>> Still trying to port our software. here's another thing I noticed:
>
>>> d = {}
>>> d[u'm\xe1s'] = 1
>>> d['m\xe1s'] = 1
>>> print d
>
> (a 2-element dictionary, because they are not equal)
>
>>> With python 2.5 I get: [ a traceback ending in ]
>
>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 1:
>>> ordinal not in range(128)
>
>> Let's put it this way: Python 2.5 uncovered a bug in your
>> application that has always been there.
>
> No; he application would only have a bug if he expected those two
> objects to compare equal. Trying to stick something hashable into a
> dictionary should not raise an Exception just because there is already
> a similar key, (regardless of whether or not the other key is equal or
> identical).
Hmm, you have a point there...
>>> d = {}
# Two different objects
>>> x = 'a'
>>> y = hash(x)
>>> x
'a'
>>> y
12416037344
# ... with the same hash value
>>> hash(x)
12416037344
>>> hash(y)
12416037344
# Put them in the dictionary, causing a hash collision ...
>>> d[x] = 1
>>> d[y] = 2
# ... which is resolved by comparing the two for equality
# and assigning them to two different slots:
>>> d
{'a': 1, 12416037344: 2}
Since Python 2.5 propagates the compare exception, you get the
exception. Python 2.4 silenced the exception.
> The only way this error could be the right thing is if you were trying
> to suggest that he shouldn't mix unicode and bytestrings at all.
Good question. I wonder whether that's a reasonable approach for
Python 2.x (I'd say it is for Py3k).
Currently you can't safely mix non-ASCII string with Unicode
keys in the same dictionary.
Perhaps we ought to add an exception to the dict lookup mechanism
and continue to silence UnicodeErrors ?!
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Aug 03 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free !
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
Nick Coghlan wrote: > Ron Adam wrote: >> Consider an example where you are combining data that had different >> number of significant digits. Keeping all the digits of your answer >> gives a false since of accuracy. The extra digits are meaningless >> because the margin of error is greater than any of the digits that >> exceed the minimum number of significant digits in your data. So you >> need to remove the extraneous digits by rounding. Then this answer >> can be further used as data, and sense the number of significant >> digits is maintained, the margin of error can be calculated accurately. > > This is a fallacy though - add two numbers together each with an error > of +/- 0.05, and the error in the total is +/- 0.1. The approach you > propose gives the misleading impression that the error in the total is > still only +/- 0.05 (as the answer will contain 1 decimal place). I did say "the margin of error can be calculated accurately". I did not go into detail on the correct method to do that. You do need to round to the correct number of significant digits AND yes, you do need to also take into account the number of items in your data set. If you don't round, the value of your answer would be shifted up or down by the amount of the non significant digits. And the error range would also be shifted up or down by that amount. In any case, my point is (and my opinion) that rounding to floating point types is very common and not something that is needed rarely as it has been expressed in several messages. I don't want to debate the correct method of calculating errors and maintaining accuracy in statistical equations. I'll let the mathematicians do that. > If you really care about error, you need to do it properly (e.g. stddev > or adding the error margins). Yes > Anyway, it's not proposed to remove the "round to x decimal places" > capability entirely - merely remove it from the builtin round() so that > the return type can be changed. That would work. This is one of those places where I wonder when should a function be included in a types methods and called by either using arguments with __init__, an external function to call a private method, or by having it be a public method to be called with the dot syntax like is common for strings? If you look at the Decimal type, it has quite of few rounding methods, (as well as other non-underscored methods), while int, float, and long, do not have them. Also the Decimal _round() method returns the number unchanged (max precision) if no precision argument is given where round() has a default precision of 0. Will there be more efforts (in Py3k to unifiy Decimal and the other types so they are more alike in both the way they work and in how they are used? Would it make since for the numeric type to have methods (other than the underscored ones) that can be called with the dot syntax like we do in unicode and the Decimal types that do not have the underscores? Or would it be better to remove those in favor of builtin or module functions? The extremes of both might be a foolish consistency, but this may also be a good area to make some basic improvements for python 3k. (I'm sure Guido and others here have though some on this in general.) If these types gain a round method that all work the same way, then round() can then use those and it would return the same type as it is given. So then round(float(x)), round(int(x)), round(long(x)) and round(Decimal(x)) will return their own type respectfully. And to get a specific type, you would call the types method directly. int.round(x) -> integer Which is nicer than int(round(x)). And the round methods could have arguments to designate the type of rounding to use. Where 'f','c', and 'e' are floor, ceiling, and even rounding. int.round(x, 'c') # 'f' is the default rounding mode The functions round(), roundcieling(), and roundeven() could then call the method of the type with the appropriate argument, although I'd be happy without these round functions if there are round methods are consistent across types. Well, this is the direction I would go in. Hopefully by Python 9000 the decimal type or something equivalent will be as fast as all the others and/or computers will be so fast as to make it irrelevant, then we can have just one scaler base type. ;-) Cheers, Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
M.-A. Lemburg wrote: > Perhaps we ought to add an exception to the dict lookup mechanism > and continue to silence UnicodeErrors ?! I'd definitely consider a UnicodeError to be an indication that two objects are not equal. At the very least, in the context of a dictionary lookup. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bad interaction of __index__ and sequence repeat
Nick Coghlan wrote: > Nick Coghlan wrote: >> Armin Rigo wrote: >>> Hi, >>> >>> There is an oversight in the design of __index__() that only just >>> surfaced :-( It is responsible for the following behavior, on a 32-bit >>> machine with >= 2GB of RAM: >>> >>> >>> s = 'x' * (2**100) # works! >>> >>> len(s) >>> 2147483647 >>> >>> This is because PySequence_Repeat(v, w) works by applying >>> w.__index__ in >>> order to call v->sq_repeat. However, __index__ is defined to clip the >>> result to fit in a Py_ssize_t. This means that the above problem >>> exists >>> with all sequences, not just strings, given enough RAM to create such >>> sequences with 2147483647 items. >>> >>> For reference, in 2.4 we correctly get an OverflowError. >>> >>> Argh! What should be done about it? >> >> I've now got a patch on SF that aims to fix this properly [1]. > > I revised this patch to further reduce the code duplication associated > with the indexing code in the standard library. > > The patch now has three new functions in the abstract C API: > > PyNumber_Index (used in a dozen or so places) > - raises IndexError on overflow > PyNumber_AsSsize_t (used in 3 places) > - raises OverflowError on overflow > PyNumber_AsClippedSsize_t() (used once, by _PyEval_SliceIndex) > - clips to PY_SSIZE_T_MIN/MAX on overflow > > All 3 have an int * output argument allowing type errors to be flagged > directly to the caller rather than through PyErr_Occurred(). > > Of the 3, only PyNumber_Index is exposed through the operator module. > > Probably the most interesting thing now would be for Travis to review > it, and see whether it makes things easier to handle for the Numeric > scalar types (given the amount of code the patch deleted from the > builtin and standard library data types, hopefully the benefits to > Numeric will be comparable). I noticed most of the checks for PyInt where removed in the patch. If I remember correctly, I left these in for "optimization." Other than that, I think the patch is great. As far as helping with NumPy, I think it will help to be able to remove special-checks for all the different integer-types. But, this has not yet been done in the NumPy code. -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] using globals
Hi,
I have got a problem with importing global variables. For instance I have
got two files:
# t1.py #t2.py
counter = 1
def counter_adder(filenr): def
show_adder():
global counter import
t1
counter+= 1 print("adder is
%d" % t1.counter)
if filenr == 1:
show_adder()
else:
import t2
t2.show_adder()
def show_adder():
print("adder is %d" % counter)
>>counter_adder(1)
adder is 2
>>counter_adder(2)
adder is 1
When I look at the outcome of two function calls I expected no differences
but much to my surprise it goes wrong when the global variable is imported.
It doesn't give the updated value but the value it get when instantiated.
Who come? What should I do to get the updated value
greetings,
Sjaak van Werkhoven
DISCLAIMER***
Deze e-mail is uitsluitend bestemd voor de geadresseerde(n). Verstrekking aan
en gebruik door anderen is niet toegestaan. Fortis sluit iedere
aansprakelijkheid uit die voortvloeit uit electronische verzending.
This e-mail is intended exclusively for the addressee(s), and may not be passed
on to, or made available for use by any person other than the addressee(s).
Fortis rules out any and every liability resulting from any electronic
transmission.
**
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] uuid module - byte order issue
The UUID module uses network byte order, regardless of the platform
byte order. On little-endian platforms like Windows the ".bytes"
property of UUID objects is not compatible with the memory layout of
UUIDs:
>>> import uuid
>>> import pywintypes
>>> s = '{00112233-4455-6677-8899-aabbccddeeff}'
>>> uuid.UUID(s).bytes.encode('hex')
'00112233445566778899aabbccddeeff'
>>> str(buffer(pywintypes.IID(s))).encode('hex')
'33221100554477668899aabbccddeeff'
>>>
Ka-Ping Yee writes* that the Windows UUID generation calls are not RFC
4122 compliant and have an illegal version field. If the correct byte
order is used the UUIDs generated by Windows XP are valid version 4
UUIDs:
>>> parts = struct.unpack('>> parts[2] >> 12# version
4
>>> ord(parts[3][0]) & 0xC0# variant
128
The first three fields (32 bit time-low, 16 bit time-mid and
time-high-and-version) are stored in the platform byte order while the
remainder is stored as a vector of 8 bytes.
The bytes property and bytes argument to the constructor should use
the platform byte order. It would be nice to have explicit little
endian and big endian versions available on platforms of either
endianness for compatibility in communication and disk formats.
There is another issue with version 1 uuid generation:
>>> len(set(uuid.uuid1() for i in range(1000)))
992
The problem is that the random clock_seq field is only 14 bits long.
If enough UUIDs are generated within the same system clock tick there
will be collisions. Suggested solution: use the high-resolution of the
time field (100ns) to generate a monotonically increasing timestamp
that advances at least by 1 for each call, when time.time() returns
the same value on subsequent calls.
Oren
[*] http://mail.python.org/pipermail/python-dev/2006-June/065869.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] using globals
"Werkhoven J.P. van (Sjaak)" <[EMAIL PROTECTED]> wrote in news:[EMAIL PROTECTED] > I have got a problem with importing global variables. Questions about using Python belong on comp.lang.python or the general python mailing list. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
On 8/3/06, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
> > ...but in the case of dictionaries this behaviour has changed and in
> > prior versions of python dictionaries did work as I expected them to.
> > Now they don't.
>
> Let's put it this way: Python 2.5 uncovered a bug in your
> application that has always been there. It's better to
> fix your application than arguing to cover up the bug again.
I would understand this assertion if Ralf were expecting dictionaries
to consider
{ u'm\xe1s': 1, 'm\xe1s': 1 } == { u'm\xe1s': 1 } == { 'm\xe1s': 1 }
This is clearly a mess waiting to explode.
But that's not what he said. He expects, as is the case in python2.4,
len({ u'm\xe1s': 1, 'm\xe1s': 1 }) == 2
because u'm\xe1s' clearly does not equal 'm\xe1s'. Because it raises
an exception, the dictionary shouldn't consider it equal, so there
should be the two keys which happen to be somewhat equivalent.
While this is in fact in the NEWS (Patch #1497053 & bug #1275608), I
think this should be raised for further discussion. Raising the
exception is good for debugging mistakes, but bad for dictionaries
holding holding inequal objects that happen to hash to the same value,
and correclty raise exceptions on comparison.
When we thought it was just a debugging tool, it made sense to put it
straight in to 2.5. Since it actually can adversely affect behavior in
only slightly edgy cases, perhaps it should go through a warning phase
(which ideally could show the exception that was thrown, thus yielding
most or all of the intended debugging advantage).
Michael
--
Michael Urman http://www.tortall.net/mu/blog
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
M.-A. Lemburg wrote: > If you are eventually rounding to say 2 decimal > places in the end of the calculation, you won't > want to find yourself presenting the user 1.12 > and 1.13 as equal values :-) Even if, before rounding, they were actually 1.124 and 1.1251? And if the difference were only due to the unrepresentability of some decimal fraction exactly in binary? I still maintain that (a) rounding a *binary* float to *decimal* places is wrongheaded, and (b) digit chopping is a bad way to decide whether two inexact numbers should be considered equal. Not just a different way, but a poorer way. > Most > such calculations do work with floats, so having > round() return an int would just add another > coercion to a float for those use-cases. I'm *not* proposing to eliminate round-to-float, despite the fact that I can't see much use for it personally. I'm also *not* advocating changing the existing behaviour of round() or int(). That was just tentative speculation. All I'm asking for is another function that does round-to-int directly. I wouldn't have thought that was such a controversial idea, given the frequency of use for that operation. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
M.-A. Lemburg wrote: > Perhaps we ought to add an exception to the dict lookup mechanism > and continue to silence UnicodeErrors ?! Seems to be that comparison of unicode and non-unicode strings for equality shouldn't raise exceptions in the first place. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
On Aug 3, 2006, at 5:47 PM, M.-A. Lemburg wrote: >> The only way this error could be the right thing is if you were >> trying >> to suggest that he shouldn't mix unicode and bytestrings at all. > > Good question. I wonder whether that's a reasonable approach for > Python 2.x (I'd say it is for Py3k). It's my understanding that in py3k, there will be no implicit conversion, bytestrings and unicodes will never be equal (no matter what the contents), and so this wouldn't be an issue. (as u"1" == "1" would be the same sort of situation as 1 == "1" is now) James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly (Re: struct module and coercing floats to integers)
On Aug 3, 2006, at 2:34 AM, Greg Ewing wrote:
> Raymond Hettinger wrote:
>
>> -1 on an extra built-in just to save the time for function call
>
> The time isn't the main issue. The main issue
> is that almost all the use cases for round()
> involve doing an int() on it afterwards. At
> least nobody has put forward an argument to
> the contrary yet.
And I bet the main reason why round() in python returns a float is
because it does in C.
And it does in C because C doesn't have arbitrary size integers, so
if round returned integers, round(1e+308) couldn't work. In python,
however, that's no problem, since python does have arbitrarily big
integers.
There's also round(float("inf")), of course, which wouldn't be
defined if the result was an integer, but I don't think rounding
infinity is much of a use case.
And I do think the extension of round to allow the specification of
number of decimal places was a mistake. If you want that, you
probably really mean to do something like round(x * 10**y) instead.
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote: > M.-A. Lemburg wrote: > >> Perhaps we ought to add an exception to the dict lookup mechanism >> and continue to silence UnicodeErrors ?! > > Seems to be that comparison of unicode and non-unicode > strings for equality shouldn't raise exceptions in the > first place. Seems like a slightly better idea than having dictionaries suppress exceptions. Still not ideal though because sticking non-ASCII strings that are supposed to be text and unicode in the same data structures is *probably* still an error. -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rounding float to int directly ...
On Tue, Aug 01, 2006, Ron Adam wrote: > > I'm -1 on implicitly converting to an int when rounding. > > One reason is if your rounded (to int type) number is then used in an > equation you my end up with a integer result when you wanted a floating > point result. > > >>> 23.5/5.2 > 4.5192307692307692 > >>> round(23.5)/round(5.2) > 4.7998 > >>> round(23.5/5.2) > 5.0 > >>> int(round(23.5))/int(round(5.2)) > 4 That's not relevant to Py3K. Which is where this discussion should proably move because round() ain't changing in 2.x. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --Brian W. Kernighan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
Bob Ippolito <[EMAIL PROTECTED]> wrote:
> On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:
>
> > M.-A. Lemburg wrote:
> >
> >> Perhaps we ought to add an exception to the dict lookup mechanism
> >> and continue to silence UnicodeErrors ?!
> >
> > Seems to be that comparison of unicode and non-unicode
> > strings for equality shouldn't raise exceptions in the
> > first place.
>
> Seems like a slightly better idea than having dictionaries suppress
> exceptions. Still not ideal though because sticking non-ASCII strings
> that are supposed to be text and unicode in the same data structures
> is *probably* still an error.
If/when 'python -U -c "import test.testall"' runs without unexpected
error (I doubt it will happen prior to the "all strings are unicode"
conversion), then I think that we can say that there aren't any
use-cases for strings and unicode being in the same dictionary.
As an alternate idea, rather than attempting to .decode('ascii') when
strings and unicode compare, why not .decode('latin-1')? We lose the
unicode decoding error, but "the right thing" happens (in my opinion)
when u'\xa1' and '\xa1' compare.
- Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
On Aug 4, 2006, at 12:34 AM, Josiah Carlson wrote:
> As an alternate idea, rather than attempting to .decode('ascii') when
> strings and unicode compare, why not .decode('latin-1')? We lose the
> unicode decoding error, but "the right thing" happens (in my opinion)
> when u'\xa1' and '\xa1' compare.
Maybe you want those to compare equal, but _I_ want u'\xa1' and '\xc2
\xa1' to compare equal, so it should obviously use .decode('utf-8')!
(okay, no, I don't really want that.)
James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] RELEASED Python 2.5 (beta 3)
On behalf of the Python development team and the Python community, I'm happy to announce the third BETA release of Python 2.5. This is an *beta* release of Python 2.5. As such, it is not suitable for a production environment. It is being released to solicit feedback and hopefully discover bugs, as well as allowing you to determine how changes in 2.5 might impact you. If you find things broken or incorrect, please log a bug on Sourceforge. In particular, note that changes to improve Python's support of 64 bit systems might require authors of C extensions to change their code. More information (as well as source distributions and Windows and Universal Mac OSX installers) are available from the 2.5 website: http://www.python.org/2.5/ There's been over 50 fixes since the second beta. This large number of changes meant we felt more comfortable cutting a third beta release, rather than charging ahead to the release candidate. As of this release, Python 2.5 is now in *feature freeze*. Unless absolutely necessary, no functionality changes will be made between now and the final release of Python 2.5. The plan is that this will be the final beta release (no, really, this time for sure (probably)). We should now move to one or more release candidates, leading to a 2.5 final release early August. PEP 356 includes the schedule and will be updated as the schedule evolves. At this point, any testing you can do would be greatly, greatly appreciated. The new features in Python 2.5 are described in Andrew Kuchling's What's New In Python 2.5. It's available from the 2.5 web page. Amongst the language features added include conditional expressions, the with statement, the merge of try/except and try/finally into try/except/finally, enhancements to generators to produce a coroutine kind of functionality, and a brand new AST-based compiler implementation. New modules added include hashlib, ElementTree, sqlite3, wsgiref and ctypes. In addition, a new profiling module "cProfile" was added. Enjoy this new release, Anthony Anthony Baxter [EMAIL PROTECTED] Python Release Manager (on behalf of the entire python-dev team) pgpgyj51XmlUG.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
On Thu, 03 Aug 2006 21:34:04 -0700, Josiah Carlson <[EMAIL PROTECTED]> wrote:
>
>Bob Ippolito <[EMAIL PROTECTED]> wrote:
>> On Aug 3, 2006, at 6:51 PM, Greg Ewing wrote:
>>
>> > M.-A. Lemburg wrote:
>> >
>> >> Perhaps we ought to add an exception to the dict lookup mechanism
>> >> and continue to silence UnicodeErrors ?!
>> >
>> > Seems to be that comparison of unicode and non-unicode
>> > strings for equality shouldn't raise exceptions in the
>> > first place.
>>
>> Seems like a slightly better idea than having dictionaries suppress
>> exceptions. Still not ideal though because sticking non-ASCII strings
>> that are supposed to be text and unicode in the same data structures
>> is *probably* still an error.
>
>If/when 'python -U -c "import test.testall"' runs without unexpected
>error (I doubt it will happen prior to the "all strings are unicode"
>conversion), then I think that we can say that there aren't any
>use-cases for strings and unicode being in the same dictionary.
>
>As an alternate idea, rather than attempting to .decode('ascii') when
>strings and unicode compare, why not .decode('latin-1')? We lose the
>unicode decoding error, but "the right thing" happens (in my opinion)
>when u'\xa1' and '\xa1' compare.
It might be right for Latin-1 strings.
However, it would be even *more* surprising for the person who has to
figure out why his program works when his program gets a string containing
'\xc0' from one user but fails when it gets '\xe3\x81\x82' from another
user.
I like the exception that 2.5 raises. I only wish it raised by default
when using 'ascii' and u'ascii' as keys in the same dictionary. ;) Oh,
and that str and unicode did not hash like they do. ;)
Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] TRUNK is UNFROZEN
The trunk is unfrozen now. Sorry about the delay - combination of a laptop motherboard replacement and a DSL provider with a network that was .. ahem... slightly slow. I'm still planning to branch at 2.5c1, the next release. That then opens the trunk up for all the bad craziness that's planned for 2.6. Anthony ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys
On 8/3/06, Josiah Carlson <[EMAIL PROTECTED]> wrote:
> As an alternate idea, rather than attempting to .decode('ascii') when
> strings and unicode compare, why not .decode('latin-1')? We lose the
> unicode decoding error, but "the right thing" happens (in my opinion)
> when u'\xa1' and '\xa1' compare.
Since I use utf-8 way more than I use latin-1, -1. Since others do
not, -1 on any not obviously correct encoding other than ascii, which
gets grandfathered.
This raises an exception for a good reason. Yes it's annoying at
times. We should fix those times, not the (unbroken) exception.
Michael
--
Michael Urman http://www.tortall.net/mu/blog
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 2.5 status
Things are getting better, but we still have some really important outstanding issues. PLEASE CONTINUE TESTING AS MUCH AS POSSIBLE. Also, it would be great to use as many tools as possible to find bugs and improve quality. It would be especially nice to run Purify on Windows. I've updated PEP 356 with the current status. The current schedule is: rc 1:August 18, 2006 [planned] final: September 12, 2006 [planned] This somewhat strange schedule is to accommodate availability of people cutting the release. A branch will be made when the release candidate is done. Don't try to rush bug fixes in. Let's try to keep things stable. Keep the doc fixes coming. I believe all 3 outstanding issues (and solutions!) could use some more discussion. All bugs/patches blocking release are set to priority 9. http://python.org/sf/1530559 - struct rejecting floats (patch pending) http://mail.python.org/pipermail/python-dev/2006-July/067774.html Problem with __index__ (patch pending) http://mail.python.org/pipermail/python-dev/2006-August/067926.html str/unicode dict keys can raise an exception All the AST issues that I know about have been addressed. There are various other issues that would be nice to fix for 2.5. These are priority 8. All exist in 2.4 and possibly earlier: http://python.org/sf/1526585 - SystemError concat long strings http://python.org/sf/1523610 - PyArg_ParseTupleAndKeywords potential core dump http://python.org/sf/1517042 - Fix crashers/gc_inspection.py http://python.org/sf/1475523 - gettext.py bug http://python.org/sf/1467929 - %-formatting and dicts I will continue to use priorities. Please review bugs and patches and let me know if anything else should be re-prioritized. I will continue to stress test Python and use various tools to keep quality up. The current suite includes in no particular order: valgrind, Coverity, Klocwork, failmalloc, pychecker, not to mention the buildbot testing on commits and refleak test runs every 12 hours. Cheers, n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
