[issue5200] unicode.normalize gives wrong result for some characters

[Peter Landgren]

Peter Landgren  added the comment:

> Martin v. Löwis  added the comment:
> > The same applies  "Å" and "A", "Ä" and "A" and "Ö" and "O"
> > which also are also different letters as "Ø" and "O" are.
>
> Sure. And rightfully, they "Å" is *not* (I repeat: not)
> normalized as "A", under NFD:
>
> py> unicodedata.normalize("NFD", u"Å")
> u'A\u030a'
>
> > Maybe not in the unicode world but in treal life.
>
> They are different letters also in the Unicode world.
>
> > That's why I'm a little confused.
>
> I think the confusion comes from your assumption that
> normalizing "Å" produces "A". It does not. Really not.

Yes, you are right.

However the confusion/problem shows up when it is used in the application to
build an alphabet and group for example all version of E, É, È, Ë, Ê
together under E. The first character in the result of normalize is
used to build alphabet labels for surnames:

letter = normalize("NFD", surname)[0].upper()
if letter != last_letter:
last_letter = letter

and this is why I get "A" when the surname begins with "Å".

This way it works for all variations of E to be grouped under "E",
but fails as "Å" is shown under the label "A", not the "A" in the
beginning of the alphabet but after "Z", where "ÅÄÖ" comes.
So a previous sorting of the surnames works correctly.
(The Swedish alphabet has 29 letters: A,B,C... X,Y,Z,Å,Ä,Ö)

Can you think of any solution to this conflict? 

u'\xd8'

u'A\u030a'

u'\xc5'

This is obviously the result of how the unicode spec is written
interpreting "Å" as a variation of "A". which it is not.

I have asked the unicode people, but not got any answer yet.

The application is GRAMPS: http://gramps-project.org/

Once again thanks for make some of the unicode stuff clear!
Regards,
Peter Landgren

Added file: http://bugs.python.org/file13025/unnamed

___
Python tracker 

___http://www.w3.org/TR/REC-html40/strict.dtd";>

p, li { white-space: pre-wrap; }

> 
Martin v. Löwis  added the comment:
> 
> The same applies  "Å" and "A", "Ä" and "A" and "Ö" and "O"
> 
> which also are also different letters as "Ø" and "O" are.
>
> 
Sure. And rightfully, they "Å" is *not* (I repeat: not)
> 
normalized as "A", under NFD:
>
> 
py> unicodedata.normalize("NFD", u"Å")
> 
u'A\u030a'
>
> 
> Maybe not in the unicode world but in treal life.
>
> 
They are different letters also in the Unicode world.
>
> 
> That's why I'm a little confused.
>
> 
I think the confusion comes from your assumption that
> 
normalizing "Å" produces "A". It does not. Really not.

Yes, 
you are right.

However the confusion/problem shows up when it is used in 
the application to
build 
an alphabet and group for example all version of E, É, È, Ë, Ê
together under E. The first character in the result of 
normalize is
used 
to build alphabet labels for surnames:

letter = normalize("NFD", surname)[0].upper()
if 
letter != last_letter:

last_letter = letter

and 
this is why I get "A" when the surname begins with "Å".

This 
way it works for all variations of E to be grouped under "E",
but 
fails as "Å" is shown under the label "A", not the "A" in the
beginning of the alphabet but after "Z", where "ÅÄÖ" 
comes.
So a 
previous sorting of the surnames works correctly.
(The 
Swedish alphabet has 29 letters: A,B,C... X,Y,Z,Å,Ä,Ö)

Can 
you think of any solution to this conflict? 

I 
still think "Å" or "Ä" or "Ö" should behave as "Ø":
>>> unicodedata.normalize("NFD",u"Ø")
u'\xd8'

Now, 
as you said:
>>> unicodedata.normalize("NFD",u"Å")
u'A\u030a'

But 
it should be (in my opinion):
>>> unicodedata.normalize("NFD",u"Å")
u'\xc5'

This 
is obviously the result of how the unicode spec is written
interpreting "Å" as a variation of "A". which it is not.

I 
have asked the unicode people, but not got any answer yet.

The 
application is GRAMPS: http://gramps-project.org/

Once 
again thanks for make some of the unicode stuff clear!
Regards,
Peter 
Landgren
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5211] Fix complex type to avoid coercion in 2.7.

[Mark Dickinson]

Mark Dickinson  added the comment:

Comment by gumtree copied from issue3734 discussion:

> While Mark Dickinson's patch fixes the documentation, it does not 
offer 
> a solution to the original problem, which was rooted in a need to 
> provide special behaviour based on the numeric types. I made the 
> original posting because I hoped that this problem could be resolved.

gumtree, would you be interested in working on a patch for this feature-
request?  As mentioned above, the necessary changes are already present 
in 3.x, so all that's entailed is figuring out which bits of the 
Object/complexobject.c in the py3k source need to be transferred to the 
trunk, and making sure that everything's properly tested.

Alternatively, could you explain in a little more detail why this change 
is important to you?  Subclassing complex doesn't seem like a very 
common thing to want to do, so I'm curious about your use case.

--
nosy: +gumtree

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3734] subclassing complex

[Mark Dickinson]

Mark Dickinson  added the comment:

[gumtree]
> [...] does not offer 
> a solution to the original problem, which was rooted in a need to 
> provide special behaviour based on the numeric types. I made the 
> original posting because I hoped that this problem could be resolved.

Please see issue 5211 for further discussion.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Adam Olsen]

Adam Olsen  added the comment:

The alignment requirements (long double) make it impossible to have
anything in those bits.

Hypothetically, a custom allocator could lower the alignment
requirements to sizeof(void *).  However, rotating to the high bits is
pointless as they're the least likely to be used — impossible in this
case, as only the 2 highest bits would contain anything, and for that
you'd need a dictionary with at least 2 billion entries on 32bit, which
is more than the 32bit address space.  64-bit is similar.

Note that mixing the bits back in, via XOR or similar, is actually more
likely to hurt than help.  It's just like ints and strings, who's hash
values are very sequential, a simple shift tends to get us sequential
hashes.  This gives us a far lower collision rate than a statistically
random hash.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1552880] Unicode Imports

[Kristján Valur Jónsson]

Kristján Valur Jónsson  added the comment:

Ah, this one is still alive?
We still use this patch at CCP for our 2.x python.  I'll give it some 
more love to answer the issues raised.
Hm, is this still an issue with 3.x?  Does the imput machinery use 
unicode as the internal format when working with the import paths?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Mark Dickinson]

Mark Dickinson  added the comment:

[Adam Olsen]
> The alignment requirements (long double) make it impossible to have
> anything in those bits.

Not necessarily, since not all pointers passed to _Py_HashPointer come
from a PyObject_Malloc.  _Py_HashPointer is used for function pointers
as well.  For example, on 64-bit linux I get:

Python 2.7a0 (trunk:69516, Feb 11 2009, 10:43:51)
[GCC 4.2.1 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from hashlib import sha224
>>> hash(sha224)
47100514454970
>>> hash(sha224) % 16
10

> for that you'd need a dictionary with at least 2 billion entries
> on 32bit,

 If I recall correctly, the higher bits of the hash value also
get used in collision resolution: they're mixed in to the algorithm
that's used to produce the probing sequence when looking for an empty
slot.  So the dictionary wouldn't necessarily have to be quite that
large for the top bits to come into play. 

But I agree that mixing the bottom bits back in (via rotate, or xor, or
whatever) doesn't seem likely to help.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1552880] Unicode Imports

[Thomas Heller]

Changes by Thomas Heller :


--
nosy: +theller

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1076233] distutils.core.setup() with unicode arguments broken

[Walter Dörwald]

Walter Dörwald  added the comment:

It does indeed work with Python 2.6 (however not with 2.5). Closing.

--
resolution:  -> out of date
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le mercredi 11 février 2009 à 03:31 +, Adam Olsen a écrit :
> 
> .. although, if I replace object() with list() I get best results with a
> shift of 6 bits.  Replacing it with dict() is best with 8 bits.

But list() and dict() don't use id() for hash.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Mark Dickinson]

Mark Dickinson  added the comment:

Here's an updated patch, that errs on the conservative side:

- rotate instead of shifting, as suggested by Raymond.  This costs
  very little, and I admit to feeling uncomfortable about the
  possibility of just throwing bits away 

- explicit check for -1

- special case for sizeof(void *) = 2*sizeof(long)

All tests pass with the patch applied.  I've left the 'convert to
PyLong' code in as a safety net: it's used on platforms where
sizeof(void *) > sizeof(long) but sizeof(void *) != 2*sizeof(long).  I
don't know of any such platforms in current use.

Sample timings on 64-bit linux (non-debug trunk build, Core 2 Duo).  

before:

dict creation (selected):  1.18751096725
dict creation (shuffled):  1.21234202385
dict creation:  1.00831198692
set creation (selected):  0.869561910629
set creation (shuffled):  0.867420911789
set creation:  0.77153301239

and after:

dict creation (selected):  1.06817317009
dict creation (shuffled):  0.987659931183
dict creation:  0.662216901779
set creation (selected):  0.735805034637
set creation (shuffled):  0.659453868866
set creation:  0.445232152939

Added file: http://bugs.python.org/file13026/pointer_hash2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5094] datetime lacks concrete tzinfo impl. for UTC

[Walter Dörwald]

Walter Dörwald  added the comment:

The patch doesn't include any changes to the documentation.

--
nosy: +doerwalter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Chema Cortés]

Changes by Chema Cortés :


--
nosy: +chemacortes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4799] handling inf/nan in '%f'

[Mark Dickinson]

Changes by Mark Dickinson :


--
assignee:  -> marketdickinson
nosy: +marketdickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4880] PyInt_FromSsize_t LONG_MIN and LONG_MAX typecasts needed

[Mark Dickinson]

Mark Dickinson  added the comment:

It seems likely that this is a wine bug rather than a Python bug.

--
resolution:  -> works for me
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1507] complex constructor loses signs of zeros

[Mark Dickinson]

Mark Dickinson  added the comment:

This is out of date for 2.5.  Closing.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1327971] HTTPResponse instance has no attribute 'fileno'

[Daniel Diniz]

Daniel Diniz  added the comment:

OK, I'll work on a test + patch.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5216] distutils.tests.test_build_ext.BuildExtTestCase fails when srcdir != builddir

[Matthias Klose]

New submission from Matthias Klose :

these tests fail, when python is built with builddir != srcdir.

==
ERROR: test_build_ext (distutils.tests.test_build_ext.BuildExtTestCase)
--
Traceback (most recent call last):
  File
"/home/packages/python/2.6/python2.6-2.6.1/Lib/distutils/tests/test_build_ext.p
y", line 23, in setUp
shutil.copy(xx_c, self.tmp_dir)
  File "/home/packages/python/2.6/python2.6-2.6.1/Lib/shutil.py", line
88, in copy
copyfile(src, dst)
  File "/home/packages/python/2.6/python2.6-2.6.1/Lib/shutil.py", line
52, in copyfile
fsrc = open(src, 'rb')
IOError: [Errno 2] No such file or directory:
'/home/packages/python/2.6/python2.6-2.6
.1/build-static/Modules/xxmodule.c'

==
ERROR: test_solaris_enable_shared
(distutils.tests.test_build_ext.BuildExtTestCase)
--
Traceback (most recent call last):
  File
"/home/packages/python/2.6/python2.6-2.6.1/Lib/distutils/tests/test_build_ext.py",
line 23, in setUp
shutil.copy(xx_c, self.tmp_dir)
  File "/home/packages/python/2.6/python2.6-2.6.1/Lib/shutil.py", line
88, in copy
copyfile(src, dst)
  File "/home/packages/python/2.6/python2.6-2.6.1/Lib/shutil.py", line
52, in copyfile
fsrc = open(src, 'rb')
IOError: [Errno 2] No such file or directory:
'/home/packages/python/2.6/python2.6-2.6.1/build-static/Modules/xxmodule.c'

--
assignee: tarek
components: Distutils, Tests
messages: 81645
nosy: doko, tarek
severity: normal
status: open
title: distutils.tests.test_build_ext.BuildExtTestCase fails when srcdir != 
builddir
type: compile error
versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5217] testExtractDir (test.test_zipfile.TestWithDirectory) fails when python built with srcdir != builddir

[Matthias Klose]

New submission from Matthias Klose :

==
ERROR: testExtractDir (test.test_zipfile.TestWithDirectory)
--
Traceback (most recent call last):
  File
"/home/packages/python/2.6/python2.6-2.6.1/Lib/test/test_zipfile.py",
line 991, in testExtractDir
zipf = zipfile.ZipFile(findfile("zipdir.zip"))
  File "/home/packages/python/2.6/python2.6-2.6.1/Lib/zipfile.py", line
680, in __init__
self.fp = open(file, modeDict[mode])
IOError: [Errno 2] No such file or directory: 'zipdir.zip'

test.test_support.findfile searches in sys.path and in the directory
where test_support.py is located. this seems to be wrong when python is
built with srcdir != builddir. should the test overwrite the optional
second argument when calling findfile?

--
components: Tests
messages: 81646
nosy: doko
severity: normal
status: open
title: testExtractDir (test.test_zipfile.TestWithDirectory) fails when python 
built with srcdir != builddir
versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1633648] incomplete numerical comparisons

[Mark Dickinson]

Mark Dickinson  added the comment:

This issue is out of date from the perspective of python-pgsql, since
that's long been fixed.

It's difficult to figure out exactly what the issue is here.  As best as
I can guess, it's something like the following.  After:

class MyInt(object):
def __init__(self, n):
self.n = n

def __coerce__(self, other):
if type(other) is float:
return float(self.n), other

x = MyInt(3)

the comparison "x < 4.0" gives False in the 2.4+, and True in 2.3.

I'm closing this as a won't fix:  it doesn't seem worth changing the
comparison behaviour for Python 2.7 (which may well be the last in the
line of 2.x releases anyway), and the whole
coercion/three-way-comparison mess is gone in 3.x.

--
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3694] Undetected error in _struct.pack_into

[Daniel Diniz]

Daniel Diniz  added the comment:

Looks like both Undetected errors were corrected by Victor's patches,
which Benjamin committed around rev66693, so trunk only needs a test.
Here are the patches.

I think IndexError fits better (and matches trunk), as the issue is that
None is passed as "offset", which should be an Integral. If you
disagree, I'll modify the patch. Is PyErr_Format'ing the exception desired? 

BTW, there's a warning in _struct.c:180 -> warning: ‘get_ulong’ defined
but not used, should I open a new issue?

Added file: http://bugs.python.org/file13027/pynumber_assizet_py3k.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3694] Undetected error in _struct.pack_into

[Daniel Diniz]

Changes by Daniel Diniz :


Removed file: http://bugs.python.org/file13017/pynumber_assizet.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3694] Undetected error in _struct.pack_into

[Daniel Diniz]

Changes by Daniel Diniz :


Added file: http://bugs.python.org/file13028/pynumber_assizet_trunk.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5094] datetime lacks concrete tzinfo impl. for UTC

[Rafe Kaplan]

Rafe Kaplan  added the comment:

I thought I had uploaded this last night, apologies.

Added file: http://bugs.python.org/file13029/datetime-utc-doc.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2279] distutils sdist add_defaults does not add data_files

[George Sakkis]

George Sakkis  added the comment:

Agreed; package_data are also ignored by sdist. Unfortunately, neither
setuptools seems to work as expected on sdist but at least bdist_egg does.

MANIFEST.in is an ugly hack and should be deprecated; you shouldn't have
to repeat yourself in two files.

--
nosy: +gsakkis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1327971] HTTPResponse instance has no attribute 'fileno'

[Daniel Diniz]

Daniel Diniz  added the comment:

IMHO, using a fileno property looks better than __getattr__. Setting an
attribute in init works too, unless fp changes during the object life
(then 3.x is broken IIUC).

It works OK as a property of either urllib.addinfourl or of
httplib.HTTPResponse (socket would work, but is this broken there or a
higher layer). 

Tests and fixes for both solutions attached. As soon as one is chosen,
some docs on fileno would be nice, even for 3.x.


This (current) test in test_urllibnet.py passes, but I don't see how to
improve it, assuming it should fail/detect this bug (maybe self.urlopen
isn't testing urllib.urlopen correctly?):

def test_fileno(self):
if (sys.platform in ('win32',) or
not hasattr(os, 'fdopen')):
# On Windows, socket handles are not file descriptors; this
# test can't pass on Windows.
return
# Make sure fd returned by fileno is valid.
open_url = self.urlopen("http://www.python.org/";)
fd = open_url.fileno()
FILE = os.fdopen(fd)

--
keywords: +patch
stage: test needed -> patch review
Added file: http://bugs.python.org/file13030/httplib_fileno.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1327971] HTTPResponse instance has no attribute 'fileno'

[Daniel Diniz]

Changes by Daniel Diniz :


--
components: +Library (Lib) -None
type:  -> behavior
Added file: http://bugs.python.org/file13031/urllib_fileno.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4910] Remove uses of nb_long slot, and rename to nb_reserved.

[Mark Dickinson]

Mark Dickinson  added the comment:

PyNumber_Int deprecated in r69517, r69518.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1327971] HTTPResponse instance has no attribute 'fileno'

[Daniel Diniz]

Daniel Diniz  added the comment:

Hmm, always run all the tests :)

fileno is set in test_urllib2. New patches.

Added file: http://bugs.python.org/file13032/httplib_fileno_2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1327971] HTTPResponse instance has no attribute 'fileno'

[Daniel Diniz]

Changes by Daniel Diniz :


Removed file: http://bugs.python.org/file13030/httplib_fileno.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1327971] HTTPResponse instance has no attribute 'fileno'

[Daniel Diniz]

Changes by Daniel Diniz :


Added file: http://bugs.python.org/file13033/urllib_fileno_2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1327971] HTTPResponse instance has no attribute 'fileno'

[Daniel Diniz]

Changes by Daniel Diniz :


Removed file: http://bugs.python.org/file13031/urllib_fileno.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1294032] Distutils writes keywords comma-separated

[Tarek Ziadé]

Changes by Tarek Ziadé :


--
assignee: akuchling -> tarek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5200] unicode.normalize gives wrong result for some characters

[Martin v. Löwis]

Martin v. Löwis  added the comment:

> Can you think of any solution to this conflict? 

I don't quite understand why you want to place É, È, Ë, Ê all along
with E, yet Å,Ä,Ö after Z. Because that's what the Swedish alphabet
says?

Please understand that collation varies across languages. For example
in German, we also have Ä, but it does *not* come after Z. Instead,
there are two ways to collate Ä (telephone book vs. dictionary):
1. Ä sorts exactly like A
2. Ä sorts as if it was transcribed as Ae

So there is no one true collation of Ä, but you have to take into
account what language rules you want to follow.

If you want to implement Swedish rules, why then do you also want
to support É, È, Ë, Ê? Do you have these letters in Swedish at all?

If you want to use obscure collation rules, you might have to
implement the collation algorithm yourself. For example, assign
each letter a unique number (different from the Unicode ordinal),
and then sort by these numbers.

Take a look at ICU, which already includes collation algorithms
for many locales.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2805] 2to3 doesn't correct divisions

[Benjamin Peterson]

Benjamin Peterson  added the comment:

I concur with Raymond. There isn't yet a "-3" warning for class
division, though. I suppose the compiler could emit one when "/" is used
without from __future__ import division. That's another issue, though.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5200] unicode.normalize gives wrong result for some characters

[Peter Landgren]

Peter Landgren  added the comment:

The È... comes from French surnames and our French developer wants to group all 
versions 
of E together. The É... can be found in French surnames in Sweden as well as in 
Germany.
The program, GRAMPS is a genealogy program used in about 20 languages, so there 
is no 
preferred language.

I know. However, Swedish telephone books and dictionaries are sorted the same:
A,B,C... X,Y,Z,Å,Ä,Ö.

True. I agree. 
GRAMPS runs in the locale of the user, but must be able to handle information 
coming from 
many other languages/countries. That's why it's hard to be universal.

We can have them in names. See above.

I think we have found a solution that can handle most cases.
We treat surnames beginning with "ÅÄÖ" special. I don't think that there are 
many surnames 
outside the Nordic countries that starts with any of these three letters.

Vielen dank!

/Peter

Added file: http://bugs.python.org/file13034/unnamed

___
Python tracker 

___http://www.w3.org/TR/REC-html40/strict.dtd";>

p, li { white-space: pre-wrap; }

> 
I don't quite understand why you want to place É, È, Ë, Ê all along
> 
with E, yet Å,Ä,Ö after Z. Because that's what the Swedish alphabet
> 
says? 
The 
È... comes from French surnames and our French developer wants to group all 
versions of E together. The É... can be found in French surnames in Sweden as 
well as in Germany.
The 
program, GRAMPS is a genealogy program used in about 20 languages, so there is 
no preferred language.

> 
Please understand that collation varies across languages. For example
> 
in German, we also have Ä, but it does *not* come after Z. Instead,
> 
there are two ways to collate Ä (telephone book vs. dictionary):
> 
1. Ä sorts exactly like A
> 
2. Ä sorts as if it was transcribed as Ae
I 
know. However, Swedish telephone books and dictionaries are sorted the same:
A,B,C... X,Y,Z,Å,Ä,Ö.

> 
So there is no one true collation of Ä, but you have to take into
> 
account what language rules you want to follow.
True. 
I agree. 
GRAMPS runs in the locale of the user, but must be able to 
handle information coming from many other languages/countries. That's why it's 
hard to be universal.

> 
If you want to implement Swedish rules, why then do you also want
> 
to support É, È, Ë, Ê? Do you have these letters in Swedish at all?
We 
can have them in names. See above.

> 
If you want to use obscure collation rules, you might have to
> 
implement the collation algorithm yourself. For example, assign
> 
each letter a unique number (different from the Unicode ordinal),
> 
and then sort by these numbers.
>
> 
Take a look at ICU, which already includes collation algorithms
> 
for many locales.
I 
think we have found a solution that can handle most cases.
We 
treat surnames beginning with "ÅÄÖ" special. I don't think that there are 
many surnames outside the Nordic countries that starts with any of these three 
letters.

Vielen dank!

/Peter
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5218] Check for tp_iter in ceval:ext_do_call before overriding exception message

[Guilherme Polo]

New submission from Guilherme Polo :

Hi,

I find it weird that given this sample code:

def g():
yield iter(None)
list(*g())

I get this traceback:

Traceback (most recent call last):
  File "a1.py", line 3, in 
list(*g())
TypeError: type object argument after * must be a sequence, not generator

At a minimum the exception message looks awkward to me. With the
proposed patch, the new traceback would be:

Traceback (most recent call last):
  File "a1.py", line 3, in 
list(*g())
  File "a1.py", line 2, in g
yield iter(None)
TypeError: 'NoneType' object is not iterable

--
files: check_tpiter_before_overriding_msg.diff
keywords: patch
messages: 81657
nosy: gpolo
severity: normal
status: open
title: Check for tp_iter in ceval:ext_do_call before overriding exception 
message
Added file: 
http://bugs.python.org/file13035/check_tpiter_before_overriding_msg.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5218] Check for tp_iter in ceval:ext_do_call before overriding exception message

[Guilherme Polo]

Changes by Guilherme Polo :


--
components: +Interpreter Core
type:  -> behavior
versions: +Python 2.7, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5218] Check for tp_iter in ceval:ext_do_call before overriding exception message

[Guilherme Polo]

Guilherme Polo  added the comment:

(Btw, the suggestion to check for tp_iter came from Antoine Pitrou. It
seemed fine to me.)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5218] Check for tp_iter in ceval:ext_do_call before overriding exception message

[Antoine Pitrou]

Antoine Pitrou  added the comment:

You must check tp_iter as well, not only Py_TPFLAGS_HAVE_ITER.
Py_TPFLAGS_HAVE_ITER only tells you that the tp_iter field exists, not
that it's non-NULL.
(see the code for PyObject_GetIter() in Objects/abstract.c for an example)

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1159] os.getenv() not updated after external module uses C putenv()

[Jason R. Coombs]

Jason R. Coombs  added the comment:

As a workaround, could you use ctypes to pull the environment back into
the python context?  For example:

http://paste.turbogears.org/paste/34734

--
nosy: +jaraco

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5200] unicode.normalize gives wrong result for some characters

[Martin v. Löwis]

Martin v. Löwis  added the comment:

> The È... comes from French surnames and our French developer wants to group 
> all versions 
> of E together. The É... can be found in French surnames in Sweden as well as 
> in Germany.
> The program, GRAMPS is a genealogy program used in about 20 languages, so 
> there is no 
> preferred language.

I think you'll find that you have to think much harder about collation,
then. If you assume that the Unicode ordinal order will give right
collation, it will be wrong many times, I predict.

For example, it appears that Croatian puts Dž as a single letter between
D and Đ.

> I think we have found a solution that can handle most cases.
> We treat surnames beginning with "ÅÄÖ" special. I don't think that there are 
> many surnames 
> outside the Nordic countries that starts with any of these three letters.

It seems they are also common in Turkish (Öksüz, Ölcüm, Önal, ..., taken
from the Berlin phonebook), and Turkish puts Ö after O. Hungarian also
uses Ö and Ü (as well as Ó, Ú, Ő, Ű), but I don't know how common they
are as first letters of surnames.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5219] IDLE: slowness, pauses enter long strings

[John DeLuca]

New submission from John DeLuca :

In Interpreted Mode, when enter long strings, such as ofile.write (" 
this is a long string...") or print ("this is a long string..."), the 
keyboard entry is delayed - sometimes for 20+ seconds. If one completes 
the string (typing blindly as not text is visible) and keys the 
closing ')', or one moves the entire shell window, the missing text is 
dropped on the entry line and one can continue typing.
This occurs after about 25 characters of the string.

--
components: IDLE
messages: 81662
nosy: pyJohn
severity: normal
status: open
title: IDLE: slowness, pauses enter long strings
type: performance
versions: Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1327971] HTTPResponse instance has no attribute 'fileno'

[Daniel Diniz]

Daniel Diniz  added the comment:

The urllib patch breaks test_pyclbr because test_pyclbr can't handle
properties. I think the test is wrong here, hence the hack attached...
otherwise, let's go with httplib.

Added file: http://bugs.python.org/file13036/test_pyclbr_property.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2437] Distutils runtime_library_dirs broken on Windows

[Roumen Petrov]

Changes by Roumen Petrov :


--
nosy: +rpetrov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5215] change value of local variable in debug

[Benjamin Peterson]

Benjamin Peterson  added the comment:

Could you describe the problem in more detail please? I've having
trouble understanding you description. For example, you could you paste
the pdb session you used and explain what you expected?

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue969718] BASECFLAGS are not passed to module build line

[Roumen Petrov]

Changes by Roumen Petrov :


--
nosy: +rpetrov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5202] wave.py cannot write wave files into a shell pipeline

[David Jones]

David Jones  added the comment:

On 10 Feb 2009, at 21:15, David Jones wrote:

>
> David Jones  added the comment:
>
> On 10 Feb 2009, at 16:57, Guilherme Polo wrote:
>
>>
>> Guilherme Polo  added the comment:
>>
>> Now, is there some problem if we remove the calls to the "tell"  
>> method
>> in _write_header ? See patch attached (tests are very welcome too).
>
> Yes

Ahem.  Pardon me for answering you without reading your patch.  I  
have now read your patch and it does more than just remove the calls  
to "tell".  In fact it looks very fine.

It makes wave.py more like sunau.py in that it "just knows" what the  
offsets into the header are.  I think I like that (especially the way  
you use the struct format string to compute the second offset).  It  
also removes that nagging question at the back of my mind: "why does  
wave.py use tell when it could simply just know the offsets, which  
are constant anyway?".

And it works.  How cool is that?  I had changed my project to use  
sunau anyway, because that worked with pipes already.

Tests, you say...

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5220] os.makedirs' mode argument has bad default value

[Joshua Bronson]

New submission from Joshua Bronson :

os.makedirs' mode argument defaults to a hardcoded value of 511 (0777 in 
octal). This forces the caller to either pass in a different hardcoded value 
(commonly 0750), or to implement a workaround that calculates the expected 
mode based on the process owner's umask:

umask, _ = subprocess.Popen(['sh', '-c', 'umask'],
   stdout=subprocess.PIPE).communicate()
umask = int(umask, 8)
mode = 0777 ^ umask
os.makedirs(dir, mode)

Preferred behavior would be to have the mode default to the value which 
takes the umask into account rather than the hardcoded value 0777, so that 
directories would be created with the same permissions as e.g. files created 
via open(..).

N.B. I'm guessing the above workaround won't work on Windows (please excuse 
my poor Windows knowledge). All the more reason to have os.makedirs 
calculate the mode properly if none is given.

--
messages: 81666
nosy: jab
severity: normal
status: open
title: os.makedirs' mode argument has bad default value
versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5216] distutils.tests.test_build_ext.BuildExtTestCase fails when srcdir != builddir

[Roumen Petrov]

Roumen Petrov  added the comment:

I guess fixed in trunk - see issue4070 and issue4151 .

--
nosy: +rpetrov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5221] help related topic doesn't exist

[David W. Lambert]

New submission from David W. Lambert :

help function is not current, then again, neither is my python3. 
Possibly sequencemethods[12] are eradicated.

$ p3
Python 3.0rc1+ (py3k, Nov  5 2008, 14:44:46) 
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> help('SEQUENCEMETHODS')
...

Related help topics: SEQUENCES, SEQUENCEMETHODS2, SPECIALMETHODS

>>> help('SEQUENCEMETHODS1')
no Python documentation found for 'SEQUENCEMETHODS1'

>>> help('SEQUENCEMETHODS2')
no Python documentation found for 'SEQUENCEMETHODS2'

--
components: Interpreter Core
messages: 81668
nosy: LambertDW
severity: normal
status: open
title: help  related topic doesn't exist
type: behavior
versions: Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5220] os.makedirs' mode argument has bad default value

[Martin v. Löwis]

Martin v. Löwis  added the comment:

I don't understand the problem. The directory creation *does* take the
umask into account - that's the whole point of the umask! The directory
being created has the bits (mode & ~umask & 0777) set - because of the
semantics of mkdir(2), not because of Python's doing.

Closing the report as invalid.

--
nosy: +loewis
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Adam Olsen]

Adam Olsen  added the comment:

Antoine, I only meant list() and dict() to be an example of objects with
a larger allocation pattern.  We get a substantial benefit from the
sequentially increasing memory addresses, and I wanted to make sure that
benefit wasn't lost on larger allocations than object().

Mark, I concede the point about rotating; I believe the cost on x86 is
the same regardless.

Why are you still only rotating 3 bits?  My results were better with 4
bits, and that should be the sweet spot for the typical use cases.

Also, would the use of size_t make this code simpler?  It should be the
size of the pointer even on windows.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1303434] Please include pdb with windows distribution

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

Hi Martin, wouldn't just uploading the pythonXX.pdb to the release page
be enough.

The distributed DLL already points to the source location on your system
:-) E.g. for Python 2.5, this was c:\loewis\25\python\PCbuild\python25.pdb

Or are you thinking of having a separate MSI installer with all the .pdb
files included ?

--
nosy: +lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1303434] Please include pdb with windows distribution

[Marc-Andre Lemburg]

Changes by Marc-Andre Lemburg :


--
versions: +Python 2.7, Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Mark Dickinson]

Mark Dickinson  added the comment:

I'm fine with rotating 4 bits instead of 3, especially if the timings look 
good on 32-bit as well as 64-bit.

We should really benchmark dict lookups (and set membership tests) as well 
as dict creation.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1303434] Please include pdb with windows distribution

[Martin v. Löwis]

Martin v. Löwis  added the comment:

> Hi Martin, wouldn't just uploading the pythonXX.pdb to the release page
> be enough.

It's not clear to me what is being requested: just that single PDB file,
or also the PDB files for all of the extension modules? Also, shouldn't
the file be compressed somehow?

I also dislike the clutter this would add to the release page, which
already has a confusing number of files available for download.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5220] os.makedirs' mode argument has bad default value

[Joshua Bronson]

Joshua Bronson  added the comment:

Ah, I was misunderstanding the behavior of mkdir, thank you for the 
explanation.

My misunderstanding stemmed from recently coming across two widely-used 
packages which both pass mode=0750 to os.makedirs. I have no idea why 
they would be doing this (as it effectively throws away part of the 
umask), unless they too are misunderstanding the mode parameter. My 
suspicion is that the mode parameter is widely misunderstood to mean the 
desired permissions of the created directory. I have filed tickets in 
the packages in which I came across this to make sure they're doing what 
they intend.

Could the __doc__ for os.mkdir and os.makedirs be expanded to make this 
clearer?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5218] Check for tp_iter in ceval:ext_do_call before overriding exception message

[Guilherme Polo]

Guilherme Polo  added the comment:

Thanks for taking a look, my bad for taking so long to reply.

I'm adding a new patch now. This one, besides the fix needed mentioned
above, disregards tp_iter from old class instances (since they are
always available and tell us nothing).

Added file: http://bugs.python.org/file13037/check_before_overriding.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5202] wave.py cannot write wave files into a shell pipeline

[Guilherme Polo]

Guilherme Polo  added the comment:

Nice.

I said tests in hope wave gets more tests, since right one there is a
single one. I will see if I can produce something.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5220] os.makedirs' mode argument has bad default value

[Martin v. Löwis]

Martin v. Löwis  added the comment:

> My misunderstanding stemmed from recently coming across two widely-used 
> packages which both pass mode=0750 to os.makedirs. I have no idea why 
> they would be doing this (as it effectively throws away part of the 
> umask), unless they too are misunderstanding the mode parameter.

You use such a mode if you want the resulting directory definitely not
world-readable, and definitely not group-writable, despite what umask
the user running the program may have set.

A common case is installation programs, where the installer wants to
create directories independent of the umask root may have set when
running the installer.

> My 
> suspicion is that the mode parameter is widely misunderstood to mean the 
> desired permissions of the created directory. I have filed tickets in 
> the packages in which I came across this to make sure they're doing what 
> they intend.

My suspicion is that people setting explicit file permissions typically
know what they are doing, and that you will find that your tickets get
closed as invalid, explaining to you that this mode usage is fully
intentional.

> Could the __doc__ for os.mkdir and os.makedirs be expanded to make this 
> clearer?

I'm skeptical that this should be necessary. Whoever explained umask to
you should have mentioned how it really works (i.e. that it is built
into and considered by the operating system every time a file or
directory is created).

By the same pattern, the doc string of mkdir should also explain what
relative file names are and how the current directory affects their
interpretation. IMO, the doc string shouldn't teach fundamental system
principles.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Raymond Hettinger]

Raymond Hettinger  added the comment:

At four bits, you may be throwing away information and I don't think
that's cool.  Even if some selected timings are better with more bits
shifted, all you're really showing is that there is more randomness in
the upper bits than the lower ones.  But that doesn't mean than the
lower one contribute nothing at all.

I'm *much* more comfortable with a byte-swap, rotation, or xoring-in
upper bits than with shifts that potentially destroy entropy. 
Otherwise, your taxing apps that build giant sets/dicts and need all
distinguishing bits to avoid collision pile-ups.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> I'm *much* more comfortable with a byte-swap, rotation, or xoring-in
> upper bits than with shifts that potentially destroy entropy. 
> Otherwise, your taxing apps that build giant sets/dicts and need all
> distinguishing bits to avoid collision pile-ups.

Would (id() >> 4) + (id() & 15) be ok?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5220] os.makedirs' mode argument has bad default value

[Joshua Bronson]

Joshua Bronson  added the comment:

> My suspicion is that people setting explicit file permissions
> typically know what they are doing, and that you will find that
> your tickets get closed as invalid, explaining to you that this
> mode usage is fully intentional.

For what it's worth, I asked Mike Bayer (author of Beaker and Mako, the 
packages in question) what the intention was, and the 0750 seems to have 
been an error: Apparently HTML::Mason (what Myghty was based on) may 
have originally used 0750 but has since changed it to 0775 (search for 
"0775" in http://svn.urth.org/filedetails.php?
repname=Mason&path=%2Ftrunk%2Flib%2FHTML%2FMason%2FInterp.pm&rev=0&sc=0)
. The 0750 propagated from Myghty into Mako and Beaker, but has no 
apparent reason to be there.

For what else it's worth, I've run this by a few other programmers I 
consider quite smart and experienced (including Ian Bicking), and none 
of them realized the meaning of the mode parameter either. (If they had 
I never would have filed this ticket, as I checked with them first!)

I agree that knowledge of relative paths should be assumed, but I'm 
convinced that the mode behavior is widely misunderstood enough to 
warrant at least one-line "gotcha" in the docstring.


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Mark Dickinson]

Mark Dickinson  added the comment:

> At four bits, you may be throwing away information and I don't think
> that's cool.

The current patch *does* do a rotation:  it doesn't throw away any 
information.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Here is an updated patch. It uses a 4-bit shift and an addition. We
should avoid the use of logical or, because it makes the outputs
non-uniformly distributed ('1' bits are more likely).

Added file: http://bugs.python.org/file13038/pointer_hash3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Here is an updated benchmark script, for both object() and an
user-defined class, and adding dict lookup, set lookup and set difference.
Set difference is massively faster: up to 60% faster.

Added file: http://bugs.python.org/file13039/time_object_hash.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Antoine Pitrou]

Changes by Antoine Pitrou :


Removed file: http://bugs.python.org/file12981/time_object_hash.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1381476] csv.reader endless loop

[Daniel Diniz]

Daniel Diniz  added the comment:

Confirmed as working on trunk and py3k.

--
resolution:  -> out of date
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Guys, let's be careful.  Make sure that efforts to randomize lower bits
don't destroy information.   Something like x |= x>>8 is reversible and
fast.  Other fun looking transformations are not:

>>> len(set((x >> 4) + (x & 15) for x in range(10**6)))
62515

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1303434] Please include pdb with windows distribution

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

On 2009-02-11 22:20, Martin v. Löwis wrote:
> Martin v. Löwis  added the comment:
> 
>> Hi Martin, wouldn't just uploading the pythonXX.pdb to the release page
>> be enough.
> 
> It's not clear to me what is being requested: just that single PDB file,
> or also the PDB files for all of the extension modules? Also, shouldn't
> the file be compressed somehow?
> 
> I also dislike the clutter this would add to the release page, which
> already has a confusing number of files available for download.

I see these possibilities:

 * add just the Python DLL PDF file to the page

 * add a ZIP file with just the PDB files for all components

 * add a second MSI file which includes and installs the PDB
   files alongside the .dll/.pyd/.exe files

 * add the PDB files to the standard MSI files

We could put such files into a separate section on the release page.

Note that the PDB files are just around 1.6MB in compressed form
(probably less if integrated into the MSI file), so the last option
looks like the easiest solution.

FWIW: I don't think that the current release page layout looks
cluttered:

http://www.python.org/download/releases/2.6.1/

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1008086] patch for 767150

[Daniel Diniz]

Daniel Diniz  added the comment:

Test from issue 767150:
"""
socket.inet_aton() returns an 8 byte string when built 
on Linux/IA64. This should be 4 bytes on all 
architectures. Example:

>>> import socket
>>> socket.inet_aton('192.168.2.1')
'\xc0\xa8\x02\x01\x00\x00\x00\x00'
"""

The bug was confirmed by a third-party in 2006:
http://mail.python.org/pipermail/python-list/2006-August/400655.html

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1103926] email.base64MIME.header_encode vs RFC 1522

[Daniel Diniz]

Changes by Daniel Diniz :


--
resolution:  -> invalid
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1008086] patch for 767150

[Gregory P. Smith]

Gregory P. Smith  added the comment:

fixing now.

--
assignee: twouters -> gregory.p.smith
keywords: +easy
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Ok, updated patch:
- uses a 4-bit rotate (not shift)
- avoids comparing an unsigned long to -1
- tries to streamline the win64 special path (but I can't test)

Added file: http://bugs.python.org/file13040/pointer_hash4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue767150] socket.inet_aton on 64bit platform

[Gregory P. Smith]

Gregory P. Smith  added the comment:

Sad that this was closed as wontfix years ago.  anyways I am fixing 
this.  see #1008086.

--
keywords: +64bit
nosy: +gregory.p.smith
superseder:  -> patch for 767150

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1008086] socket.inet_aton returns 8 bytes on LP64 platforms

[Gregory P. Smith]

Gregory P. Smith  added the comment:

fixed in trunk r69519.

needs backporting to release26-maint and release30-maint.  leaving open 
until that happens.

--
keywords: +64bit -patch
title: patch for 767150 -> socket.inet_aton returns 8 bytes on LP64 platforms
type:  -> behavior
versions: +Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1046092] HTMLParser fix to accept mailformed tag attributes

[Daniel Diniz]

Daniel Diniz  added the comment:

Heh, the patch applies cleanly to trunk more than four years later and
tests pass fine. We'll surely need better tests if the behavior change
is considered an improvement.

--
nosy: +ajaksu2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue755660] allow HTMLParser to continue after a parse error

[Daniel Diniz]

Daniel Diniz  added the comment:

Tests still pass with updated patch, but new tests (and docs!) for this
feature are needed if Titus' positive review stands.

--
nosy: +ajaksu2
stage:  -> test needed
versions: +Python 2.7 -Python 2.3
Added file: http://bugs.python.org/file13041/htmlparser_error.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue755660] allow HTMLParser to continue after a parse error

[Daniel Diniz]

Changes by Daniel Diniz :


--
type:  -> feature request

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue775321] plistlib error handling

[Daniel Diniz]

Changes by Daniel Diniz :


--
dependencies: +plistlib error handling
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5211] Fix complex type to avoid coercion in 2.7.

[Blair]

Blair  added the comment:

I am happy to collaborate in finding a solution, but I do not have 
enough knowledge or skill to work with the code alone.

Simply documenting it does not remove the frustration that a few people 
will enounter. The key point being that you can subclass the other 
numeric types, but not complex. Worse, you may think that you have 
succeeded in subclassing complex too, because it is only when the 
__ropn__ binary operators are used that the type reverts to the base 
class type.

Would you want to subclass a numeric type? I agree, it is a bit obsure, 
but I did a search on this before making the post and there have been 
others who found the problem too. 

In my case, I think that the motivation may seem a bit obscure. I had 
developed some numerical-like types (from scratch -- no subclassing) 
and I wanted to be able to write functions taking as possible arguments 
these types and Python numerical types indifferently. I realised that I 
could not achieve exactly what I wanted, however, by subclassing float, 
int, etc I could add a few methods that would allow my generic 
functions to work with either my types or the subclassed Python types. 
At the same time, the subclassed numerical types could still be used as 
numerical quantities (float, int,...). It seemed like a pretty elegant 
solution.

If that explanation does not make sense, then I suppose other simpler 
motivations could be, eg, to subclass float so that only positive 
values are acceptable; to subclass complex so that only values lying 
within the unit circle are acceptable, etc. That is, one might like to 
define a type that can only take on physically meaningful values (mass 
cannot be negative, a complex reflection coeffcieint cannot have a 
magnitude greater than unity, ..)

So, my feeling is that this is worth fixing because the work done on 
float, int etc, is clearly useful and it appears (to me) that the 
complex case is an oversight.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue816059] popen2 work, fixes bugs 768649 and 761888

[Daniel Diniz]

Daniel Diniz  added the comment:

The test code seems to run fine in a debug build of trunk, but the
linked issues requires a more complex context.

--
dependencies: +popen2.Popen3 and popen2.Popen4 leaks filedescriptors
stage:  -> test needed
type:  -> behavior
versions: +Python 2.6 -Python 2.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1153027] http_error_302() crashes with 'HTTP/1.1 400 Bad Request

[Daniel Diniz]

Changes by Daniel Diniz :


--
stage:  -> test needed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1153027] http_error_302() crashes with 'HTTP/1.1 400 Bad Request

[Daniel Diniz]

Changes by Daniel Diniz :


--
versions: +Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue918368] urllib doesn't correct server returned urls

[Daniel Diniz]

Daniel Diniz  added the comment:

Any idea about how to test this one reliably?

--
stage:  -> test needed
type:  -> behavior
versions: +Python 2.6, Python 2.7 -Python 2.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5219] IDLE: slowness, pauses enter long strings

[John DeLuca]

John DeLuca  added the comment:

(pyJohn)... additional information since this is a performance issue: 
Lenovo s10; Win XP; 1GB; Intel Atom 1.66 Mhz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1562822] decimal module borks thread

[Daniel Diniz]

Daniel Diniz  added the comment:

Here's the output of trunk (debug) for OP's test:

./python ../thread.py
Exiting test.
Exception KeyError: KeyError(-1210869568,) in  ignored
[31071 refs]

--
nosy: +ajaksu2
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1548371] filterwarnings('error') has no effect

[Daniel Diniz]

Daniel Diniz  added the comment:

Confirmed in trunk.

--
nosy: +ajaksu2
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Adam Olsen]

Adam Olsen  added the comment:

> At four bits, you may be throwing away information and I don't think
> that's cool.  Even if some selected timings are better with more bits
> shifted, all you're really showing is that there is more randomness in
> the upper bits than the lower ones.  But that doesn't mean than the
> lower one contribute nothing at all.

On the contrary, the expected collision rate for a half-full dictionary
is about 21%, whereas I'm getting less than 5%.  I'm taking advantage of
the sequentiality of addresses, just as int and str hashes do for their
values.

However, you're right that it's only one use case.  Although creating a
burst of objects for a throw-away set may itself be common, it's
typically with int or str, and doing it with custom objects is
presumably fairly rare; certainly not a good microbenchmark for the rest
of the interpreter.

Creating a list of 10 objects, then shuffling and picking a few
increases my collision rate back up to 21%.  That should more accurately
reflect a long-running program using custom objects as keys in a dict.

That said, I still prefer the simplicity of a rotate.  Adding an
arbitrary set of OR, XOR, or add makes me uneasy; I know enough to do
them wrong (reduce entropy), but not enough to do them right.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1534764] sys.path gets munged with certain directory structures

[Daniel Diniz]

Daniel Diniz  added the comment:

Reproduced on trunk, needed unsetting PYTHONPATH. A test case with
os.mkdir, chdir, etc. would be great.

--
nosy: +ajaksu2
stage:  -> test needed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1074015] current directory in sys.path handles symlinks badly

[Daniel Diniz]

Daniel Diniz  added the comment:

Still happens in trunk.

--
nosy: +ajaksu2
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1161031] Neverending warnings from asyncore

[Daniel Diniz]

Daniel Diniz  added the comment:

Josiah: that'd be around rev36408, as  1.53 corresponds to rev36033
(from the log and issue 957240)

--
nosy: +ajaksu2
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5208] urllib2.build_opener([handler, ...]) incorrect signature in docs

[Daniel Diniz]

Daniel Diniz  added the comment:

I believe those square brackets mean "supplying handlers is optional",
but even so it'd be very ambiguous.

--
nosy: +ajaksu2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[Raymond Hettinger]

Raymond Hettinger  added the comment:

[Antoine]
> Ok, updated patch:
> - uses a 4-bit rotate (not shift)
> - avoids comparing an unsigned long to -1
> - tries to streamline the win64 special path (but I can't test)

pointer_hash4.patch looks fine to me.  Still, I think it's worth
considering the simpler and faster:  x |= x>>4.  The latter doesn't
require any special-casing for various pointer sizes.  It just works.

[Adam]
> Adding an arbitrary set of OR, XOR, or add makes me uneasy;
> I know enough to do them wrong (reduce entropy), but not 
> enough to do them right.

It's easy enough to prove (just show that the function is reversible)
and easy enough to test:

   assert len(set(ids)) == len(set(map(f, set(ids # for any large
group of ids

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1661108] base64.urlsafe_b64encode() shouldn't use the = character

[Daniel Diniz]

Daniel Diniz  added the comment:

Not following the spec doesn't seem reasonable. Adding a caveat about
how we follow the spec doesn't make much sense (I started to write one
and dropped it at "The resulting string" :D).

--
nosy: +ajaksu2
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1777134] minidom pretty xml output improvement

[Daniel Diniz]

Daniel Diniz  added the comment:

@Roy: we can try :)

Patch updated, tests pass. However, keeping the default output and
adding an option to prettyprint should keep lots of current users sane,
while those wanting prettier output could have it.

--
keywords: +easy
nosy: +ajaksu2
stage:  -> patch review
type:  -> feature request
versions: +Python 2.6, Python 2.7
Added file: http://bugs.python.org/file13042/minidom_output_improvement_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1777134] minidom pretty xml output improvement

[Roy Wood]

Roy Wood  added the comment:

Thanks!  :-)

On Wed, Feb 11, 2009 at 9:28 PM, Daniel Diniz wrote:

>
> Daniel Diniz  added the comment:
>
> @Roy: we can try :)
>
> Patch updated, tests pass. However, keeping the default output and
> adding an option to prettyprint should keep lots of current users sane,
> while those wanting prettier output could have it.
>
> --
> keywords: +easy
> nosy: +ajaksu2
> stage:  -> patch review
> type:  -> feature request
> versions: +Python 2.6, Python 2.7
> Added file:
> http://bugs.python.org/file13042/minidom_output_improvement_v3.patch
>
> ___
> Python tracker 
> 
> ___
>

Added file: http://bugs.python.org/file13043/unnamed

___
Python tracker 

___Thanks!  :-)On Wed, Feb 11, 2009 at 9:28 
PM, Daniel Diniz rep...@bugs.python.org> 
wrote:

Daniel Diniz aja...@gmail.com> 
added the comment:

@Roy: we can try :)

Patch updated, tests pass. However, keeping the default output and
adding an option to prettyprint should keep lots of current users sane,
while those wanting prettier output could have it.

--
keywords: +easy
nosy: +ajaksu2
stage:  -> patch review
type:  -> feature request
versions: +Python 2.6, Python 2.7
Added file: http://bugs.python.org/file13042/minidom_output_improvement_v3.patch"; 
target="_blank">http://bugs.python.org/file13042/minidom_output_improvement_v3.patch

___
Python tracker rep...@bugs.python.org>
http://bugs.python.org/issue1777134>
___

___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3991] urllib.request.urlopen does not handle non-ASCII characters

[Daniel Diniz]

Changes by Daniel Diniz :


--
components: +Library (Lib)
keywords: +easy
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5186] Reduce hash collisions for objects with no __hash__ method

[David W. Lambert]

David W. Lambert  added the comment:

"x |= x>>4"

Are you (Ray) sure you didn't mean

"x ^= x>>4"?

--
nosy: +LambertDW

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue776100] new function 'islastline' in fileinput

[Daniel Diniz]

Daniel Diniz  added the comment:

Issue 968063 has better patch, use case and discussion.

--
nosy: +ajaksu2
resolution:  -> duplicate
stage:  -> committed/rejected
status: open -> closed
superseder:  -> Add fileinput.islastline()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com