STINNER Victor added the comment:
> It is possible to retrieve "f" from the caller, PyEval_EvalCodeEx()
It does not always work, but it works sometimes, so it's better to try :-)
I applied my fix to Python 2.7, 3.2 and 3.3. lipython.py of Python 2.7 is
outdated, it should
STINNER Victor added the comment:
> Currently when running Python on a non-OSX posix environment
> under either the C locale, or with an invalid or missing locale,
> it's not possible to operate using unicode filenames outside
> the ascii range.
It was already discussed:
STINNER Victor added the comment:
> under either the C locale, or with an invalid or missing locale
The right fix is to fix your locale, not Python.
--
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
> If there was a separate LC_FILENAMES then Python could respect
> that and insist people set it, but there isn't.
During 1 month, we had PYTHONFSENCODING environment variable. It was not a good
idea. Again: please read the discussion (in cl
STINNER Victor added the comment:
> There are two problems with this: one is just the practical
> one that it scales poorly to have to tell every user to do this
> and to take them through working out how to set this in a way
> that covers cron jobs, daemons, things run over ssh, e
STINNER Victor added the comment:
> The main problem I see being discussed is that
> changing the encoding after Python starts would
> be dangerous, which I agree with, but we're not
> proposing to do that.
Not after Python start. Using two encodings at the same would just a
STINNER Victor added the comment:
I would be possible to implement incremental decoder with mbsrtowcs() and
incremental encoder with wcsrtombs(), by serializing mbstate_t to a long
integer (TextIOWrapper.tell() does something like that). The problem is that
mbsrtowcs() and wcsrtombs() are
STINNER Victor added the comment:
I should not write comments so late :-p
> Not after Python start. Using two encodings at the same would just ...
at the same time
> ... because I would like to inconsistency.
because it would lead to inconsist
STINNER Victor added the comment:
> Having more than one encoding on unix is already a reality, there's nothing
> to stop someone setting LANG=de_DE.UTF-8 and LC_MESSAGES=C say.
Nope. The locale encoding is chosen using LC_ALL, LC_CTYPE or LANG
variable: use the first non-emp
STINNER Victor added the comment:
> By default the Python SSL/TLS Stack (client/server) expose
> unsecure protocols (SSLv2) and unsecure ciphers (EXPORT 40bit DES).
If there is a problem, it should not be fixed in Python, but in the underlying
library (OpenSSL) or in applications. Pytho
STINNER Victor added the comment:
+ self.name = self.name.encode("iso-8859-1", "replace")
Why did you chose ISO-8859-1? I think that the filesystem encoding should be
used instead:
-self.name = self.name.encode("iso-8859-1", "replace")
+
STINNER Victor added the comment:
> I'm not sure about the best module to host this, though: os.path ?
Some OS don't provide atomic rename. If we only define a function when it is
atomic (as we do in the posix module, only expose functions available on the
OS), programs will
STINNER Victor added the comment:
"The gzip format (defined in RFC 1952) allows storing the original filename
(without the .gz suffix) in an additional field in the header (the FNAME
field). Latin-1 (iso-8859-1) is required."
Hum, it looks like the author of the gzip program (on Li
STINNER Victor added the comment:
> it will still be passing values that can't be
> interpreted by other processes as you highlighed earlier.
On UNIX, data going outside Python has be be encoded: you pass byte strings,
not directly Unicode. Surrogates are encoded back to ori
STINNER Victor added the comment:
This discussion is becoming very long, I didn't remember the original
purpose. You want to use UTF-8 instead of ASCII, so what? What do you
want to do with your nicely well decoded filenames? You cannot print it
to your terminal nor pass it to a subpr
STINNER Victor added the comment:
>> Nope. The locale encoding is chosen using LC_ALL, LC_CTYPE or LANG
>> variable: use the first non-empty variable. LC_MESSAGES doesn't affect
>> the encoding. Example:
>
> That's good to know, thanks. Only leaves the case
STINNER Victor added the comment:
On 22/12/2011 02:16, Martin Pool wrote:
> The proposal is that in some cases where Python currently assumes
> filenames are ascii on Linux, it ought to instead assume they are
> utf-8.
Oh, I expected a use case describing the problem, not the
STINNER Victor added the comment:
> The problem as I see it is this:
>
> On Linux, filenames are generally (but not always) in UTF-8; people
> fairly commonly end up with no locale configured, which causes Python
> to decode filenames as ascii. It is easy for this to end up with
STINNER Victor added the comment:
+ encoding = locale.getpreferredencoding()
It should be locale.getpreferredencoding(False).
--
___
Python tracker
<http://bugs.python.org/issue13
Changes by STINNER Victor :
--
title: Python Crashes When Saving Or Opening -> IDLE: Python Crashes When
Saving Or Opening
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
> Victor, could you try the attached script on FreeBSD,
> to see if you get ECONNREFUSED?
Yes, I get a ECONNREFUSED. I tested backlog.py on FreeBSD 8.2.
--
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
timemodule.c has the following check:
#if defined(_MSC_VER) || defined(sun)
if (buf.tm_year + 1900 < 1 || < buf.tm_year + 1900) {
PyErr_SetString(PyExc_ValueError,
"strftime() requires year
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue13703>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
> Unless there's evidence of performance regressions
> or backward incompatibilities, I agree.
If hash() is modified, str(dict) and str(set) will change for example. It may
break doctests. Can we consider that the application should not rely
(ind
Changes by STINNER Victor :
--
keywords: +patch
Added file: http://bugs.python.org/file24135/3106cc0a2024.diff
___
Python tracker
<http://bugs.python.org/issue13
STINNER Victor added the comment:
> I assume this is left over from the PEP 393 changes.
Correct.
> I'm not sure such a restriction needs to exist any more.
The restriction was introduced to simplify the implementation. maxchar has to
be computed exactly in format_stri
STINNER Victor added the comment:
Christian Heimes proposes the following change in its randomhash branch (see
issue #13704):
-x = (Py_uhash_t) *p << 7;
+x = Py_RndHashSeed + ((Py_uhash_t) *p << 7);
for (i = 0; i < len; i++)
x = (103U * x) ^ (Py_uhash_
STINNER Victor added the comment:
Paul first proposition (on python-dev) was to replace:
...
x = (ord(s[0]) << 7)
while i < length:
x = intmask((103*x) ^ ord(s[i]))
...
by:
...
x = (ord(s[0]) << 7)
while i < length:
x =
STINNER Victor added the comment:
> https://gist.github.com/0a91e52efa74f61858b5
Please, attach directly a file to the issue, or copy/paste the code in your
comment. Interesting part the code:
---
#Proposed replacement
#--
import os, array
size_expon
STINNER Victor added the comment:
I read that the attack cannot be computed with actual computers (it's too
expensive) against Python 64 bits. I tried to change str.__hash__ in Python 32
bits to compute the hash in 64 bits and than truncate the hash to 32 bits: it
doesn't chang
STINNER Victor added the comment:
> This affects the python implementation of RLock only.
In the issue #13550, it was discussed to remove completly the logging machinery
from the threading module. If we remove it, we don't need the Python
implementation of RLock.
We already rem
STINNER Victor added the comment:
"test_gdb has started failing recently on my Ubuntu Natty system:
...
>>> sysconfig.get_config_vars()['PY_CFLAGS']
'-Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes'
...
Marking haypo as nosy since it migh
STINNER Victor added the comment:
Yet another random hash function, simplified version of Paul's function. It
always use exactly 256 bits of entropy and so 32 bytes of memory, and doesn't
keep the loop. I don't expect my function to be secure, but just give more work
to
STINNER Victor added the comment:
Work-in-progress patch implementing my randomized hash function (random.patch):
- add PyOS_URandom() using CryptoGen, SSL (only on VMS!!) or /dev/urandom,
will a fallback on a dummy LCG if the OS urandom failed
- posix.urandom() is always defined and reuses
STINNER Victor added the comment:
> add PyOS_URandom() using CryptoGen, SSL (only on VMS!!)
> or /dev/urandom
Oh, OpenSSL (RAND_pseudo_bytes) should be used on Windows, Linux, Mac OS X,
etc. if OpenSSL is available. I was just too lazy to add a define or pyconfig.h
option to indic
STINNER Victor added the comment:
+printf("read %i bytes\n", size);
Oops, I forgot a debug message.
--
___
Python tracker
<http://bugs.python.o
STINNER Victor added the comment:
> If PHP uses it, I'm confident it is secure.
If I remember correctly, it is only used for the Windows version of PHP, but
PHP doesn't implement it correctly because it uses all bits.
--
___
Python t
STINNER Victor added the comment:
"Since speed is a concern, I think that the proposal to avoid using the random
hash for short strings is a good idea."
My proposition only adds two XOR to hash(str) (outside the loop on Unicode
characters), so I expect a ridiculous overhead. I don&
STINNER Victor added the comment:
> New changeset dfffb293f4b3 by Vinay Sajip in branch 'default'
The fix should also be applied to 3.2.
--
resolution: fixed ->
status: closed -> open
___
Python tracker
<http://bugs
STINNER Victor added the comment:
> I fear that an attacker may guess the seed from several small strings
hash(a) ^ hash(b) "removes" the suffix, but I don't see how to guess the prefix
from this new value. It doesn't mean that it is not possible, just that I don'
STINNER Victor added the comment:
"Calculating the hash of a null byte gives you the xor of your two seeds."
Not directly because prefix is first multiplied by 103. So hash("\0") gives
you (prefix*103) % 2^32 ^ suffix.
Example:
$ ./python
secret={b7abfbbf, db6c
STINNER Victor added the comment:
> At least for Python 2.x hash(str) and hash(unicode) have to yield
> the same result for ASCII only strings.
Ah yes, I forgot Python 2: I wrote my patch for Python 3.3. The two hash
functions should be modified to be randomized.
> hash("&q
STINNER Victor added the comment:
Patch version 2:
- hash("") is always 0
- Remove a debug message
--
Added file: http://bugs.python.org/file24143/random-2.patch
___
Python tracker
<http://bugs.python.o
STINNER Victor added the comment:
> What I propose is to make the amount of information necessary
> to analyze and generate collisions impractically large.
Not only: the attacker has to compute the collisions for the new seed. I don't
know how long it is, the code to generate co
STINNER Victor added the comment:
Some comments about termsize.diff.3.
I don't see why there are two functions, one should be enough:
get_terminal_size() should be dropped, and query_terminal_size() renamed to
get_terminal_size(). As said before, I don't think that reading ROWS an
STINNER Victor added the comment:
Note for myself, random-2.patch: _PyRandom_Init() must generate a prefix and a
suffix different than zero (call PyOS_URandom in a loop, and fail after 100
tries).
--
___
Python tracker
<http://bugs.python.
STINNER Victor added the comment:
"Given that a user has an application with an oracle function that returns the
hash of a unicode string, an attacker can probe tenth of thousand one and two
character unicode strings. That should give him/her enough data to calculate
both seeds.
STINNER Victor added the comment:
hash-attack.patch does never decrement the collision counter.
--
___
Python tracker
<http://bugs.python.org/issue13703>
___
___
STINNER Victor added the comment:
> - fd argument is retained, because we might want to test terminals
> opened with openpty.
You mean a terminal different than the one used for stdin, stdout and stderr?
> - two functions: still there. I think that get_terminal_size() should
>
STINNER Victor added the comment:
> on linux the underlying functionality is implemented in librt; the extension
> doesn't check for this or links with -lrt.
The changeset 35e4b7c4bafa changed configure.in to check clock_gettime(). It
checks without and with librt:
7.7 +AC_
STINNER Victor added the comment:
Version 3 of my patch:
- Add PYTHONHASHSEED environment variable to get a fixed seed or to
disable the randomized hash function (PYTHONHASHSEED=0)
- Add tests on the randomized hash function
- Add more tests on os.urandom()
--
Added file: http
STINNER Victor added the comment:
Patch version 4:
- os.urandom() raises again exceptions on failure
- drop support of VMS (which used RAND_pseudo_bytes from OpenSSL): I don't see
how to link Python/random.c to libcrypto on VMS, I don't have VMS, and it don't
see how it was
STINNER Victor added the comment:
Patch version 5 fixes test_unicode for 64-bit system.
--
Added file: http://bugs.python.org/file24198/random-5.patch
___
Python tracker
<http://bugs.python.org/issue13
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file24142/random.patch
___
Python tracker
<http://bugs.python.org/issue13703>
___
___
Python-bugs-list m
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file24193/random-3.patch
___
Python tracker
<http://bugs.python.org/issue13703>
___
___
Python-bugs-list m
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file24143/random-2.patch
___
Python tracker
<http://bugs.python.org/issue13703>
___
___
Python-bugs-list m
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file24196/random-4.patch
___
Python tracker
<http://bugs.python.org/issue13703>
___
___
Python-bugs-list m
STINNER Victor added the comment:
> * it is exceedingly complex
Which part exactly? For hash(str), it just add two extra XOR.
> * the method would need to be implemented for all hashable Python types
It was already discussed, and it was said that only hash(str) need to
be mo
STINNER Victor added the comment:
Patch version 6:
- remove a debug code in dev_urandom() (did always raise an exception for
testing)
- dev_urandom() raises an exception if open() fails
- os.urandom() uses again the right exception type and message (instead of a
generic exception
STINNER Victor added the comment:
I wrote bench_startup.py to measure the startup time on Windows. The precision
of the script is quite bad because Windows timer has a bad resolution (e.g.
15.6 ms on Windows 7) :-/
In release mode, the best startup time is 45.2 ms without random, 50.9 ms
STINNER Victor added the comment:
SafeDict.py: with this solution, the hash of key has to be recomputed at each
access to the dict (creation, get, set), the hash is not cached in the string
object.
--
___
Python tracker
<http://bugs.python.
STINNER Victor added the comment:
Update the patch and describe the new feature in regrtest doc (--help).
--
Added file: http://bugs.python.org/file22067/regrtest_sigusr1-2.patch
___
Python tracker
<http://bugs.python.org/issue12
Changes by STINNER Victor :
--
nosy: +loewis
___
Python tracker
<http://bugs.python.org/issue12100>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
@charles-francois.natali: Your patch is ok, you can commit it into 3.1, 3.2,
3.3. But you may wait after 3.2.1.
@Georg Brandl: Should we wait after Python 3.2.1 for this issue?
--
nosy: +georg.brandl
___
Python
STINNER Victor added the comment:
Issue #12105 adds os.O_CLOEXEC flag, so we will be able to write
open(os.open(filename, os.O_RDONLY|os.O_CLOEXEC)). Do you want to work on a doc
patch?
--
___
Python tracker
<http://bugs.python.org/issue12
STINNER Victor added the comment:
>> open() documentation may explain the os.fdopen(os.open()) "trick"
>> to use low-level options like O_SYNC or O_CLOEXEC.
> Why not, but I leave it to someone more comfortable with
> documentation than me :-)
Issue #12103
STINNER Victor added the comment:
I don't know if it's related, but SimpleXMLRPCServer in Python 2.7 uses
fcntl(self.fileno(), fcntl.F_SETFD, flags):
class SimpleXMLRPCServer(SocketServer.TCPServer,
SimpleXMLRPCDispatcher):
...
def __i
STINNER Victor added the comment:
Oh, by the way: it would also be nice to add os.O_CLOEXEC to Python 2.7.
For example, tempfile._mkstemp_inner() uses:
fd = _os.open(file, flags, 0600)
_set_cloexec(fd) # fcntl(fd, F_SETFD, flags | FD_CLOEXEC)
which is not atomic
STINNER Victor added the comment:
> These changes cause test failure on 3.1 branch when verbose mode is disabled
What a shame! I commited a debug "print()":
1.39 +print(fname)
It should be fixed by my last commit.
--
___
P
STINNER Victor added the comment:
> Patch looks fine to me. Is it easily testable?
test_subprocess has some tests checking "cloexec": test_pipe_cloexec() and
test_pipe_cloexec_real_tools(). You may reuse some code from these tests?
--
_
STINNER Victor added the comment:
> One moment -- adding a new value to the os module looks like a new
> feature to me. Is there any convincing reason why this needs to go to
> 3.2? (And it most definitely shouldn't go to 3.1.)
Python doesn't suppose atomic open+CLOEXEC
New submission from STINNER Victor :
test_logging does sometimes fail.
A recent example on Windows XP:
http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/4645/steps/test/logs/stdio
test test_logging failed -- Traceback (most recent call last):
File
"D:\cygwin\home
STINNER Victor added the comment:
I cannot reproduce the problem: "python -m test test_packaging test_packaging".
I suppose that it was fixed by tarek.
--
nosy: +haypo
resolution: -> fixed
status: open -> closed
___
Pytho
STINNER Victor added the comment:
It looks like Tarek fixed this bug because I'm unable to reproduce it.
--
nosy: +haypo
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.
STINNER Victor added the comment:
Duplicate of #12125.
--
nosy: +haypo
resolution: -> duplicate
status: open -> closed
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
Issue #12150 has been marked as a duplicate of this issue:
==
ERROR: test_get_path (test.test_sysconfig.TestSysConfig
New submission from STINNER Victor :
The queue doc contains the following example:
--
def worker():
while True:
item = q.get()
do_work(item)
q.task_done()
q = Queue()
for i in range(num_worker_threads):
t = Thread(target=worker)
t.daemon
Changes by STINNER Victor :
--
nosy: haypo
priority: normal
severity: normal
status: open
title: test_multiprocessing
___
Python tracker
<http://bugs.python.org/issue12
New submission from STINNER Victor :
http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%207.2%203.x/builds/1819/steps/test/logs/stdio
[180/355] test_multiprocessing
Timeout (1:00:00)!
Thread 0x28401d00:
File
"/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/multiproce
STINNER Victor added the comment:
> or just skip the test on Linux kernels older than 2.6.23
I like this solution, but I don't know how to test that the kernel doesn't
support O_CLOEXEC. My commit bff9265d677d will tell use the value of O_CLOEXEC
on the
"Linux-2.6.22-vs2
STINNER Victor added the comment:
Oh, Linux 2.6.27+ has a SOCK_CLOEXEC option: we may use it (but it should be
done in another issue). See also #12105.
--
___
Python tracker
<http://bugs.python.org/issue5
STINNER Victor added the comment:
Story of O_CLOEXEC in the GNU libc, by Ulrich Drepper: "Secure File Descriptor
Handling"
http://udrepper.livejournal.com/20407.html
--
> I could either add some voodoo configure checks to ensure
> that O_CLOEXEC is indeed supported
Hum, if
STINNER Victor added the comment:
Another idea is to write a best-effort function to open a file with CLOEXEC
flag:
* use O_CLOEXEC if available
* use fcntl(fd, F_SETFD, flags | FD_CLOEXEC) if O_CLOEXEC is missing or was
silently ignored by the kernel (by open)
Attached open_cloexec.py is
STINNER Victor added the comment:
> My commit bff9265d677d will tell use the value of O_CLOEXEC on the
> "Linux-2.6.22-vs2.2.0.7-gentoo-..." buildbot.
Here you have:
"AssertionError: 0 is not true : CLOEXEC flag not set (O_CLOEXEC=0x8)"
It's the same value
STINNER Victor added the comment:
> The real issue is that the libc defines O_CLOEXEC, but kernels prior
> to 2.6.23 don't support it: instead of returning EINVAL, the socket
> syscall silently ignores the flag (don't know why I made the comment
> about this flag being d
New submission from STINNER Victor :
Sometimes, we need to know the version of the Linux kernel. Recent examples:
test if SOCK_CLOEXEC or O_CLOEXEC are supported by the kernel or not. Linux <
2.6.23 *silently* ignores O_CLOEXEC flag of open().
linux_version() is already implemented
STINNER Victor added the comment:
> test_socket: ...has a nice linux_version() which should be moved to
> the platform module
I created the issue #12158 for that.
--
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
> The returned value should be a version string in a fixed format,
> not a tuple. I'd suggest to use _norm_version() for this.
How do you compare version strings? I prefer tuples, as sys.version_info,
because the comparaison is more natural:
STINNER Victor added the comment:
Use "%s.%s.%s" % linux_version() if you would like to format the version. The
format is well defined. (You should only do that under Linux.)
--
___
Python tracker
<http://bugs.python.o
STINNER Victor added the comment:
len(obj) is implemented using PyObject_Size() which is stores the result into a
Py_ssize_t, and so is limited to sys.maxsize (2**31-1 or 2**63-1).
This implementation detail should be documented.
--
nosy: +haypo
Changes by STINNER Victor :
--
title: threading._get_ident(): remove it in the doc and make it public ->
threading._get_ident(): remove it in the doc or make it public
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
deprecate_codecs.patch: "Deprecate open(), StreamReader, StreamWriter,
StreamReaderWriter, StreamRecord and EncodedFile() of the codec module. Use the
builtin open() function or io.TextIOWrapper instead."
EncodedFile() and StreamRecord cannot b
Changes by STINNER Victor :
--
title: Deprecate codecs.open() -> Deprecate codecs.open(), codecs.StreamReader
and codecs.StreamWriter
___
Python tracker
<http://bugs.python.org/iss
Changes by STINNER Victor :
--
status: -> open
___
Python tracker
<http://bugs.python.org/issue8796>
___
___
Python-bugs-list mailing list
Unsubscri
STINNER Victor added the comment:
> You can then use linux_version().split('.') in code that want
> to do version comparisons.
It doesn't give the expected result:
>>> ('2', '6', '9') < ('2', '6', '
STINNER Victor added the comment:
Thanks for your review, here is an updated patch.
--
Added file: http://bugs.python.org/file22082/regrtest_sigusr1-3.patch
___
Python tracker
<http://bugs.python.org/issue12
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file21995/regrtest_sigusr1.patch
___
Python tracker
<http://bugs.python.org/issue12073>
___
___
Python-bug
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file22067/regrtest_sigusr1-2.patch
___
Python tracker
<http://bugs.python.org/issue12073>
___
___
Pytho
STINNER Victor added the comment:
Merwok's review:
<< Looks like a useful feature.
http://bugs.python.org/review/12073/diff/2659/6441
File Lib/test/regrtest.py (right):
http://bugs.python.org/review/12073/diff/2659/6441#newcode164
Lib/test/regrtest.py:164: You can send a SIGUSR
New submission from STINNER Victor :
Codec.encode() and Codec.decode() refer to StreamCode, but I cannot find this
class in the doc nor in the code.
I suppose that it should be replaced by IncrementalEncoder and
IncrementalDecoder. If I'm correct, see attached patch.
--
1301 - 1400 of 35284 matches
Mail list logo
