[issue18590] 'Search' and 'Replace' dialogs don't work on quoted text in Windows
New submission from Sarah: In Windows, the 'find' and 'replace' dialogs do not work properly on text that has been commented out using quotation marks. More specifically, the dialog *finds* the text in question, however, it does not *highlight* it. Without the highlighting, a user can't see which text has been found. Since the whole point of the find function is to show the user where the text is located, this is a bug. Credit for the fix goes to Roger Serwy, who suggested raising the hit tag. My patch uses the show_hit function from patch 17511*. I have included the show_hit function, but not the rest of patch 17511. Are there guidelines for submitting patches that utilize elements of other patches? *show_hit is originally from the ReplaceDialog. 17511 added it to SearchDialog -- components: IDLE, Windows files: SearchDialogPatch.patch keywords: patch messages: 193895 nosy: Sarah priority: normal severity: normal status: open title: 'Search' and 'Replace' dialogs don't work on quoted text in Windows type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file31077/SearchDialogPatch.patch ___ Python tracker <http://bugs.python.org/issue18590> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18590] 'Search' and 'Replace' dialogs don't work on quoted text in Windows
Changes by Sarah : -- resolution: -> works for me Added file: http://bugs.python.org/file31078/ReplaceDialogPatch.patch ___ Python tracker <http://bugs.python.org/issue18590> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17511] Idle find function closes after each find operation
New submission from Sarah: When I use the 'find' function in Idle, the dialogue closes after the first instance is found. To find the next instance, one must then use the 'find again' function, rather than the 'find' function. One can also use Ctrl+G, but that may not be intuitive for new users, who may not frequently use keyboard shortcuts. It would be nice to have a persistent box so that users can simply click 'find' repeatedly to iterate the 'find again' function. We would then have to alter when highlighting happens, as it currently happens only after the box is closed. I think the relevant code is located in the SearchDialog.py file on line 55. Unfortunately, I can't figure out how to adjust this myself, as I am a n00b. Note: I am working on Windows 7 with python ver 2.7.3 -- components: IDLE messages: 184915 nosy: Kuchinsky priority: normal severity: normal status: open title: Idle find function closes after each find operation versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue17511> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17511] Idle find function closes after each find operation
Sarah added the comment: This should allow the 'find' window to stay open. It also circumvents an issue in Windows, wherein highlighted text did not show while the 'find' window was open. The found text should now show as highlighted for any OS (as far as I know; please test). When the find window is closed, the cursor/highlighting is released. ------ nosy: +Sarah Added file: http://bugs.python.org/file29723/issue17511_v2.patch ___ Python tracker <http://bugs.python.org/issue17511> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17511] Idle find function closes after each find operation
Sarah added the comment: This changes the button's text from "Find" to "Find Again", as per Todd Rovito and Terry J. Reedy's suggestion. -- Added file: http://bugs.python.org/file29735/issue17511_FindAgain.patch ___ Python tracker <http://bugs.python.org/issue17511> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17511] Idle find function closes after each find operation
Sarah added the comment: Terry- There should be a checkbutton option labeled 'Wrap Around'. This is instantiated on lines 7 and 105-111 of SearchDialogBase.py Admittedly, it is standard practice to have a 'find' function wrap around. I suppose we could have the dialog start up with the 'Wrap around' box checked. Thoughts? Note: If we're going to make a change to the 'wrap around' settings, I think we should start a new issue. Wrap around really isn't related to the 'find box closing' issue. -- ___ Python tracker <http://bugs.python.org/issue17511> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17511] Idle find function closes after each find operation
Sarah added the comment: 'Find Again' switched to 'Find Next' -- Added file: http://bugs.python.org/file29740/issue_17511_FindNext.patch ___ Python tracker <http://bugs.python.org/issue17511> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17511] Idle find function closes after each find operation
Sarah added the comment: I agree that discussion is a good idea. Personally, though, I don't want to make the user hit ^F once to set the key and use ^G to find it again. It works well and, for a more advanced user, is great, but it adds to what a user must learn to get up and running in Python and doesn't conform to general conventions. I started using Python 6 months ago, and it took me a while to realize how 'find' worked in IDLE. For a beginner's IDE, I'd really like to see all of the features accessible through the GUI, and have those features conform to currently accepted standards. I'm certainly not saying there isn't a better way to do this. I'd love to have a conversation on what that might be. NOTE: This patch makes 'find' function's functionality consistent with the 'find' function in the 'Replace' menu. If we find a better way to implement the find function, we should extend that functionality to the 'Replace' menu. -- ___ Python tracker <http://bugs.python.org/issue17511> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2704] IDLE: Patch to make PyShell behave more like a Terminal interface
Changes by Sarah : -- nosy: +Sarah ___ Python tracker <http://bugs.python.org/issue2704> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38873] find_library for libcrypto and libssl on Catalina returns the unversioned library
New submission from Sarah Harvey : I've been tracking this through a bunch of different projects now. With the release of Mac OS X Catalina, libcrypto.dylib is a dummy library that causes an automatic segfault, to prevent upstream software from relying on it. However a large amount of software makes use of ctypes find_library() which will return the path to the unversioned library. It would be nice if we could specify "find the latest version" or similar so that we don't have to manually munge in a version, especially for strongly cross-platform libraries such as oscrypto. I've filed a similar bug here https://github.com/wbond/oscrypto/issues/35 where it's very clear how it manifests itself. -- components: ctypes messages: 357125 nosy: worldwise001 priority: normal severity: normal status: open title: find_library for libcrypto and libssl on Catalina returns the unversioned library type: behavior versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue38873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14060] Implement a CSP-style channel
Changes by Sarah Mount : -- nosy: +snim2 ___ Python tracker <http://bugs.python.org/issue14060> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6058] Add cp65001 to encodings/aliases.py
David-Sarah Hopwood added the comment:
This problem causes {{{os.getcwdu()}}} to fail when the console code page is
set to 65001 (always, I think):
{{{
t:\>ver
Microsoft Windows [Version 6.0.6002]
t:\>chcp
Active code page: 65001
t:\>python -c "import os; print os.getcwdu()"
Traceback (most recent call last):
File "", line 1, in
LookupError: unknown encoding: cp65001
t:\>chcp 1252
Active code page: 1252
t:\>python -c "import os; print os.getcwdu()"
t:\
}}}
Incidentally, I don't agree that this codepage needs to be distinguished from
UTF-8. The deviations in the Microsoft codec are just their bugs. There is only
one correct way to encode/decode UTF-8, and cp65001 is supposed to be UTF-8
according to Microsoft (e.g.
http://msdn.microsoft.com/en-us/library/86hf4sb8%28en-US,VS.80%29.aspx ).
--
nosy: +davidsarah
___
Python tracker
<http://bugs.python.org/issue6058>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6058] Add cp65001 to encodings/aliases.py
David-Sarah Hopwood added the comment: I said: "There is only one correct way to encode/decode UTF-8". This is true modulo differences in the treatment of initial byte order marks. -- ___ Python tracker <http://bugs.python.org/issue6058> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6058] Add cp65001 to encodings/aliases.py
David-Sarah Hopwood added the comment: I meant to say that the os.getcwdu() test in msg119440 was done with Windows native Python 2.6.2. -- ___ Python tracker <http://bugs.python.org/issue6058> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6058] Add cp65001 to encodings/aliases.py
David-Sarah Hopwood added the comment: Oops, false alarm. python -c "import os; print repr(os.getcwdu())" works as expected, so the exception is part of issue 1602. (My command about there being no need to distinguish this codepage from UTF-8 stands.) -- ___ Python tracker <http://bugs.python.org/issue6058> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment:
(For anyone wondering about the hold-up on this bug, I ended up switching to
Ubuntu. Not to worry, I now have Python 3 building in XP under VirtualBox --
which is further than I ever got with my broken Vista install :-/ It seems to
behave identically to native XP as far as this bug is concerned.)
Victor STINNER wrote:
> The question is now how to integrate WriteConsoleW() into Python without
> breaking the API, for example:
> - Should sys.stdout be a TextIOWrapper or not?
It pretty much has to be a TextIOWrapper for compatibility. Also it's easier to
implement it that way, because the text stream object has to be able to fall
back to using the buffer if the fd is redirected.
> - Should sys.stdout.fileno() returns 1 or raise an error?
Return sys.stdout.buffer.fileno(), which is 1 unless redirected.
This is the Right Thing because in Windows, fds are an abstraction of the C
runtime library, and the C runtime allows an fd to be associated with a
console. In that case, from the application's point of view it is still writing
to the same fd. In fact, we'd be implementing this by calling the WriteConsoleW
win32 API directly in order to avoid bugs in the CRT's Unicode support, but
that's an implementation detail.
> - What about sys.stdout.buffer: should sys.stdout.buffer.write() calls
> WriteConsoleA() or sys.stdout should not have a buffer attribute?
I was thinking that sys.std{out,err}.buffer would still be set up exactly as
they are now. Then if an app writes to that buffer, it will get interleaved
with any writes via the text stream. (The writes to the buffer go to the
underlying fd, which probably ends up calling WriteFile at the win32 level.)
> I think that many modules and programs now rely on sys.stdout.buffer to write
> directly bytes into stdout. There is at least python -m base64.
That would just work. The only caveat would be that if you write a partial line
to the buffer object (or if you set the buffer object to be fully buffered and
write to it), and then write to the text stream, the buffer wouldn't be flushed
before the text is written. I think that is fine as long as it is documented.
If an app sets the .buffer attribute of sys.std{out,err}, it would fall back to
using that buffer in the same way as when the fd is redirected.
> - Should we use ReadConsoleW() for stdin?
Yes. I'll probably start with a patch that just handles std{out,err}, though.
--
___
Python tracker
<http://bugs.python.org/issue1602>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment:
I wrote:
> The only caveat would be that if you write a partial line to the buffer
> object (or if you set the buffer object to be fully buffered and write to
> it), and then write to the text stream, the buffer wouldn't be flushed before
> the text is written.
Actually it looks like that already happens (because the sys.std{out,err}
TextIOWrappers are line-buffered separately to their underlying buffers), so it
would not be an incompatibility:
$ python3 -c 'import sys; sys.stdout.write("foo");
sys.stdout.buffer.write(b"bar"); sys.stdout.write("baz\n")'
barfoobaz
--
___
Python tracker
<http://bugs.python.org/issue1602>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment:
I wrote:
$ python3 -c 'import sys; sys.stdout.write("foo");
sys.stdout.buffer.write(b"bar"); sys.stdout.write("baz\n")'
barfoobaz
Hmm, the behaviour actually would differ here: the proposed implementation
would print
foobaz
bar
(the "foobaz\n" is written by a call to WriteConsoleW and then the "bar" gets
flushed to stdout when the process exits).
But since the naive expectation is "foobarbaz\n" and you already have to flush
after each call in order to get that, I think this change in behaviour would be
unlikely to affect correct applications.
--
___
Python tracker
<http://bugs.python.org/issue1602>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment: Glenn Linderman wrote: > Presently, a correct application only needs to flush between a sequence of > writes and a sequence of buffer.writes. Right. The new requirement would be that a correct app also needs to flush between a sequence of buffer.writes (that end in an incomplete line, or always if PYTHONUNBUFFERED or python -u is used), and a sequence of writes. > Don't assume the flush happens after every write, for a correct application. It's rather hard to implement this without any change in behaviour. Or rather, it isn't hard if the TextIOWrapper were to flush its underlying buffer before each time it writes to the console, but I'd be concerned about the extra overhead of that call. I'd prefer not to do that unless the new requirement above leads to incompatibilities in practice. -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment:
First a minor correction:
> The new requirement would be that a correct app also needs to flush between a
> sequence of buffer.writes (that end in an incomplete line, or always if
> PYTHONUNBUFFERED or python -u is used), and a sequence of writes.
That should be "and only if PYTHONUNBUFFERED or python -u is not used".
I also said:
> If an app sets the .buffer attribute of sys.std{out,err}, it would fall back
> to using that buffer in the same way as when the fd is redirected.
but the .buffer attribute is readonly, so this case can't occur.
Glenn Linderman wrote:
> Would it suffice if the new scheme internally flushed after every
> buffer.write? It wouldn't be needed after write, because the correct
> application would already do one there?
Yes, that would be sufficient.
> Am I off-base in supposing that the performance of buffer.write is expected
> to include a flush (because it isn't expected to be buffered)?
It is expected to be line-buffered. So an app might expect that printing
characters one-at-a-time will have reasonable performance.
In any case, given that the buffer of the initial std{out,err} will always be a
BufferedWriter object (since .buffer is readonly), it would be possible for the
TextIOWriter to test a dirty flag in the BufferedWriter, in order to check
efficiently whether the buffer needs flushing on each write. I've looked at the
implementation complexity cost of this, and it doesn't seem too bad.
A similar issue arises for stdin: to maintain strict compatibility, every read
from a TextIOWrapper attached to an input console would have to drain the
buffer of its buffer object, in case the app has read from it. This is a bit
tricky because the bytes drained from the buffer have to be converted to
Unicode, so what happens if they end part-way through a multibyte character?
Ugh, I'll have to think about that one.
Victor STINNER wrote:
> Some developers already think that adding sys.stdout.flush() after
print("Processing.. ", end='') is too hard (#11633).
IIUC, that bug is about the behaviour of 'print', and didn't suggest to change
the fact that sys.stdout is line-buffered.
By the way, are these changes going to be in a major release? If I understand
correctly, the layout of structs (for standard library types not prefixed with
'_', such as 'buffered' in bufferedio.c or 'textio' in textio.c) can change
with major releases but not with minor releases, correct?
--
___
Python tracker
<http://bugs.python.org/issue1602>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment: I wrote: > A similar issue arises for stdin: to maintain strict compatibility, every > read from a TextIOWrapper attached to an input console would have to drain > the buffer of its buffer object, in case the app has read from it. This is a > bit tricky because the bytes drained from the buffer have to be converted to > Unicode, so what happens if they end part-way through a multibyte character? > Ugh, I'll have to think about that one. It seems like there is no correct way for an app to read from both sys.stdin, and sys.stdin.buffer (even without these console changes). It must choose one or the other. -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment: Glenn wrote: > So if flush checks that bit, maybe TextIOWriter could just call buffer.flush, > and it would be fast if clean and slow if dirty? Yes. I'll benchmark how much overhead is added by the calls to flush; there's no point in breaking the abstraction boundary of BufferedWriter if it doesn't give a significant performance benefit. (I suspect that it might not, because Windows is very slow at scrolling a console, which might make the cost of flushing insignificant in comparison.) -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11395] print(s) fails on Windows with long strings
David-Sarah Hopwood added the comment:
If I understand the bug in the Windows console functions correctly, a limit of
32767 bytes might not always be small enough. The problem is that if two or
more threads are concurrently using any console functions (which all use the
same 64 KiB heap), they could try to allocate up to 32767 bytes plus overhead
at the same time, which will fail.
I wasn't able to provoke this by writing to sys.stdout.buffer (maybe there is
locking that prevents concurrent writes), but the following code that calls
WriteFile directly, does provoke it. GetLastError() returns 8
(ERROR_NOT_ENOUGH_MEMORY; see
http://msdn.microsoft.com/en-us/library/ms681382%28v=vs.85%29.aspx), indicating
that it's the same bug.
# Warning: this test may DoS your system.
from threading import Thread
import sys
from ctypes import WINFUNCTYPE, windll, POINTER, byref, c_int
from ctypes.wintypes import BOOL, HANDLE, DWORD, LPVOID, LPCVOID
GetStdHandle = WINFUNCTYPE(HANDLE, DWORD)(("GetStdHandle", windll.kernel32))
WriteFile = WINFUNCTYPE(BOOL, HANDLE, LPCVOID, DWORD, POINTER(DWORD), LPVOID) \
(("WriteFile", windll.kernel32))
GetLastError = WINFUNCTYPE(DWORD)(("GetLastError", windll.kernel32))
STD_OUTPUT_HANDLE = DWORD(-11)
INVALID_HANDLE_VALUE = DWORD(-1).value
hStdout = GetStdHandle(STD_OUTPUT_HANDLE)
assert hStdout is not None and hStdout != INVALID_HANDLE_VALUE
L = 32760
data = b'a'*L
def run():
n = DWORD(0)
while True:
ret = WriteFile(hStdout, data, L, byref(n), None)
if ret == 0 or n.value != L:
print(ret, n.value, GetLastError())
sys.exit(1)
[Thread(target=run).start() for i in range(10)]
--
nosy: +davidsarah
___
Python tracker
<http://bugs.python.org/issue11395>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue410547] os.statvfs support for Windows
David-Sarah Hopwood added the comment: "Is there a portable way to get the available disk space by now?" No, but http://tahoe-lafs.org/trac/tahoe-lafs/browser/src/allmydata/util/fileutil.py?rev=4894#L308 might be helpful (uses pywin32). -- nosy: +davidsarah ___ Python tracker <http://bugs.python.org/issue410547> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
David-Sarah Hopwood added the comment: It is certainly possible to write Unicode to the console successfully using WriteConsoleW. This works regardless of the console code page, including 65001. The code http://tahoe-lafs.org/trac/tahoe-lafs/browser/src/allmydata/windows/fixups.py";>here does so (it's for Python 2.x, but you'd be calling WriteConsoleW from C anyway). WriteConsoleW has one bug that I know of, which is that it http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1232";>fails when writing more than 26608 characters at once. That's easy to work around by limiting the amount of data passed in a single call. Fonts are not Python's problem, but encoding is. It doesn't make sense to fail to output the right characters just because some users might not have selected fonts that can display those characters. This bug should be reopened. (For completeness, it is possible to display Unicode on the console using fonts other than Lucida Console and Consolas, but it http://stackoverflow.com/questions/878972/windows-cmd-encoding-change-causes-python-crash/3259271#3259271";>requires a registry hack.) -- nosy: +davidsarah ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
David-Sarah Hopwood added the comment: Glenn Linderman wrote: > I skipped the unmangling of command-line arguments, because it produced an > error I didn't understand, about needing a buffer protocol. If I understand correctly, that part isn't needed on Python 3 because issue2128 is already fixed there. -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2128] sys.argv is wrong for unicode strings
David-Sarah Hopwood added the comment:
The following code is being used to work around this issue for Python 2.x in
Tahoe-LAFS:
# This works around <http://bugs.python.org/issue2128>.
GetCommandLineW = WINFUNCTYPE(LPWSTR)(("GetCommandLineW", windll.kernel32))
CommandLineToArgvW = WINFUNCTYPE(POINTER(LPWSTR), LPCWSTR, POINTER(c_int)) \
(("CommandLineToArgvW", windll.shell32))
argc = c_int(0)
argv_unicode = CommandLineToArgvW(GetCommandLineW(), byref(argc))
argv = [argv_unicode[i].encode('utf-8') for i in range(0, argc.value)]
if not hasattr(sys, 'frozen'):
# If this is an executable produced by py2exe or bbfreeze, then it will
# have been invoked directly. Otherwise, unicode_argv[0] is the Python
# interpreter, so skip that.
argv = argv[1:]
# Also skip option arguments to the Python interpreter.
while len(argv) > 0:
arg = argv[0]
if not arg.startswith("-") or arg == "-":
break
argv = argv[1:]
if arg == '-m':
# sys.argv[0] should really be the absolute path of the module
source,
# but never mind
break
if arg == '-c':
argv[0] = '-c'
break
--
nosy: +davidsarah
versions: +Python 2.5, Python 2.6, Python 2.7
___
Python tracker
<http://bugs.python.org/issue2128>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2128] sys.argv is wrong for unicode strings
David-Sarah Hopwood added the comment: Sorry, missed out the imports: from ctypes import WINFUNCTYPE, windll, POINTER, byref, c_int from ctypes.wintypes import LPWSTR, LPCWSTR -- ___ Python tracker <http://bugs.python.org/issue2128> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
David-Sarah Hopwood added the comment: haypo wrote: > davidsarah wrote: >> It is certainly possible to write Unicode to the console >> successfully using WriteConsoleW > > Did you tried with characters not encodable to the code page and with > character that cannot be rendeded by the font? Yes, characters not encodable to the code page do work (as confirmed by Glenn Linderman, since code page 437 does not include Cyrillic). Characters that cannot be rendered by the font print as missing-glyph boxes, as expected. They don't cause any other problem, and they can be cut-and-pasted to other Unicode-aware applications, showing up as the original characters. > See msg120414 for my tests with WriteConsoleOutputW Even if it handled encoding correctly, WriteConsoleOutputW (http://msdn.microsoft.com/en-us/library/ms687404%28v=vs.85%29.aspx) would not be the right API to use in any case, because it prints to a rectangle of characters without scrolling. WriteConsoleW does scroll in the same way that printing to a console output stream normally would. (Redirection to a non-console stream can be detected and handled differently, as the code in unicode2.py does.) -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue410547] os.statvfs support for Windows
David-Sarah Hopwood added the comment:
Don't use win32file.GetDiskFreeSpace; the underlying Windows API only supports
drives up to 2 GB
(http://blogs.msdn.com/b/oldnewthing/archive/2007/11/01/5807020.aspx). Use
GetFreeDiskSpaceEx, as the code I linked to does.
I'm not sure it makes sense to provide an exact clone of os.statvfs, since some
of the statvfs fields don't have equivalents that are obtainable by any Windows
API as far as I know. What would make sense is a cross-platform way to
get total disk space, and the space free for root/Administrator and for the
current user. This would actually be somewhat easier to use on Unix as well.
Anyway, here's some code for Windows that only uses ctypes (whichdir should be
Unicode):
from ctypes import WINFUNCTYPE, windll, POINTER, byref, c_ulonglong
from ctypes.wintypes import BOOL, DWORD, LPCWSTR
# <http://msdn.microsoft.com/en-us/library/aa383742%28v=VS.85%29.aspx>
PULARGE_INTEGER = POINTER(c_ulonglong)
# <http://msdn.microsoft.com/en-us/library/aa364937%28VS.85%29.aspx>
GetDiskFreeSpaceExW = WINFUNCTYPE(BOOL, LPCWSTR, PULARGE_INTEGER,
PULARGE_INTEGER, PULARGE_INTEGER)(
("GetDiskFreeSpaceExW", windll.kernel32))
# <http://msdn.microsoft.com/en-us/library/ms679360%28v=VS.85%29.aspx>
GetLastError = WINFUNCTYPE(DWORD)(("GetLastError", windll.kernel32))
# (This might put up an error dialog unless
# SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX)
# has been called.)
n_free_for_user = c_ulonglong(0)
n_total = c_ulonglong(0)
n_free = c_ulonglong(0)
retval = GetDiskFreeSpaceExW(whichdir,
byref(n_free_for_user),
byref(n_total),
byref(n_free))
if retval == 0:
raise OSError("Windows error %d attempting to get disk statistics for
%r"
% (GetLastError(), whichdir))
free_for_user = n_free_for_user.value
total = n_total.value
free = n_free.value
--
versions: +Python 2.7, Python 3.3 -Python 2.6
___
Python tracker
<http://bugs.python.org/issue410547>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
David-Sarah Hopwood added the comment: I'll have a look at the Py3k I/O internals and see what I can do. (Reopening a bug appears to need Coordinator permissions.) -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
David-Sarah Hopwood added the comment: > The script unicode2.py uses the console STD_OUTPUT_HANDLE iff > sys.stdout.fileno()==1. You may have missed "if not_a_console(hStdout): real_stdout = False". not_a_console uses GetFileType and GetConsoleMode to check whether that handle is directed to something other than a console. > But is it always the case? The technique used here for detecting a console is almost the same as the code for IsConsoleRedirected at http://blogs.msdn.com/b/michkap/archive/2010/05/07/10008232.aspx , or in WriteLineRight at http://blogs.msdn.com/b/michkap/archive/2010/04/07/9989346.aspx (I got it from that blog, can't remember exactly which page). [This code will give a false positive in the strange corner case that stdout/stderr is redirected to a console *input* handle. It might be better to use GetConsoleScreenBufferInfo instead of GetConsoleMode, as suggested by http://stackoverflow.com/questions/3648711/detect-nul-file-descriptor-isatty-is-bogus/3650507#3650507 .] > What about pythonw.exe? I just tested that, using pythonw run from cmd.exe with stdout redirected to a file; it works as intended. It also works (for both console and non-console cases) when the handles are inherited from a parent process. Incidentally, what's the earliest supported Windows version for Py3k? I see that http://www.python.org/download/windows/ mentions Windows ME. I can fairly easily make it fall back to never using WriteConsoleW on Windows ME, if that's necessary. -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
David-Sarah Hopwood added the comment: Note: Michael Kaplan's code checks whether GetConsoleMode failed due to ERROR_INVALID_HANDLE. My code intentionally doesn't do that, because it is correct and conservative to fall back to the non-console behaviour when there is *any* error from GetConsoleMode. (It could also fail due to not having the GENERIC_READ right on the handle, for example.) -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5924] When setting complete PYTHONPATH on Python 3.x, paths in the PYTHONPATH are ignored
David-Sarah Hopwood added the comment: Looking at http://svn.python.org/view/python/branches/py3k/PC/getpathp.c?r1=73322&r2=73321&pathrev=73322 , wouldn't it be better to add a Py_WGETENV function? There are likely to be other cases where that would be the correct thing to use. -- nosy: +davidsarah ___ Python tracker <http://bugs.python.org/issue5924> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
David-Sarah Hopwood added the comment:
"... os.dup2() ..."
Good point, thanks.
It would work to change os.dup2 so that if its second argument is 0, 1, or 2,
it calls _get_osfhandle to get the Windows handle for that fd, and then reruns
the console-detection logic. That would even allow Unicode output to work after
redirection to a different console.
Programs that directly called the CRT dup2 or SetStdHandle would bypass this.
Can we consider such programs to be broken? Methinks a documentation patch for
os.dup2 would be sufficient, something like:
"When fd1 refers to the standard input, output, or error handles (0, 1 and 2
respectively), this function also ensures that state associated with Python's
initial sys.{stdin,stdout,stderr} streams is correctly updated if needed. It
should therefore be used in preference to calling the C library's dup2, or
similar APIs such as SetStdHandle on Windows."
--
___
Python tracker
<http://bugs.python.org/issue1602>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
Changes by David-Sarah Hopwood : -- nosy: +BreamoreBoy versions: +Python 3.1, Python 3.2 -Python 3.3 Added file: http://bugs.python.org/file20360/doc-patch.diff ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
Changes by David-Sarah Hopwood : -- nosy: +BreamoreBoy versions: +Python 3.1, Python 3.2 -Python 3.3 Added file: http://bugs.python.org/file20361/doc-patch.diff ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print utf8 (Py30a2)
Changes by David-Sarah Hopwood : Added file: http://bugs.python.org/file20362/doc-patch.diff ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
Changes by David-Sarah Hopwood : -- title: windows console doesn't print utf8 (Py30a2) -> windows console doesn't print or input Unicode Added file: http://bugs.python.org/file20363/doc-patch.diff ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment: Feedback from Julie Solon of Microsoft: > These console functions share a per-process heap that is 64K. There is some > overhead, the heap can get fragmented, and calls from multiple threads all > affect how much is available for this buffer. > I am working to update the documentation for this function [WriteConsoleW] > and other affected functions with information along these lines, and will > post it within the next week or two. I replied thanking her and asking for clarification: When you say that the heap can get fragmented, is this true only when there are concurrent calls to the console functions, or can it occur even with single-threaded use? I'm trying to determine whether acquiring a process-global lock while calling these functions would be sufficient to ensure that the available heap space will not be unexpectedly low. (This assumes that the functions not used outside the lock by other libraries in the same process.) ReadConsoleW seems also to be affected, incidentally. I've asked for clarification about whether acquiring a process-global lock when using these functions ... Julie -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7952] fileobject.c can switch between fread and fwrite without an intervening flush or seek, invoking undefined behaviour
New submission from David-Sarah Hopwood :
The C standard (any version, or POSIX), says in the description of fopen that:
{{{
When a file is opened with update mode ( '+' as the second or third character
in the mode argument), both input and output may be performed on the associated
stream. However, the application shall ensure that output is not directly
followed by input without an intervening call to fflush() or to a file
positioning function ( fseek(), fsetpos(), or rewind()), and input is not
directly followed by output without an intervening call to a file positioning
function, unless the input operation encounters end-of-file.
}}}
Objects/fileobject.c makes calls to fread and fwrite without taking this into
account. So calls from Python to read or write methods of a file object opened
in any "rw" mode, may invoke undefined behaviour. It isn't reasonable to rely
on Python code to avoid this situation, even if were considered acceptable in
C. (Arguably this is a bug in the C standard, but it is unlikely to be fixed
there or in POSIX, because of differences in philosophy about language safety.)
To fix this, fileobject.c should keep track of whether the last I/O operation
was an input or output, and perform a call to fflush whenever an input follows
an output or vice versa. This should not significantly affect performance in
any case where the behaviour was previously defined (in cases where it wasn't,
correctness trumps performance). fflush does not affect the file position and
should have no other negative effect, because the stdio implementation is free
to flush buffered data at any time (and certainly on I/O operations).
Despite the undefined behaviour, I don't currently know of a platform where
this would lead to an exploitable security bug. I'm marking this issue as
security-relevant anyway, because it may prevent analysing whether Python
applications behave securely only on the basis of documented behaviour.
--
components: IO
messages: 99483
nosy: davidsarah
severity: normal
status: open
title: fileobject.c can switch between fread and fwrite without an intervening
flush or seek, invoking undefined behaviour
type: security
versions: Python 2.7
___
Python tracker
<http://bugs.python.org/issue7952>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7952] fileobject.c can switch between fread and fwrite without an intervening flush or seek, invoking undefined behaviour
David-Sarah Hopwood added the comment: Correction: when input is followed by output, the call needed to avoid undefined behaviour has to be to a "file positioning function" (fseek, fsetpos, or rewind, but not fflush). Since fileobject.c does not use wide I/O operations, it should be sufficient to use _portable_fseek(fp, 0, SEEK_SET). (_portable_fseek may call some function that is not strictly defined to be a "file positioning function", e.g. fseeko() or fseek64(). However, it would be insane for a stdio implementation not to treat those as being file positioning functions as far as the intent of the C or POSIX standards is concerned.) -- ___ Python tracker <http://bugs.python.org/issue7952> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1602] windows console doesn't print or input Unicode
David-Sarah Hopwood added the comment: Giampaolo: See #msg120700 for why that won't work, and the subsequent comments for what will work instead (basically, using WriteConsoleW and a workaround for a Windows API bug). Also see the prototype win_console.patch from Victor Stinner: #msg145963 -- ___ Python tracker <http://bugs.python.org/issue1602> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
