[issue36193] Redirected stderr not reset properly when using logging
New submission from Andrius Laukavičius :
It looks like logging library uses always first assigned stderr object and
won't change it even if it was reset.
capture_output function redirects stdout and stderr to io.StringIO object,
saves what was captured in string and returns it. And then context manager
resets back stdout and stderr where it should be originally. Though it seems
logging library ignores that.
Here is the code I encountered issue with:
import io
import sys
import contextlib
from typing import Optional
import logging
def capture_output(
target: object,
args: Optional[tuple] = None,
kwargs: Optional[dict] = None) -> str:
"""Redirect stdout and stderr into string buffer and capture it.
target object is executed with optional args, kwargs and all stdout/
stderr that is captured, returned in string form.
Args:
target: object to execute, usually a function.
args: target positional arguments (default: {None})
kwargs: target keyword arguments (default: {None})
"""
if not args:
args = ()
if not kwargs:
kwargs = {}
with io.StringIO() as sio:
with contextlib.redirect_stdout(sio), contextlib.redirect_stderr(sio):
target(*args, **kwargs)
output = sio.getvalue()
print(output)
def dummy():
print('dummy test')
logging.warning('dummy test')
def dummy2():
print('dummy2 test')
capture_output(dummy) # works
capture_output(dummy) # won't work anymore.
capture_output(dummy2) # works
capture_output(dummy2) # works
Here is what I get running this code:
dummy test
WARNING:root:dummy test
dummy test
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.5/logging/__init__.py", line 982, in emit
stream.write(msg)
ValueError: I/O operation on closed file
Call stack:
File "tt.py", line 43, in
capture_output(dummy) # won't work anymore.
File "tt.py", line 28, in capture_output
target(*args, **kwargs)
File "tt.py", line 36, in dummy
logging.warning('dummy test')
Message: 'dummy test'
Arguments: ()
dummy2 test
dummy2 test
P.S. here is original question I asked on stack overflow:
https://stackoverflow.com/questions/54999650/python3-redirected-stderr-not-reset-properly.
I got suggestion to report it as a bug.
--
components: Library (Lib)
messages: 337178
nosy: Andrius Laukavičius
priority: normal
severity: normal
status: open
title: Redirected stderr not reset properly when using logging
type: behavior
versions: Python 3.5
___
Python tracker
<https://bugs.python.org/issue36193>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36193] Redirected stderr not reset properly when using logging
Andrius Laukavičius added the comment: Peter Otten thanks for examples, though with capture_output function, I had a little bit different intentions. At first I actually did similarly as you described in case 2, where I called BasicConfig explicitly. Then I could specify where to redirect stream and what to do after I was done with capturing stderr (just reset back where it would be on default). Though I actually wanted to just capture stdout and stderr indirectly, meaning I don't want to specifically initiate logging object or modify it. I mean, I want to just capture any stdout/stderr that would be outputed while running some function that was passed via `target` argument (in my capture_output function). In this case, some function that I capture output from, might have some different way to handle sys.stderr (e.g. different basicConfig or some other implementation where sys.stderr is handled?). So it is not possible to consistently manage stderr when it involves logging library without explicitly "manage" it? P.S. don't know if I'm clear enough, so please ask me to clarify anything if its not clear..:) -- ___ Python tracker <https://bugs.python.org/issue36193> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36193] Redirected stderr not reset properly when using logging
Andrius Laukavičius added the comment: Peter, well I don't have deep experience with logging lib to know all ins and outs. So I might not be correct person to define what should be changed specifically. But I think it should be consistent on all standard libraries how it handles stdout/stderr, unless there is a reason to do it differently? For example `print` to me seems to behave how I expect, because if stdout is redirected elsewhere, print uses redirected stdout without a problem. Now with logging, it does feel strange that on first call of `logging`, it sets internally stderr where it is pointing at that moment and forgets about it. You call logging again, and it naively expects stderr to be where it was on initial call, when in reality it might be redirected elsewhere (as in my case). Maybe you know the reasoning why logging even needs to behave this way? Though if logging behaves this way, why print behaves differently? So to summarize this, I think all libraries should respect stdout/stderr where it is and have some unified way/interface (like I expected redirect_stdout/redirect_stderr to be) to specify that stdout/stderr has changed and related libraries must update their state regarding that (logging library case) or simply rely on where stdout/stderr is pointed, so nothing needs to be done on that library once stdout/stderr is redirected. To comment on your view about `dump_stuff(dest)`. Well I think this suits well, when you have something to dump already, but what if something to dump is what is produced via stdout/stderr and you need to capture it first? How can I do that without first redirecting stdout where I need to? -- ___ Python tracker <https://bugs.python.org/issue36193> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36193] Redirected stderr not reset properly when using logging
Andrius Laukavičius added the comment: Ok. Thanks for the clarification. On Thu, Mar 7, 2019, 08:13 Vinay Sajip wrote: > > Vinay Sajip added the comment: > > The StreamHandler allows you to specify *exactly* which stream you want to > use to log to; sys.stderr is provided as a convenient default argument > because that's what a lot of people want a lot of the time. > > This is typically done at logging configuration time, or whenever a > StreamHandler is created. This is done implicitly by your logging.warning() > call (as documented, this calls logging.basicConfig(), which adds a > StreamHandler using whatever sys.stderr is set to at the time the > StreamHandler is instantiated). Also documented is that basicConfig() is > only effective once (it will not do anything if a handler has already been > added to the root logger - it is only meant to be use for simple one-off > scripts). The documentation for basicConfig() is clear: > > "Does basic configuration for the logging system by creating a > StreamHandler with a default Formatter and adding it to the root logger. > The functions debug(), info(), warning(), error() and critical() will call > basicConfig() automatically if no handlers are defined for the root logger. > > This function does nothing if the root logger already has handlers > configured for it." > > If you want to use the real console streams, don't use logging.warning(), > but instead explicitly call basicConfig() using __stderr__, as Peter says. > Alternatively, use the approach suggested in the cookbook for > context-sensitive logging: > > > https://docs.python.org/3/howto/logging-cookbook.html#using-a-context-manager-for-selective-logging > > Closing, as this is not a bug in logging. > > -- > resolution: -> not a bug > stage: -> resolved > status: open -> closed > > ___ > Python tracker > <https://bugs.python.org/issue36193> > ___ > -- ___ Python tracker <https://bugs.python.org/issue36193> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
