Re: [Python-Dev] Regarding stdlib socket module, _fileobject.flush() method using ._rbufsize instead of ._wbufsize
On Wed, 16 Oct 2013 21:30:52 -0400 Terry Reedy wrote: > On 10/16/2013 5:01 PM, Peter Portante wrote: > > Hello, > > > > Is there a reason why the stdlib socket module _fileobject.flush() > > method is using ._rbufsize instead of ._wbufsize at line 297 (Python > > 2.7.3), where it determines the buffer_size value to be used for > > _sock.sendall()? Does anybody know the history behind this? > > Annotations and history show that line is from > 61875, 2010-5-23, J P Calderone, Merged revisions 74426... > I do not understand the message. When mentioning Mercurial changesets, please post the (hexadecimal) changeset id, not the revision number, since revision numbers are not stable accross clones. http://mercurial.selenic.com/wiki/RevisionNumber Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Bug tracker and docs@python
Good day! What is the effect of assigning a bug tracker issue to docs@python? -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().
On 17 October 2013 01:24, Barry Warsaw wrote: > On Oct 16, 2013, at 08:31 AM, Eric Snow wrote: > >>When a module's maintainer makes a decision on a relatively insignificant >>addition to the module, I'd expect little resistance or even comment (the >>original commit was months ago). That's why I'm surprised by the reaction >>to this change. It just seems like the whole thing is being blown way out >>of proportion to the detriment of other interesting problems. Sure, things >>could have been done differently. But in this case it's not that big a >>deal. > > Any project as big and diverse as Python needs a hierarchical structure of > trust and responsibility. I see it *roughly* as core dev < module maintainer > < release manager < bdfl delegate < bdfl. > > However, it's imperative to remain vigilantly transparent so that everyone > understands the rationale and motivation behind a change, even if they > disagree with it. Trust is extended upwards when this transparency is > extended downloads. "'Cause I said so" only works at the top of the chain. ;) Right, and that's why I never opted out of the thread entirely, despite being seriously tempted at times :) It's also caused me to reflect on a few things over the past few days, including why the bar for contextlib is (at this point in time) comparatively lower than the one for functools, collections and itertools. I was also asked a very good question off-list as to why TransformDict escalated to needing a PEP, while I still believe a couple of issues is adequate for keeping this minor addition to contextlib (for the record: the patch Zero posted to http://bugs.python.org/issue19266 has been committed, so the pattern is now called "suppress"). An interesting lightning talk at BrisPy last night also included a key point: the speaker spent a lot of time creating custom internal tools for process engineers to use as an alternative to getting expensive Matlab licenses, and his experience was that object-oriented programming was *just* beyond what most process engineers could cope with in terms of writing their own code. If you look at the history of various constructs in Python, it takes years for idioms and patterns to build up around them, even after the core syntax additions is made to the language. Once those idioms are in place, though, you get to a point where you can first teach people how to use them effectively and only later teach them how to build their own. Because the patterns have been enshrined in things with *names*, you also don't need to be able to recognise them on sight - you have something to look up the first you set, rather than having to interpret a particular shape of code as indicating a particular idiom. This pattern recurs again and again: - you learn how to call functions before you learn how to write them - you learn how to instantiate and use classes before you learn how to write new ones - you learn how to import modules before you learn how to write your own - you learn how to iterate before you learn how to write iterators - you learn how to apply decorators before you learn how to build them - you learn how to invoke context managers before you learn how to write your own - you often don't even have to learn how to use descriptors per se, as it's often hidden in learning to use decorators. Learning to create your own then lets you realise how much of what you thought you knew about classes is actually just based on the way a few standard descriptors work. - you also often don't even have to learn how to use metaclasses, as it's often hidden behind class inheritance (e.g. from abc.ABC or enum.Enum, or from a base class in an ORM or web framework). Learning to create your own metaclasses, on the other hand, can be completely mindbending. Over time, this progressively lowers the barrier to entry for Python programming, as the intent is to enable users to *do* more without necessarily needing to learn (much) more, as well as allowing code authors to more clearly communicate their *intent* to code readers by using *named* patterns rather than leaving them implicit in the code structure. Exception handling, however, is a notable exception(!) to that general pattern. Appropriately *handling* exceptions is actually harder than raising them: raise Exception vs: try: except Exception: The introduction of context managers several years ago basically provides an opportunity to change that over time: by standardising various common try/except idioms, it becomes feasible to postpone the explanations of the underlying details. Compared to the extensive toolkits we already have for functional abstractions, iterator based abstractions and data containers, the general purpose context manipulation idioms are still quite weak. contextlib.closing was part of the initial toolkit in Python 2.5, while contextlib.nested was later removed as fundamentally broken in the presence of resources that acquire t
Re: [Python-Dev] Bug tracker and docs@python
2013/10/17 Ethan Furman : > Good day! > > What is the effect of assigning a bug tracker issue to docs@python? The assignee field is set to d...@python.org. In other words, nothing magical happens. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #16129: Add `Py_SetStandardStreamEncoding`
On Thu, 17 Oct 2013 15:22:03 +0200 (CEST) nick.coghlan wrote: > > +.. c:function:: int Py_SetStandardStreamEncoding(char *encoding, char > *errors) > + > + .. index:: > + single: Py_Initialize() > + single: main() > + triple: stdin; stdout; sdterr > + > + This function should be called before :c:func:`Py_Initialize`. It > + specifies which encoding and error handling to use with standard io, > + with the same meanings as in :func:`str.encode`. > + > + It overrides :envvar:`PYTHONIOENCODING` values, and allows embedding code > + to control io encoding when the environment variable does not work. > + > + ``encoding`` and/or ``errors`` may be NULL to use > + :envvar:`PYTHONIOENCODING` and/or default values (depending on other > + settings). > + > + Note that :data:`sys.stderr` always uses the "backslashreplace" error > + handler, regardless of this (or any other) setting. > + > + If :c:func:`Py_Finalize` is called, this function will need to be called > + again in order to affect subsequent calls to :c:func:`Py_Initialize`. > + > + Returns 0 if successful. > + > + Needs a versionadded tag. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Try doing a raw test of os.fork()/os.kill().
On Thu, 17 Oct 2013 15:33:02 +0200 (CEST) richard.oudkerk wrote: > http://hg.python.org/cpython/rev/9558e9360afc > changeset: 86401:9558e9360afc > parent: 86399:9cd88b39ef62 > user:Richard Oudkerk > date:Thu Oct 17 14:24:06 2013 +0100 > summary: > Try doing a raw test of os.fork()/os.kill(). For this kind of ad-hoc testing, you can also use a custom builder to avoid disrupting the main source tree: http://docs.python.org/devguide/buildbots.html#custom-builders Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug tracker and docs@python
Am 17.10.2013 17:29, schrieb Benjamin Peterson: > 2013/10/17 Ethan Furman : >> Good day! >> >> What is the effect of assigning a bug tracker issue to docs@python? > > The assignee field is set to d...@python.org. > > In other words, nothing magical happens. Correct so far, but the point of the exercise is that the mailing list d...@python.org is now nosy and will get updates for the issue. Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On Oct 18, 2013, at 01:26 AM, Nick Coghlan wrote: >By contrast, suppress() and redirect_stdout() are the *first* general >purpose context managers added to contextlib since its incarnation in >Python 2.5 (although there have been many various domain specific >context manager additions elsewhere in the standard library). There's a fundamental conceptual shift here that's worth exploring more, and which I think was first identified by RDM. Until now, context managers were at their heart (at least IMHO) about managing "resources". A general resource might be an open file, or it might be a database transaction, or even the current working directory. Context managers (as expressed elegantly by the `with` statement) are used to ensure that a resource acquired for some limited operation is - to Python's best ability - "released" at the end of that operation, no matter what happens. E.g. the file is closed even if there's a write error, or the current working directory is restored to its original location. We need only look at the typical @contextmanager use to see the idiom they embody. As shown in the docstring: @contextmanager def acquire(): resource = get_some_resource() try: yield # execute the operation finally: # No matter what happened above... resource.free() redirect_stdout() conforms to this fine tradition, with the resource being sys.stdout. suppress() breaks the mold, which I think is what is breaking people's brains. It isn't about guaranteeing that a resource is restored to its original value after some operation. It's about short circuiting that operation. Just look at the implementation to see this shift in perspective. It doesn't use try/finally, it uses try/except. So it's important to acknowledge that suppress() is charting new territory and it will take some exploration and experimentation to get used to, or maybe even to decide whether it's a good idea. It'll be interesting to see whether this fundamental difference is easily explained, understood, and internalized and that will go a long way to figuring out whether this is a good idea to be expanded on in the future. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Help needed: problems getting the expected default encoding for embedded IO config test
http://bugs.python.org/issue16129 added a Py_SetStandardStreamEncoding API to allow embedding applications like Blender to explicitly configure the encoding and error handler for the standard IO streams. However, the test case didn't get along with the buildbots, and I'm struggling to figure out why. To deal with the possible variation in the default encoding across systems, I use sys.std*.encoding to work out the expected default encoding for the streams. However, as you can see in an example like http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.0%203.x/builds/579/steps/test/logs/stdio, this is failing, with the encoding in the *expected* data being None, while the values from the subprocess actually look right. In the "re-run at the end" case, it all works fine, so it doesn't appear to be anything inherently wrong with the affected systems. The test pass without any problems on my own system (otherwise I wouldn't have checked it in). Running the full test suite the same way the buildbots do also worked fine locally. I've marked the offending test as skipped for now (and reopened the tracker issue), and will look into it more tomorrow. but figured I'd mention it here in case anyone had any good suggestions (the next thing on my to try list is running with the hash seed from one of the failing buildbots, in case it's a cross-test interference problem). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Try doing a raw test of os.fork()/os.kill().
2013/10/17 Antoine Pitrou : > On Thu, 17 Oct 2013 15:33:02 +0200 (CEST) > richard.oudkerk wrote: >> http://hg.python.org/cpython/rev/9558e9360afc >> changeset: 86401:9558e9360afc >> parent: 86399:9cd88b39ef62 >> user:Richard Oudkerk >> date:Thu Oct 17 14:24:06 2013 +0100 >> summary: >> Try doing a raw test of os.fork()/os.kill(). > > For this kind of ad-hoc testing, you can also use a custom builder to > avoid disrupting the main source tree: AFAICT, the problem he's trying to debug (issue #19227) only occurs on two specific - stable - buildbots. cf ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Rename contextlib.ignored() to contextlib.ignore().
On 10/17/2013 08:26 AM, Nick Coghlan wrote: # Arbitrarily nested search loop with exit_label() as found: for i in range(x): for j in range(y): if matches(i, j): found.exit((i, j)) if found: print(found.value) +1 ! @property def exc(self): if self._exc is self._exc: typo Thanks for this, I like it! :) -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug tracker and docs@python
On 10/17/2013 08:29 AM, Benjamin Peterson wrote: 2013/10/17 Ethan Furman : Good day! What is the effect of assigning a bug tracker issue to docs@python? The assignee field is set to d...@python.org. In other words, nothing magical happens. Heh, okay. So commit the doc patch myself, then. :) So when should I assign something to docs@python? -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug tracker and docs@python
On Thu, 17 Oct 2013 08:52:42 -0700 Ethan Furman wrote: > On 10/17/2013 08:29 AM, Benjamin Peterson wrote: > > 2013/10/17 Ethan Furman : > >> Good day! > >> > >> What is the effect of assigning a bug tracker issue to docs@python? > > > > The assignee field is set to d...@python.org. > > > > In other words, nothing magical happens. > > Heh, okay. So commit the doc patch myself, then. :) > > So when should I assign something to docs@python? You don't. You add the issue to the "Documentation" category and it will get assigned automatically. But anybody can of course commit doc patches. (d...@python.org is a mailing-list, AFAIK) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Try doing a raw test of os.fork()/os.kill().
On 17/10/2013 5:35pm, Charles-François Natali wrote: For this kind of ad-hoc testing, you can also use a custom builder to avoid disrupting the main source tree: AFAICT, the problem he's trying to debug (issue #19227) only occurs on two specific - stable - buildbots. It looks like there are corresponding custom builders, so I will try using those. -- Richard ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Help needed: problems getting the expected default encoding for embedded IO config test
sys.stdout.encoding must never be None, it must be a str. If it is None, it is a regression. I modified Python (maybe in 3.2) to ensure that .encoding is always set. For your failure: what is the locale encoding? What are the values of LC_ALL, LANG, LC_CTYPES and PYTHONIOENCODING env vars? I will try to reproduce the issue. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Regarding stdlib socket module, _fileobject.flush() method using ._rbufsize instead of ._wbufsize
So that's hex rev dca3cb19dbdd. I'm confident that it is a copy/paste bug. It doesn't have any effect AFAICT though, since given the analysis the OP posted, max(self._wbufsize, self.bufsize) and max(self._rbufsize, self.bufsize) always come out to the same. Additionally, the bug is no longer present in Python 3. On Thu, Oct 17, 2013 at 3:23 AM, Antoine Pitrou wrote: > On Wed, 16 Oct 2013 21:30:52 -0400 > Terry Reedy wrote: > > > On 10/16/2013 5:01 PM, Peter Portante wrote: > > > Hello, > > > > > > Is there a reason why the stdlib socket module _fileobject.flush() > > > method is using ._rbufsize instead of ._wbufsize at line 297 (Python > > > 2.7.3), where it determines the buffer_size value to be used for > > > _sock.sendall()? Does anybody know the history behind this? > > > > Annotations and history show that line is from > > 61875, 2010-5-23, J P Calderone, Merged revisions 74426... > > I do not understand the message. > > When mentioning Mercurial changesets, please post the > (hexadecimal) changeset id, not the revision number, since revision > numbers are not stable accross clones. > http://mercurial.selenic.com/wiki/RevisionNumber > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 10/17/2013 9:06 AM, Barry Warsaw wrote: There's a fundamental conceptual shift here that's worth exploring more, and which I think was first identified by RDM. Until now, context managers were at their heart (at least IMHO) about managing "resources". A general resource might be an open file, or it might be a database transaction, or even the current working directory. Context managers (as expressed elegantly by the `with` statement) are used to ensure that a resource acquired for some limited operation is - to Python's best ability - "released" at the end of that operation, no matter what happens. E.g. the file is closed even if there's a write error, or the current working directory is restored to its original location. The conceptual shift exists, as you two have pointed out. But that is mostly because Structured programming and OO have created sufficient conceptual idioms that people have forgotten that all of these idioms are to avoid bare gotos. The real purpose of all such constructs is to manage the instruction pointer, and context managers, while useful for managing "resources" in the abstract, are truly about managing the control flow -- hopefully in understandable ways. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 2013-10-17, at 18:06 , Barry Warsaw wrote: > On Oct 18, 2013, at 01:26 AM, Nick Coghlan wrote: >> By contrast, suppress() and redirect_stdout() are the *first* general >> purpose context managers added to contextlib since its incarnation in >> Python 2.5 (although there have been many various domain specific >> context manager additions elsewhere in the standard library). > > There's a fundamental conceptual shift here that's worth exploring more, and > which I think was first identified by RDM. > > Until now, context managers were at their heart (at least IMHO) about managing > "resources". A general resource might be an open file, or it might be a > database transaction, or even the current working directory. Context managers > (as expressed elegantly by the `with` statement) are used to ensure that a > resource acquired for some limited operation is - to Python's best ability - > "released" at the end of that operation, no matter what happens. E.g. the > file is closed even if there's a write error, or the current working directory > is restored to its original location. I think there's already a significant split between context managers which handle the lifecycle of a local resource (file, transaction) and those which purport to locally alter global-ish state (cwd, decimal.localcontext, logging.captureWarnings, redirect_stdout). And the latter worries me (much more than the very localized behavior of suppress) because I don't see any way to implement them safely and correctly when mixing it with coroutines in today's Python (some of them aren't even thread-safe), all of that while I expect coroutines will see significantly more use in the very near future with yield from and tulip's promotion of coroutine-style async. > Just look at the implementation to see this shift in perspective. It doesn't > use try/finally, it uses try/except. Transaction CMs will also use try/except: @contextmanager def transaction(): resource = open_tnx() try: yield resource resource.commit() except: resource.rollback() raise ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On Oct 17, 2013, at 08:40 PM, Xavier Morel wrote: >I think there's already a significant split between context managers >which handle the lifecycle of a local resource (file, transaction) and >those which purport to locally alter global-ish state (cwd, >decimal.localcontext, logging.captureWarnings, redirect_stdout). > >And the latter worries me (much more than the very localized behavior of >suppress) because I don't see any way to implement them safely and >correctly when mixing it with coroutines in today's Python (some of them >aren't even thread-safe), all of that while I expect coroutines will see >significantly more use in the very near future with yield from and >tulip's promotion of coroutine-style async. Although slightly orthogonal to my previous distinction, this is a great point. Anything affecting global state will have this problem of course. It's definitely something to keep thinking about. >> Just look at the implementation to see this shift in perspective. It >> doesn't use try/finally, it uses try/except. > >Transaction CMs will also use try/except: > >@contextmanager >def transaction(): >resource = open_tnx() >try: >yield resource >resource.commit() >except: >resource.rollback() >raise True. I'm sure lots of us have code almost exactly like this. ;) For me, it's closer to the original intent though because the bare-except combined with the re-raising of the exception feels kind of like a finally. In both exit paths, the resource is being released, it's just that you have to know which "release" operation to perform. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 17 October 2013 19:40, Xavier Morel wrote: > I think there's already a significant split between context managers > which handle the lifecycle of a local resource (file, transaction) and > those which purport to locally alter global-ish state (cwd, > decimal.localcontext, logging.captureWarnings, redirect_stdout). > > And the latter worries me (much more than the very localized behavior of > suppress) because I don't see any way to implement them safely and > correctly when mixing it with coroutines in today's Python (some of them > aren't even thread-safe), all of that while I expect coroutines will see > significantly more use in the very near future with yield from and > tulip's promotion of coroutine-style async. I maybe misunderstanding how the coroutine-style async works but I would have thought that it would be as simple as: don't use global-state-restoring-context-managers around statements that yield control (it would be simpler if there was a good term for describing that kind of CM). That's simpler to implement and computationally cheaper than e.g. the thread-local state used by the decimal module. Oscar ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On Thu, Oct 17, 2013 at 11:55 AM, Oscar Benjamin wrote: > On 17 October 2013 19:40, Xavier Morel wrote: > > I think there's already a significant split between context managers > > which handle the lifecycle of a local resource (file, transaction) and > > those which purport to locally alter global-ish state (cwd, > > decimal.localcontext, logging.captureWarnings, redirect_stdout). > > > > And the latter worries me (much more than the very localized behavior of > > suppress) because I don't see any way to implement them safely and > > correctly when mixing it with coroutines in today's Python (some of them > > aren't even thread-safe), all of that while I expect coroutines will see > > significantly more use in the very near future with yield from and > > tulip's promotion of coroutine-style async. > > I maybe misunderstanding how the coroutine-style async works but I > would have thought that it would be as simple as: don't use > global-state-restoring-context-managers around statements that yield > control (it would be simpler if there was a good term for describing > that kind of CM). That's simpler to implement and computationally > cheaper than e.g. the thread-local state used by the decimal module. > Context managers that actually save and restore *global* state are already not thread-safe, so concluding they are also not coroutine-safe (or task-safe?) seems a small step. I'd be more worried about context manager that use thread-local state -- there is no similar concept in Tulip. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 2013-10-17, at 20:55 , Oscar Benjamin wrote: > On 17 October 2013 19:40, Xavier Morel wrote: >> I think there's already a significant split between context managers >> which handle the lifecycle of a local resource (file, transaction) and >> those which purport to locally alter global-ish state (cwd, >> decimal.localcontext, logging.captureWarnings, redirect_stdout). >> >> And the latter worries me (much more than the very localized behavior of >> suppress) because I don't see any way to implement them safely and >> correctly when mixing it with coroutines in today's Python (some of them >> aren't even thread-safe), all of that while I expect coroutines will see >> significantly more use in the very near future with yield from and >> tulip's promotion of coroutine-style async. > > I maybe misunderstanding how the coroutine-style async works but I > would have thought that it would be as simple as: don't use > global-state-restoring-context-managers around statements that yield > control You have to know which contextmanagers to what and how, and avoid them in these specific situations. I'm really bothered by these being unsafe by default. Technically they're already broken but the chance of them being used in such a context are low, whereas it wouldn't be unexpected for a user to e.g. create a local decimal context in a coroutine and *not* expect that local context to affect other coroutines running concurrently. > Context managers that actually save and restore *global* state are already > not thread-safe Hence my use of "global-ish", a global is process-local after all. A threadlocal is greenlet-global (for the greenlets running in that thread), and a greenlet-local is coroutine-global (although I don't expect multiplexing coroutines inside greenlets to be common, the natural consequence is that threadlocals are also coroutine-global). > , so concluding they are also not coroutine-safe (or > task-safe?) seems a small step. Yes, but not necessarily a step most users will remember to take, and of course the lack of thread-safety must be documented and noticed (as with warnings.catch_warnings[0]). Although I agree that > I'd be more worried about context manager that use thread-local state -- > there is no similar concept in Tulip. indeed. [0] not logging.captureWarnings() which I previously wrote, captureWarnings is not a context manager ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 10/17/2013 12:06 PM, Barry Warsaw wrote: On Oct 18, 2013, at 01:26 AM, Nick Coghlan wrote: By contrast, suppress() and redirect_stdout() are the *first* general purpose context managers added to contextlib since its incarnation in Python 2.5 (although there have been many various domain specific context manager additions elsewhere in the standard library). There's a fundamental conceptual shift here that's worth exploring more, I noticed the same thing upon reading that. With statements are a general special-purpose tool. and which I think was first identified by RDM. I missed this at the time. Until now, context managers were at their heart (at least IMHO) about managing "resources". ... We need only look at the typical @contextmanager use to see the idiom they embody. As shown in the docstring: @contextmanager def acquire(): resource = get_some_resource() try: yield # execute the operation finally: # No matter what happened above... resource.free() redirect_stdout() conforms to this fine tradition, with the resource being sys.stdout. From the with statement doc, second sentence: "This allows common try...except...finally usage patterns to be encapsulated for convenient reuse." suppress() breaks the mold, which I think is what is breaking people's brains. It isn't about guaranteeing that a resource is restored to its original value after some operation. It's about short circuiting that operation. A suppress() is, in a sense, an empty cm as its only effect is to exploit the following: "If the suite was exited due to an exception, and the return value from the __exit__() method was false, the exception is reraised. If the return value was true, the exception is suppressed, and execution continues with the statement following the with statement." This is clear with a direct implementation instead of the magical indirect one that Nick used (with @contextmanager and a generator function). I believe the following is equivalent, and one line shorter. class suppress: def __init__(self, *exceptions): self.exceptions = exceptions def __exit__(self, etype, eval, etrace): return etype in self.exceptions One might consider this an abuse, like comprehensions written purely for their side effect. Just look at the implementation to see this shift in perspective. It doesn't use try/finally, it uses try/except. So it's important to acknowledge that suppress() is charting new territory and it will take some exploration and experimentation to get used to, or maybe even to decide whether it's a good idea. It'll be interesting to see whether this fundamental difference is easily explained, understood, and internalized and that will go a long way to figuring out whether this is a good idea to be expanded on in the future. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
Xavier Morel wrote: > On 2013-10-17, at 20:55 , Oscar Benjamin wrote: > > On 17 October 2013 19:40, Xavier Morel wrote: > >> I think there's already a significant split between context managers > >> which handle the lifecycle of a local resource (file, transaction) and > >> those which purport to locally alter global-ish state (cwd, > >> decimal.localcontext, logging.captureWarnings, redirect_stdout). Well, naming is difficult. In the case of decimal it's a temporary global but also thread-local context. > You have to know which contextmanagers to what and how, and avoid them > in these specific situations. I'm really bothered by these being unsafe > by default. Technically they're already broken but the chance of them > being used in such a context are low, whereas it wouldn't be unexpected > for a user to e.g. create a local decimal context in a coroutine and > *not* expect that local context to affect other coroutines running > concurrently. I don't think localcontext() is broken. The fine manual is quite clear that "current contexts" are per thread. The localcontext() docs explicitly say that the "current context" is swapped. Of course there are multiple ways to use individual contexts for each coroutine. Use the context methods or pass the context to the Decimal methods. Stefan Krah ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On Thu, Oct 17, 2013 at 9:06 AM, Barry Warsaw wrote:
> On Oct 18, 2013, at 01:26 AM, Nick Coghlan wrote:
>
> >By contrast, suppress() and redirect_stdout() are the *first* general
> >purpose context managers added to contextlib since its incarnation in
> >Python 2.5 (although there have been many various domain specific
> >context manager additions elsewhere in the standard library).
>
> There's a fundamental conceptual shift here that's worth exploring more,
> and
> which I think was first identified by RDM.
>
> Until now, context managers were at their heart (at least IMHO) about
> managing
> "resources". A general resource might be an open file, or it might be a
> database transaction, or even the current working directory. Context
> managers
> (as expressed elegantly by the `with` statement) are used to ensure that a
> resource acquired for some limited operation is - to Python's best ability
> -
> "released" at the end of that operation, no matter what happens. E.g. the
> file is closed even if there's a write error, or the current working
> directory
> is restored to its original location.
>
> We need only look at the typical @contextmanager use to see the idiom they
> embody. As shown in the docstring:
>
> @contextmanager
> def acquire():
> resource = get_some_resource()
> try:
> yield # execute the operation
> finally:
> # No matter what happened above...
> resource.free()
>
> redirect_stdout() conforms to this fine tradition, with the resource being
> sys.stdout.
>
> suppress() breaks the mold, which I think is what is breaking people's
> brains. It isn't about guaranteeing that a resource is restored to its
> original value after some operation. It's about short circuiting that
> operation.
>
> Just look at the implementation to see this shift in perspective. It
> doesn't
> use try/finally, it uses try/except.
>
> So it's important to acknowledge that suppress() is charting new territory
> and
> it will take some exploration and experimentation to get used to, or maybe
> even to decide whether it's a good idea. It'll be interesting to see
> whether
> this fundamental difference is easily explained, understood, and
> internalized
> and that will go a long way to figuring out whether this is a good idea to
> be
> expanded on in the future.
>
Is this shift to do non-resourcey-things really new?
In 2009 when doing *great things* to the unittest module for the upcoming
Python 2.7 we (michael foord I believe) turned the previous None return
value of assertRaises into a context manager so that it could be used in a
non-resource-management manner similar to:
with self.assertRaises(SomeError) as context:
state = set_something_up()
call_offending_thing_on_it(state)
self.assertIn('Bloody Murder!', context.exception.msg)
I thought this was great. As did a bunch of other people in the room. Thus
we have it and happily use it.
It certainly is _not at all_ how I had ever been thinking of context
managers before this feature was added. I had pigeon holed them into
resource management tasks in my mind rather than thinking of them as a
replacement for try..except syntax in common cases.
-gps
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 10/17/2013 01:03 PM, Terry Reedy wrote: class suppress: def __init__(self, *exceptions): self.exceptions = exceptions def __exit__(self, etype, eval, etrace): return etype in self.exceptions This fails when etype is a subclass of the exceptions, as mentioned in the original issue. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 2013-10-17, at 22:11 , Ethan Furman wrote: > On 10/17/2013 01:03 PM, Terry Reedy wrote: >> >> class suppress: >> def __init__(self, *exceptions): >> self.exceptions = exceptions >> def __exit__(self, etype, eval, etrace): >> return etype in self.exceptions > > This fails when etype is a subclass of the exceptions, as mentioned in the > original issue. That's fixed by using `issubclass` and does not infirm Terry's point does it? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 10/17/2013 01:57 PM, Xavier Morel wrote: On 2013-10-17, at 22:11 , Ethan Furman wrote: On 10/17/2013 01:03 PM, Terry Reedy wrote: class suppress: def __init__(self, *exceptions): self.exceptions = exceptions def __exit__(self, etype, eval, etrace): return etype in self.exceptions This fails when etype is a subclass of the exceptions, as mentioned in the original issue. That's fixed by using `issubclass` and does not infirm Terry's point does it? Nope, and thanks for posting the correct verbage. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] non-US zip archives support in zipfile.py
On Tue, Oct 15, 2013 at 03:46:15PM +0200, "Martin v. Löwis" wrote: > Am 15.10.13 14:49, schrieb Daniel Holth: > > It is part of the ZIP specification. CP437 or UTF-8 are the two > > official choices, but other encodings happen on Russian, Japanese > > systems. > > Indeed. Formally, the other encodings are not supported by the > ZIP specification, and are thus formally misuse of the format. > But the tools in the wild misuse the format in this manner. CP437 can encode any byte so zip and unzip on Linux, for instance, take the bytes that represent the filename on the filesystem and use those in the zip file without setting the utf-8 flag. When the files are extracted, the same byte sequence are used as the filename for the new files. > I believe (without having proof) that early versions of the > specification failed to discuss the file name encoding at all, > These might be helpful: No mention of file name encodings in this version of the spec: http://www.pkware.com/documents/APPNOTE/APPNOTE-6.2.2.TXT Appendix D, Language Encoding, shows up here: http://www.pkware.com/documents/APPNOTE/APPNOTE-6.3.0.TXT (Most recent version is 6.3.2) > making people believe that it is unspecified and always the > system encoding (which is useless, of course, as you create > zip files to move them across systems). > Not always. Backups are another use. Also it's not useless. If the files are being moved within an organization (or in some cases geographical regions have standardized on an encoding in practice), the same system encoding could very well be in use on the machines where the files end up. -Toshio pgp9rjopytsng.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 17 October 2013 20:01, Guido van Rossum wrote: > On Thu, Oct 17, 2013 at 11:55 AM, Oscar Benjamin > wrote: >> >> On 17 October 2013 19:40, Xavier Morel wrote: >> > I think there's already a significant split between context managers >> > which handle the lifecycle of a local resource (file, transaction) and >> > those which purport to locally alter global-ish state (cwd, >> > decimal.localcontext, logging.captureWarnings, redirect_stdout). >> > >> > And the latter worries me (much more than the very localized behavior of >> > suppress) because I don't see any way to implement them safely and >> > correctly when mixing it with coroutines in today's Python (some of them >> > aren't even thread-safe), all of that while I expect coroutines will see >> > significantly more use in the very near future with yield from and >> > tulip's promotion of coroutine-style async. >> >> I maybe misunderstanding how the coroutine-style async works but I >> would have thought that it would be as simple as: don't use >> global-state-restoring-context-managers around statements that yield >> control (it would be simpler if there was a good term for describing >> that kind of CM). That's simpler to implement and computationally >> cheaper than e.g. the thread-local state used by the decimal module. > > Context managers that actually save and restore *global* state are already > not thread-safe, so concluding they are also not coroutine-safe (or > task-safe?) seems a small step. > > I'd be more worried about context manager that use thread-local state -- > there is no similar concept in Tulip. It's unnecessary in Tulip. The need for thread-local state in e.g. decimal contexts is driven by the fact that multi-threaded execution switches in an uncontrollable way. Tulip specifically makes it possible to control the points at which a switch occurs making this safe (even if localcontext() wasn't thread-safe): with decimal.localcontext() as ctx: ctx.prec = 100 c = a + b # more synchronous decimal calculations # State is restored before allowing other code to execute yield from save_in_database(c) So it's fine to use global/thread-local state modifying/restoring context managers in Tulip as long as you don't yield control to other code within the with block. (unless I misunderstand - I lost track of Tulip some time ago). The issue with decimal.localcontext() and yield arises when using generators as much as coroutines e.g.: def exact_sum(nums): start = Decimal(0) with decimal.localcontext() as ctx: ctx.traps[decimal.Inexact] = True for num in nums: try: total += Decimal(num) except decimal.Inexact: ctx.prec *= 2 return total The above is fine for computing the sum of a list of Decimals/ints/floats. However it fails if you pass in a generator that carelessly modifies the arithmetic context around yield calls: def compute(): with decimal.localcontext() as ctx: ctx.prec = 15 ctx.traps[decimal.Inexact] = False yield a + b yield b - c # etc. exact_sum(compute()) There needs to be a convention that either functions like exact_sum() mustn't assume continuity of the context between iterations or a function like compute() must restore before yielding. IMO the only sane approach for async coroutines is to say that if you yield or yield from then it is your responsibility to restore any temporarily altered global/thread-local state first. Oscar ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On Thu, Oct 17, 2013 at 3:51 PM, Oscar Benjamin wrote: > On 17 October 2013 20:01, Guido van Rossum wrote: > > On Thu, Oct 17, 2013 at 11:55 AM, Oscar Benjamin > > wrote: > >> > >> On 17 October 2013 19:40, Xavier Morel wrote: > >> > I think there's already a significant split between context managers > >> > which handle the lifecycle of a local resource (file, transaction) and > >> > those which purport to locally alter global-ish state (cwd, > >> > decimal.localcontext, logging.captureWarnings, redirect_stdout). > >> > > >> > And the latter worries me (much more than the very localized behavior > of > >> > suppress) because I don't see any way to implement them safely and > >> > correctly when mixing it with coroutines in today's Python (some of > them > >> > aren't even thread-safe), all of that while I expect coroutines will > see > >> > significantly more use in the very near future with yield from and > >> > tulip's promotion of coroutine-style async. > >> > >> I maybe misunderstanding how the coroutine-style async works but I > >> would have thought that it would be as simple as: don't use > >> global-state-restoring-context-managers around statements that yield > >> control (it would be simpler if there was a good term for describing > >> that kind of CM). That's simpler to implement and computationally > >> cheaper than e.g. the thread-local state used by the decimal module. > > > > Context managers that actually save and restore *global* state are > already > > not thread-safe, so concluding they are also not coroutine-safe (or > > task-safe?) seems a small step. > > > > I'd be more worried about context manager that use thread-local state -- > > there is no similar concept in Tulip. > > It's unnecessary in Tulip. The need for thread-local state in e.g. > decimal contexts is driven by the fact that multi-threaded execution > switches in an uncontrollable way. Tulip specifically makes it > possible to control the points at which a switch occurs making this > safe (even if localcontext() wasn't thread-safe): > > with decimal.localcontext() as ctx: > ctx.prec = 100 > c = a + b > # more synchronous decimal calculations > > # State is restored before allowing other code to execute > yield from save_in_database(c) > > So it's fine to use global/thread-local state modifying/restoring > context managers in Tulip as long as you don't yield control to other > code within the with block. (unless I misunderstand - I lost track of > Tulip some time ago). > You've got it exactly right. > The issue with decimal.localcontext() and yield arises when using > generators as much as coroutines e.g.: > > def exact_sum(nums): > start = Decimal(0) > with decimal.localcontext() as ctx: > ctx.traps[decimal.Inexact] = True > for num in nums: > try: > total += Decimal(num) > except decimal.Inexact: > ctx.prec *= 2 > return total > > The above is fine for computing the sum of a list of > Decimals/ints/floats. However it fails if you pass in a generator that > carelessly modifies the arithmetic context around yield calls: > > def compute(): > with decimal.localcontext() as ctx: > ctx.prec = 15 > ctx.traps[decimal.Inexact] = False > yield a + b > yield b - c > # etc. > > exact_sum(compute()) > > There needs to be a convention that either functions like exact_sum() > mustn't assume continuity of the context between iterations or a > function like compute() must restore before yielding. IMO the only > sane approach for async coroutines is to say that if you yield or > yield from then it is your responsibility to restore any temporarily > altered global/thread-local state first. > Right again. The simplest rule to remember seems to be "don't use yield or yield-from inside a with-statement". You can relax it by limiting it to context managers that manage any kind of shared resource, but that is probably already too subtle: e.g. yielding inside "with open(file) as f" seems fine, but yielding inside "with lock" is problematic, since the other side might try to acquire the same lock, and deadlock. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] The Tulip Has Landed
For those interested parties: Guido just checked "asyncio", aka "Tulip", aka "PEP 3156", in to trunk. I expect it to be part of Python 3.4.0a4, hopefully to be released this weekend. Maybe /that'll/ generate some interest in the alpha ;-) //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 18 Oct 2013 09:05, "Guido van Rossum" wrote: > > On Thu, Oct 17, 2013 at 3:51 PM, Oscar Benjamin < oscar.j.benja...@gmail.com> wrote: >> >> On 17 October 2013 20:01, Guido van Rossum wrote: >> > On Thu, Oct 17, 2013 at 11:55 AM, Oscar Benjamin >> > wrote: >> >> >> >> On 17 October 2013 19:40, Xavier Morel wrote: >> >> > I think there's already a significant split between context managers >> >> > which handle the lifecycle of a local resource (file, transaction) and >> >> > those which purport to locally alter global-ish state (cwd, >> >> > decimal.localcontext, logging.captureWarnings, redirect_stdout). >> >> > >> >> > And the latter worries me (much more than the very localized behavior of >> >> > suppress) because I don't see any way to implement them safely and >> >> > correctly when mixing it with coroutines in today's Python (some of them >> >> > aren't even thread-safe), all of that while I expect coroutines will see >> >> > significantly more use in the very near future with yield from and >> >> > tulip's promotion of coroutine-style async. >> >> >> >> I maybe misunderstanding how the coroutine-style async works but I >> >> would have thought that it would be as simple as: don't use >> >> global-state-restoring-context-managers around statements that yield >> >> control (it would be simpler if there was a good term for describing >> >> that kind of CM). That's simpler to implement and computationally >> >> cheaper than e.g. the thread-local state used by the decimal module. >> > >> > Context managers that actually save and restore *global* state are already >> > not thread-safe, so concluding they are also not coroutine-safe (or >> > task-safe?) seems a small step. >> > >> > I'd be more worried about context manager that use thread-local state -- >> > there is no similar concept in Tulip. >> >> It's unnecessary in Tulip. The need for thread-local state in e.g. >> decimal contexts is driven by the fact that multi-threaded execution >> switches in an uncontrollable way. Tulip specifically makes it >> possible to control the points at which a switch occurs making this >> safe (even if localcontext() wasn't thread-safe): >> >> with decimal.localcontext() as ctx: >> ctx.prec = 100 >> c = a + b >> # more synchronous decimal calculations >> >> # State is restored before allowing other code to execute >> yield from save_in_database(c) >> >> So it's fine to use global/thread-local state modifying/restoring >> context managers in Tulip as long as you don't yield control to other >> code within the with block. (unless I misunderstand - I lost track of >> Tulip some time ago). > > > You've got it exactly right. > >> >> The issue with decimal.localcontext() and yield arises when using >> generators as much as coroutines e.g.: >> >> def exact_sum(nums): >> start = Decimal(0) >> with decimal.localcontext() as ctx: >> ctx.traps[decimal.Inexact] = True >> for num in nums: >> try: >> total += Decimal(num) >> except decimal.Inexact: >> ctx.prec *= 2 >> return total >> >> The above is fine for computing the sum of a list of >> Decimals/ints/floats. However it fails if you pass in a generator that >> carelessly modifies the arithmetic context around yield calls: >> >> def compute(): >> with decimal.localcontext() as ctx: >> ctx.prec = 15 >> ctx.traps[decimal.Inexact] = False >> yield a + b >> yield b - c >> # etc. >> >> exact_sum(compute()) >> >> There needs to be a convention that either functions like exact_sum() >> mustn't assume continuity of the context between iterations or a >> function like compute() must restore before yielding. IMO the only >> sane approach for async coroutines is to say that if you yield or >> yield from then it is your responsibility to restore any temporarily >> altered global/thread-local state first. > > > Right again. The simplest rule to remember seems to be "don't use yield or yield-from inside a with-statement". You can relax it by limiting it to context managers that manage any kind of shared resource, but that is probably already too subtle: e.g. yielding inside "with open(file) as f" seems fine, but yielding inside "with lock" is problematic, since the other side might try to acquire the same lock, and deadlock. Even the file case can be tricky - if you actually yield *the file* (rather than nothing or its contents), then the recipient may be surprised when it is later closed automatically. It wouldn't surprise me if we see idioms to temporarily revert the effects of context managers start to grow in popularity once asyncio is released as part of 3.4. Cheers, Nick. > > -- > --Guido van Rossum (python.org/~guido) > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/nc
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 18 Oct 2013 06:59, "Xavier Morel" wrote: > > On 2013-10-17, at 22:11 , Ethan Furman wrote: > > > On 10/17/2013 01:03 PM, Terry Reedy wrote: > >> > >> class suppress: > >> def __init__(self, *exceptions): > >> self.exceptions = exceptions > >> def __exit__(self, etype, eval, etrace): > >> return etype in self.exceptions > > > > This fails when etype is a subclass of the exceptions, as mentioned in the original issue. > > That's fixed by using `issubclass` and does not infirm Terry's point does it? Yeah, it looks like it's worth switching to the class based implementation in this case. I guess I'm too accustomed to that being the more complex alternative, as I hadn't even tried it :) Cheers, Nick. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
Guido van Rossum wrote: yielding inside "with open(file) as f" seems fine, but yielding inside "with lock" is problematic, since the other side might try to acquire the same lock, and deadlock. So maybe the documentation of a context manager should include whether it's safe to yield inside the block. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The Tulip Has Landed
On 18 Oct 2013 09:07, "Larry Hastings" wrote: > > > For those interested parties: Guido just checked "asyncio", aka "Tulip", aka "PEP 3156", in to trunk. I expect it to be part of Python 3.4.0a4, hopefully to be released this weekend. Huzzah! :) Cheers, Nick. > > Maybe that'll generate some interest in the alpha ;-) > > > /arry > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
> Right again. The simplest rule to remember seems to be "don't use yield or > yield-from inside a with-statement". You can relax it by limiting it to > context managers that manage any kind of shared resource, but that is > probably already too subtle: e.g. yielding inside "with open(file) as f" > seems fine, but yielding inside "with lock" is problematic, since the other > side might try to acquire the same lock, and deadlock. > This seems like the kind of thing that might warrant a pylint warning. Probably the more general case, requiring people to think about it and suppress the warning with an explanatory comment (or write the code in a different manner) when they are doing something known to be safe. -gps ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On suppress()'s trail blazing (was Re: cpython: Rename contextlib.ignored() to contextlib.ignore())
On 10/17/2013 7:35 PM, Nick Coghlan wrote: On 18 Oct 2013 06:59, "Xavier Morel" mailto:catch-...@masklinn.net>> wrote: > > On 2013-10-17, at 22:11 , Ethan Furman wrote: > > > On 10/17/2013 01:03 PM, Terry Reedy wrote: > >> > >> class suppress: > >> def __init__(self, *exceptions): > >> self.exceptions = exceptions > >> def __exit__(self, etype, eval, etrace): > >> return etype in self.exceptions > > > > This fails when etype is a subclass of the exceptions, as mentioned in the original issue. There are two other failures; see below. I know better than to post code without testing or saying 'untested'; I just wish I would always remember... > That's fixed by using `issubclass` and does not infirm Terry's point does it? Yeah, it looks like it's worth switching to the class based implementation in this case. I guess I'm too accustomed to that being the more complex alternative, as I hadn't even tried it :) With two more changes (add __enter__, account for etype is None), the suppress tests pass. import unittest class suppress: def __init__(self, *exceptions): self.exceptions = exceptions def __enter__(self): pass def __exit__(self, etype, val, tb): return etype is None or issubclass(etype, self.exceptions) class Testsuppress(unittest.TestCase): def test_no_exception(self): with suppress(ValueError): self.assertEqual(pow(2, 5), 32) def test_exact_exception(self): with suppress(TypeError): len(5) def test_multiple_exception_args(self): with suppress(ZeroDivisionError, TypeError): len(5) def test_exception_hierarchy(self): with suppress(LookupError): 'Hello'[50] unittest.main() >>> Ran 4 tests...OK -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Checking in Argument Clinic early?
On 10/15/2013 09:22 AM, Larry Hastings wrote: If this love-in continues I'll prep a release tonight and commit it in the morning... just before my flight home across the Atlantic. Okay, so, I slowed it down a little--some last-minute things cropped up. I've uploaded a fresh patch to the issue: http://bugs.python.org/issue16612 I plan to check it in about 24 hours from now, so it can be in trunk for most of a day before I tag for 3.4a4. If you'd like feedback at this late stage, please head to the issue above and go through the side-by-side diffs on Rietveld. And thanks for your interest! //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #16129: Add `Py_SetStandardStreamEncoding`
Am 17.10.2013 17:36, schrieb Antoine Pitrou: > On Thu, 17 Oct 2013 15:22:03 +0200 (CEST) > nick.coghlan wrote: >> >> +.. c:function:: int Py_SetStandardStreamEncoding(char *encoding, char >> *errors) >> + >> + .. index:: >> + single: Py_Initialize() >> + single: main() >> + triple: stdin; stdout; sdterr >> + >> + This function should be called before :c:func:`Py_Initialize`. It >> + specifies which encoding and error handling to use with standard io, >> + with the same meanings as in :func:`str.encode`. >> + >> + It overrides :envvar:`PYTHONIOENCODING` values, and allows embedding code >> + to control io encoding when the environment variable does not work. >> + >> + ``encoding`` and/or ``errors`` may be NULL to use >> + :envvar:`PYTHONIOENCODING` and/or default values (depending on other >> + settings). >> + >> + Note that :data:`sys.stderr` always uses the "backslashreplace" error >> + handler, regardless of this (or any other) setting. >> + >> + If :c:func:`Py_Finalize` is called, this function will need to be called >> + again in order to affect subsequent calls to :c:func:`Py_Initialize`. >> + >> + Returns 0 if successful. >> + >> + > > Needs a versionadded tag. Also, shouldn't it be excluded from the stable/limited API? Georg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The Tulip Has Landed
Yay! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
