New submission from Yury Selivanov :
I'm investigating a memory leak in 3.10, and while looking at the offending
commit I stumbled upon this: in ceval.c:PyEval_EvalCodeEx, there's this
allocation:
PyObject **kwargs = PyMem_Malloc(sizeof(PyObject *)*kwcount);
The problem is
Yury Selivanov added the comment:
(this is the context of our current investigation btw:
https://github.com/MagicStack/asyncpg/issues/874#issuecomment-1009383262)
--
___
Python tracker
<https://bugs.python.org/issue46
Yury Selivanov added the comment:
This is the questionable place in the code:
https://github.com/python/cpython/blob/dce642f24418c58e67fa31a686575c980c31dd37/Python/ceval.c#L6131-L6166
See that `kwargs` is allocated but never freed or used
Change by Yury Selivanov :
--
keywords: +patch
pull_requests: +28746
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/30546
___
Python tracker
<https://bugs.python.org/issu
Yury Selivanov added the comment:
New changeset 607d8a838f29ad3c4c4e85b39f338dade5f9cafe by Yury Selivanov in
branch 'main':
bpo-46347: Fix memory leak in PyEval_EvalCodeEx. (#30546)
https://github.com/python/cpython/commit/607d8a838f29ad3c4c4e85b39f338d
Change by Yury Selivanov :
--
pull_requests: +28752
pull_request: https://github.com/python/cpython/pull/30551
___
Python tracker
<https://bugs.python.org/issue46
Yury Selivanov added the comment:
> The function will still leak "kwnames" and "default" if creating the "func"
> object fails.
Yeah, here's another PR to address that:
https://github.com/python/cpython/pull/30551
--
_
Yury Selivanov added the comment:
New changeset 20b5791ce9b47195ce51cfd5acb223b1ca59cdf0 by Yury Selivanov in
branch 'main':
bpo-46347: Fix PyEval_EvalCodeEx to correctly cleanup in error paths (#30551)
https://github.com/python/cpython/commit/20b5791ce9b47195ce51cfd5acb223
Change by Yury Selivanov :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by Yury Selivanov :
--
pull_requests: +28754
pull_request: https://github.com/python/cpython/pull/30553
___
Python tracker
<https://bugs.python.org/issue46
Change by Yury Selivanov :
--
pull_requests: +28755
pull_request: https://github.com/python/cpython/pull/30554
___
Python tracker
<https://bugs.python.org/issue46
Yury Selivanov added the comment:
New changeset 6f9ca53a6ac343a5663cc5c52546acf9a63b605a by Yury Selivanov in
branch '3.10':
bpo-46347: Fix PyEval_EvalCodeEx to correctly cleanup in error paths (#30553)
https://github.com/python/cpython/commit/6f9ca53a6ac343a5663cc5c52546ac
Yury Selivanov added the comment:
New changeset be578e0c063dad1dbb273f86d5bc77e4e6f14583 by Yury Selivanov in
branch 'main':
bpo-46347: Yet another fix in the erorr path of PyEval_EvalCodeEx (#30554)
https://github.com/python/cpython/commit/be578e0c063dad1dbb273f86d5bc77
Yury Selivanov added the comment:
> There's one thing that gather() does that TaskGroup doesn't: it gives us the
> return values from the tasks.
That's easy to do with task groups too:
async with TaskGroup() as g:
r1 = g.create_task(coro1())
r2
Yury Selivanov added the comment:
Couple thoughts:
I'm +1 for adding TaskGroup.cancel() method.
I'd be -1 on abusing `Task.cancel()` to signal something with a nonce. Whatever
problem we are trying to solve here, it should be solvable without resorting to
hacks like this. It
Yury Selivanov added the comment:
> * `async with` vs `with`: if Andrew thinks `async with` is easier to teach
> and less error-prone,
It has to be 'async with' like most asyncio apis to avoid user confusion.
--
___
Python
Yury Selivanov added the comment:
> I believe that this approach would allow more control over tasks implicitly
> spawned by 3rd-party libraries that cannot control.
Please elaborate. I'm not sure what are the benefits of this.
--
___
Changes by Yury Selivanov :
--
nosy: +Yury.Selivanov
___
Python tracker
<http://bugs.python.org/issue13062>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Yury Selivanov :
--
nosy: +Yury.Selivanov
___
Python tracker
<http://bugs.python.org/issue13241>
___
___
Python-bugs-list mailing list
Unsubscribe:
Yury Selivanov added the comment:
It looks like there is a memory leak bug (on StopIteration exception instances).
Attached is the test to expose it.
It seems that adding 'Py_DECREF(e);' after
'PyErr_SetObject(PyExc_StopIteration, e);' in 'genobject.c' fixes th
Changes by Yury Selivanov :
--
nosy: +Yury.Selivanov
___
Python tracker
<http://bugs.python.org/issue10882>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Yury Selivanov :
As discussed on python-dev mailing list
(http://mail.python.org/pipermail/python-dev/2010-August/103178.html),
'hasattr' default behaviour should be changed to suppress only AttributeError
exceptions. Other should pass through.
The fi
New submission from Yury Selivanov :
The patch is intended to fix behaviour of 'return' statement in python's
generators.
Please read this message before looking at the patch.
http://mail.python.org/pipermail/python-dev/2010-August/103297.html
--
components: Interpr
Changes by Yury Selivanov :
--
nosy: +Yury.Selivanov
___
Python tracker
<http://bugs.python.org/issue11682>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Yury Selivanov :
--
nosy: +Yury.Selivanov
___
Python tracker
<http://bugs.python.org/issue10576>
___
___
Python-bugs-list mailing list
Unsubscribe:
Yury Selivanov added the comment:
Andrew asked me for my opinion on the matter -- I think we should get rid of
the message. Exception messages for "signals" can be easily lost if an
exception was re-raised. If the reason of why something is being cancelled is
important (in my exp
Yury Selivanov added the comment:
IOW I think that supporting custom messages is a needless complication of our
API. Given how complex task trees can become with TaskGroups collecting those
messages and presenting them all to the user isn't trivial, showing just
first/last defeat
Yury Selivanov added the comment:
Yeah, +1 to add a parameter. Fwiw it was on my idea list when i was working on
the pep
--
___
Python tracker
<https://bugs.python.org/issue46
Yury Selivanov added the comment:
Andrew, the same problem exists in asyncio.Queue which is also critical. Here's
how I fixed it in edgedb code base:
https://github.com/edgedb/edgedb/blob/08e41341024828df22a01cd690b11fcff00bca5e/edb/server/compiler_pool/queue.py#L5
Yury Selivanov added the comment:
Sorry, I don't like the committed change and I think it should be reverted.
Futures and Tasks have a documented `get_loop()` method which simply should be
called by Task machinery, without the need to expose (and document) private
me
Yury Selivanov added the comment:
> I have a feeling that we need a low-level API section that describes
> *supported* API that is intended to task extenders only (guys who want to
> provide an alternative task factory for example).
I'm not sure how exposing all private method
Yury Selivanov added the comment:
Andrew, can you fix ctrl-c/windows issue? I'm basically afk until monday.
And we're not going to do anything about add_signal_handler in 3.8.
--
___
Python tracker
<https://bugs.python.o
Yury Selivanov added the comment:
Here's the API I propose to solve this problem:
https://github.com/python/cpython/pull/15735#pullrequestreview-285389412
Summary:
* Add a new loop.shutdown_threadpool() method. Just like with
shutdown_asyncgens() -- it would be invalid to
Yury Selivanov added the comment:
> What do you think of the "shutdown_default_executor()" name?
Yeah, I think it's a better name!
--
___
Python tracker
<https://bugs.
Yury Selivanov added the comment:
🙏
--
___
Python tracker
<https://bugs.python.org/issue33523>
___
___
Python-bugs-list mailing list
Unsubscribe:
Yury Selivanov added the comment:
BTW, if we are doing this -- maybe not expose the underlying transport at all?
I don't like that we do that currently, and there multiple ways for the user to
screw up the Stream object (and potentially break start_tls()/sendfile(
Yury Selivanov added the comment:
We want to add TaskGroups to asyncio (a similar concept to Trio's nurseries).
TaskGroups use the `async with` statement to clearly define where Tasks are
created and at which point they are expected to be completed or destroyed.
asyncio.g
Yury Selivanov added the comment:
FWIW I've been using TaskGroups in EdgeDB codebase extensively:
https://github.com/edgedb/edgedb/blob/master/edb/common/taskgroup.py (you can
use the code, it's Apache 2)
The only thing that prevented us from merging them in 3.8 is that we need to
Yury Selivanov added the comment:
> it will help simplify some Mock things.
Yeah, we'll need to chat about that so I can use Mock requirements in the PEP.
:)
--
___
Python tracker
<https://bugs.python.org
Yury Selivanov added the comment:
(copying my GitHub message on this topic from
https://github.com/python/cpython/pull/15889#issuecomment-530030002)
I thought we were going to be more subtle about this.
We have two scenarios:
Somebody creates an instance of asyncio.Lock outside of a
Change by Yury Selivanov :
--
Removed message: https://bugs.python.org/msg351728
___
Python tracker
<https://bugs.python.org/issue36373>
___
___
Python-bug
Yury Selivanov added the comment:
(copying my GitHub message on this topic from
https://github.com/python/cpython/pull/15889#issuecomment-530030002)
I thought we were going to be more subtle about this.
We have two scenarios:
1. Somebody creates an instance of asyncio.Lock outside of a
New submission from Yury Selivanov :
Summary of discussion at https://github.com/python-trio/trio/issues/1208:
* `asyncio.Stream.write()` will start throwing a `DeprecationWarning` asking
people to add an `await` if they didn't;
* `asyncio.Stream.close()` will start throw
Change by Yury Selivanov :
--
title: poloshing asyncio Streams API -> polishing asyncio Streams API
___
Python tracker
<https://bugs.python.org/issu
Yury Selivanov added the comment:
> If a python piece of code imports cython code with async defs,
> `asyncio.iscoroutinefunction` cannot determine that the code is async.
Well, this seems to be a regression. IIRC Cython used to emulate a Python code
object (__code__) for functi
New submission from Yury Selivanov :
== Context
1. Andrew and I implemented a new streaming API in asyncio 3.8. The key idea
is that there's a single Stream object, combining old StreamReader &
StreamWriter APIs. On top of that, `Stream.write()` and `Stream.close()`
became c
Yury Selivanov added the comment:
> Task group is another very required thing. If I choose between groups and
> steams I probably invest my time into the former.
We should get them in 3.9. I'm going to be working on the ExceptionGroup
Yury Selivanov added the comment:
> The obvious fix would be to modify the 'except StopIteration' branch to
> handle this case by calling super().cancel() instead of
> super().set_exception(...).
Yeah, I think this is the solution
Change by Yury Selivanov :
--
keywords: +patch
pull_requests: +15907
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16330
___
Python tracker
<https://bugs.python.org/issu
Change by Yury Selivanov :
--
nosy: +aeros167
___
Python tracker
<https://bugs.python.org/issue38260>
___
___
Python-bugs-list mailing list
Unsubscribe:
Yury Selivanov added the comment:
Nathaniel, thanks for the summary!
A few comments to address some points:
> So Yury and I are tentatively thinking we'll make some kind of PEP for the
> 3.9 timescale, probably just for the core ABCs
Yes! At the very least for things like a
Yury Selivanov added the comment:
> So like... both of these approaches are definitely possible, but to me it
> seems like if you look at it holistically, your approach is actually making
> the documentation and deprecations *more* complicated, not less.
I think Nathaniel might hav
Yury Selivanov added the comment:
I slept on this and discussed this issue privately with a few non-involved
people who use asyncio daily.
My final opinion on this issue: we must revert the new streaming API from
asyncio and take our time to design it properly. I don't like what we ha
Yury Selivanov added the comment:
I already mentioned that in the PR, but we'll have to hit a pause on this. We
decided to revert the latest streams implementation from 3.8, see
https://bugs.python.org/issue38242. The upshot is that what we have in 3.9
should be more amenable for t
Change by Yury Selivanov :
--
resolution: -> later
status: open -> pending
___
Python tracker
<https://bugs.python.org/issue37141>
___
___
Python-bugs-
Yury Selivanov added the comment:
> Since this has been elevated to a release blocker, I wouldn't mind helping to
> revert this ASAP. I can open a PR to fix it today.
Sure, by all means, any help would be hugely appreciated. Thank you, Kyle.
You'll need to be careful to onl
Change by Yury Selivanov :
--
pull_requests: +16066
pull_request: https://github.com/python/cpython/pull/16482
___
Python tracker
<https://bugs.python.org/issue38
Change by Yury Selivanov :
--
pull_requests: +16067
pull_request: https://github.com/python/cpython/pull/16482
___
Python tracker
<https://bugs.python.org/issue36
Yury Selivanov added the comment:
New changeset 6758e6e12a71ef5530146161881f88df1fa43382 by Yury Selivanov in
branch 'master':
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
https://github.com/python/cpython/commit/6758e6e12a71ef553014
Yury Selivanov added the comment:
New changeset 6758e6e12a71ef5530146161881f88df1fa43382 by Yury Selivanov in
branch 'master':
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
https://github.com/python/cpython/commit/6758e6e12a71ef553014
Change by Yury Selivanov :
--
pull_requests: +16070
pull_request: https://github.com/python/cpython/pull/16485
___
Python tracker
<https://bugs.python.org/issue38
Change by Yury Selivanov :
--
pull_requests: +16071
pull_request: https://github.com/python/cpython/pull/16485
___
Python tracker
<https://bugs.python.org/issue36
Yury Selivanov added the comment:
New changeset 1c19d656a79a00f58361ceb61c0a6d1faf90c686 by Yury Selivanov in
branch '3.8':
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
(#16485)
https://github.com/python/cpython/commit/1c19d656a79a00f58361
Yury Selivanov added the comment:
I've reverted the code. Andrew, would really appreciate if you could quickly do
a post commit review.
--
stage: patch review -> commit review
___
Python tracker
<https://bugs.python.org
Yury Selivanov added the comment:
New changeset 1c19d656a79a00f58361ceb61c0a6d1faf90c686 by Yury Selivanov in
branch '3.8':
bpo-38242: Revert "bpo-36889: Merge asyncio streams (GH-13251)" (#16482)
(#16485)
https://github.com/python/cpython/commit/1c19d656a79a00f58361
Yury Selivanov added the comment:
New changeset fc4a044a3c54ce21e9ed150f7d769fb479d34c49 by Yury Selivanov in
branch 'master':
bpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel
(#7468)
https://github.com/python/cpyt
Yury Selivanov added the comment:
New changeset 2f87a7dc5a1ad7f37787f0adee242c931643f878 by Yury Selivanov (Miss
Islington (bot)) in branch '3.8':
bpo-30773: Fix ag_running; prohibit running athrow/asend/aclose in parallel
(GH-7468) (#16486)
https://github.com/python/cpyt
Change by Yury Selivanov :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Yury Selivanov added the comment:
> It seems like nobody came up with a solution for the debug mode since June. I
> close the issue. Reopen it if you would like to propose a solution.
I think the only solution is to have a flag to disable optimizations,
inlcluding th
Change by Yury Selivanov :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Yury Selivanov added the comment:
New changeset b23a8423a923077e4f83d3f328bb7542b4c940ed by Yury Selivanov
(idomic) in branch 'master':
bpo-34344 Fix AbstractEventLoopPolicy.get_event_loop docstring (GH-16463)
https://github.com/python/cpython/commit/b23a8423a923077e4f83d3f328bb75
Yury Selivanov added the comment:
New changeset 321def805abc5b7c92c7e90ca90cb2434fdab855 by Yury Selivanov (Ben
Harper) in branch 'master':
bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598)
https://github.com/python/cpython/commit/321def805abc5b7c92c7e90ca90cb2
Yury Selivanov added the comment:
> In bpo-38392, I modified PyObject_GC_Track() in debug mode to detect this bug.
Awesome!!!
--
___
Python tracker
<https://bugs.python.org/issu
Change by Yury Selivanov :
--
nosy: -yselivanov
___
Python tracker
<https://bugs.python.org/issue36356>
___
___
Python-bugs-list mailing list
Unsubscribe:
Yury Selivanov added the comment:
Yes, docs updates sound good.
--
___
Python tracker
<https://bugs.python.org/issue38423>
___
___
Python-bugs-list mailin
Change by Yury Selivanov :
--
nosy: -yselivanov
___
Python tracker
<https://bugs.python.org/issue32856>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Yury Selivanov :
I believe I might have discovered a problem with asynchronous generators in 3.8.
# Prelude
In Python prior to 3.8 it was possible to overlap running of "asend()" and
"athrow()" methods for the same asynchronous generator.
In p
Yury Selivanov added the comment:
Just discussed this issue off-list with Nathaniel. Maybe this isn't as severe
as I thought it is.
If we cancel all asyncio tasks before calling `loop.shutdown_asyncgens()` this
*probably* becomes a non-issue. And "asyncio.run()" does just
Yury Selivanov added the comment:
Was fixed as part of #30773.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Yury Selivanov added the comment:
> So I think asyncio.run is actually totally fine with the 3.8.0 behavior.
Yes. Looks like it.
> It's only explicit calls to shutdown_asyncgens that might run into this, and
> I think that's probably OK?
Yes. Calling shutdown_asyncg
Yury Selivanov added the comment:
FWIW we discussed my patch with Eric at PyCon 2019 and I think he had no issues
with it. Eric, can I merge it? Should we backport to 3.8?
--
___
Python tracker
<https://bugs.python.org/issue34
Yury Selivanov added the comment:
> Yury: I'm okay with merging. If I recall, you were going to add a comment or
> two about the approach (a closure, if I remember correctly).
Sure, I'll rebase and add a comment in a couple of days. I'll elevate the
status so that
Yury Selivanov added the comment:
I think this broke asyncio a bit. These two uvloop sock_* API tests no longer
pass on asyncio 3.8:
* https://github.com/MagicStack/uvloop/blob/master/tests/test_sockets.py#L192
* https://github.com/MagicStack/uvloop/blob/master/tests/test_sockets.py#L238
Yury Selivanov added the comment:
I'll elevate the status so that we don't forget before 3.8.1 is too close
--
priority: normal -> deferred blocker
___
Python tracker
<https://bugs.pytho
Yury Selivanov added the comment:
The more I think about this the more I like new 3.8 behavior. I'm going to
close this issue; if anything comes up I'll reopen it later. Sorry for the
noise.
--
resolution: -> not a bug
stage: -> resolved
status
Change by Yury Selivanov :
--
nosy: -yselivanov
___
Python tracker
<https://bugs.python.org/issue20443>
___
___
Python-bugs-list mailing list
Unsubscribe:
Yury Selivanov added the comment:
Didn't we just add MultiLoopChildWatcher in 3.8?
--
status: pending -> open
___
Python tracker
<https://bugs.python.org
Yury Selivanov added the comment:
> but perhaps if we go through with that we should remove MultiLoopChildWatcher
> from 3.8 (if it's not too late to do so) instead of deprecating it.
I'll leave that up to Andrew to decide, but I'd be +1 to drop it asap,
especially if
Yury Selivanov added the comment:
Kyle, why are you resetting the status to "Pending"?
> I think it will be necessary to fix the issues in the near future if we want
> to keep MultiLoopChildWatcher in 3.8 (as they apply to 3.8 and 3.9). I would
> certainly not be ecstati
Yury Selivanov added the comment:
I'd add `and will be removed in 3.11.` now.
--
___
Python tracker
<https://bugs.python.org/issue34790>
___
___
Pytho
Yury Selivanov added the comment:
Yes.
As a remedy for this particular problem we can add checks here and there
comparing `asyncio.get_running_loop()` and `self._loop`. Performance will
suffer a bit but the usability will greatly improve
Yury Selivanov added the comment:
Yes, I've experienced this bug. We need to fix this in 3.8.1.
--
nosy: +lukasz.langa
priority: normal -> release blocker
___
Python tracker
<https://bugs.python.org
Yury Selivanov added the comment:
> It would be nice if asyncio.run() uses the default loop if it's available
> (and not running), and that the default loop remains intact after the call
> returns.
Unfortunately it's not possible to implement that reliably and
Yury Selivanov added the comment:
> Well, I'm basically using a run method defined as a shorthand for
> self.loop.run_until_complete (without closing loop, reusing it throughout).
> It would be nice if asyncio.run could simply be used instead, but I
> understand you're
Yury Selivanov added the comment:
Yeah, please do the change! Thanks!
--
___
Python tracker
<https://bugs.python.org/issue38652>
___
___
Python-bugs-list mailin
Yury Selivanov added the comment:
> The change is slightly not backward compatible but
Yeah, that's my main problem with converting `loop.run_in_executor()` to a
coroutine. When I attempted doing that I discovered that there's code that
expects the method to return a Future, a
Yury Selivanov added the comment:
> It is a more complex solution but definitely 100% backward compatible; plus
> the solution we can prepare people for removing the deprecated code
> eventually.
Yeah. Do you think it's worth it bothering with this old low-level API instea
Yury Selivanov added the comment:
Few thoughts:
1. I like the idea of having a context manager to create a thread pool. It
should be initialized in a top-level coroutine and the passed down to other
code, as in:
async def main():
async with asyncio.ThreadPool(concurrency=10) as pool
Yury Selivanov added the comment:
>> async with asyncio.ThreadPool(concurrency=10) as pool:
> I'm definitely on board with the usage of an async context manager and the
> functionality shown in the example, but I'm not sure that I entirely
> understand what t
Yury Selivanov added the comment:
>
>
>
> IMO, I think it would be a bit more clear to just explicitly call it
> "threads" or "max_threads", as that explains what it's effectively doing.
> While "concurrency" is still a perfectly correct
1 - 100 of 3098 matches
Mail list logo
