[issue34155] email.utils.parseaddr mistakenly parse an email

2019-08-16 Thread Kyle Stanley


Kyle Stanley  added the comment:

> This is already backported to 3.6. I am not sure about what gets backported 
> to 3.5 right now, I don't even see a 'Backport to 3.5' label on Github (which 
> made me think we are discouraged to backport to 3.5). I can work on a manual 
> backport if needed?

As far as I'm aware, backports to 3.5 have to be manually approved by those 
with repository management permissions, such the the organization owners 
(https://devguide.python.org/devcycle/#current-owners) and admins 
(https://devguide.python.org/devcycle/#current-administrators)

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue34155>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37890] Modernize several tests in test_importlib

2019-08-19 Thread Kyle Stanley


New submission from Kyle Stanley :

Last month, several tests were moved into test_importlib 
(https://bugs.python.org/issue19696): "test_pkg_import.py", 
"test_threaded_import.py", and  "threaded_import_hangers.py".

Those tests were created quite a while ago though and do not currently utilize 
importlib directly. They should be updated accordingly.

Brett Cannon:
> BTW, if you want to open a new issue and modernize the tests to use importlib 
> directly that would be great!

I'm interested in helping with this issue, but I may require some assistance as 
I'm not overly familiar with the internals of importlib. I'll probably start 
with "test_pkg_import.py". 

Anyone else can feel free to work on the other two in the meantime, but they 
should be worked on together as "threaded_import_hangers.py" is a dependency 
for "test_threaded_import.py".

--
components: Tests
messages: 349984
nosy: aeros167, brett.cannon
priority: normal
severity: normal
status: open
title: Modernize several tests in test_importlib
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37890] Modernize several tests in test_importlib

2019-08-19 Thread Kyle Stanley


Kyle Stanley  added the comment:

I'm not entirely certain as to which parts should be modernized, and which ones 
can remain the same. A large part of my uncertainty is that there are no header 
comments for "test_pkg_import.py" to explain the test coverage, so I don't know 
if there are additional tests beyond the existing ones that should be added.

The first steps I can think of: 

1) Use ``importlib.import_module()`` instead of the built-in ``__import__()``

2) Use ``with self.assertRaises(, ): ...`` instead of

```
try: __import__(self.module_name)
except SyntaxError: pass
else: raise RuntimeError('Failed to induce SyntaxError') # self.fail()?

...
```

3) Replace ``self.assertEqual(getattr(module, var), 1)`` with 
``self.assertEqual(getattr(module, var, None), 1)`` so that AttributeErrors are 
not raised when unexpected behaviors occur

4) Provide useful error messages for failed assertions

I'll begin working on those, probably with a separate PR for each of them for 
ease of review. Let me know if there's anything else I should do, or if any of 
the above steps are unneeded. If I think of anything else I'll update the issue 
accordingly.

--

___
Python tracker 
<https://bugs.python.org/issue37890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37890] Modernize several tests in test_importlib

2019-08-19 Thread Kyle Stanley


Change by Kyle Stanley :


--
nosy: +eric.snow, ncoghlan

___
Python tracker 
<https://bugs.python.org/issue37890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37890] Modernize several tests in test_importlib

2019-08-19 Thread Kyle Stanley


Change by Kyle Stanley :


--
stage:  -> needs patch
type:  -> enhancement

___
Python tracker 
<https://bugs.python.org/issue37890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37890] Modernize several tests in test_importlib

2019-08-20 Thread Kyle Stanley


Kyle Stanley  added the comment:

Ah okay, I wasn't sure what exactly would be involved with the "modernization" 
process, so those points were just rough ideas more than anything. I haven't 
started working on anything yet since I figured it'd be worthwhile to wait for 
approval first.

> 1) __import__() can be used for purpose. I would not change this.

Okay, I'll keep that part as is then. This idea was primarily based on 
importlib's documentation:

"Programmatic importing of modules should use import_module() instead of 
[importlib.__import__()]" 

But that probably applies more to users of importlib, rather than the internal 
tests for it. That would make sense.

3) How would you distinguish the case when the module have an attribute with 
the value is None and when it does not have the attribute at all? This 
information would lost with your change.

Good point. This might be a decent way to prevent the AttributeErrors, but 
still allows for differentiation of actual None values:

```
try:
self.assertEqual(getattr(module, var), 1)
except AttributeError:
self.fail(f"{module} should have attribute {var}")
```

Personally I think it makes a bit more sense to use self.fail() with a helpful 
message rather than raising errors within the tests. There's a comment on line 
56, "# self.fail() ?", which gave me this idea. Is there a particular 
preference in the context of Python's tests?

Also, do either of you (or anyone else) have any ideas for other modernization 
improvements that could be made to either test_pkg_import.py or to the other 
two? In the meantime, I can start working on ideas (2) and (4) if those ones 
would be appropriate. (2) should be fairly straightforward, but (4) will 
probably be a bit more subjective.

--

___
Python tracker 
<https://bugs.python.org/issue37890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37890] Modernize several tests in test_importlib

2019-08-20 Thread Kyle Stanley


Kyle Stanley  added the comment:

> This might be a decent way to prevent the AttributeErrors, but still allows 
> for differentiation of actual None values

Another alternative solution might be to use hasattr() before getattr(), if it 
is not desirable for test_pkg_import.py to raise exceptions:

```
 self.assertTrue(hasattr(module, var), msg=f"{module} should have attribute 
{var}")
 self.assertEqual(getattr(module, var), 1)
```

That would follow more of a LBYL style, but I know that EAFP is more common 
within Python. The better alternative depends on the answer to my earlier 
question regarding exceptions being raised from the unit tests:

> Is there a particular preference in the context of Python's tests?

--

___
Python tracker 
<https://bugs.python.org/issue37890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37890] Modernize several tests in test_importlib

2019-08-21 Thread Kyle Stanley


Kyle Stanley  added the comment:

> A key question here is why are you trying to avoid the AttributeError case so 
> much?

> but there's a reason that we don't have attribute existence tests before 
> every single attribute access throughout the test suite

Hmm, good point. I may have been fixating on avoiding exceptions within the 
tests a bit too much, perhaps the exception would be appropriate in this case.

> Basically unless the attribute is dynamic and thus you're explicitly testing 
> the attribute got set then don't worry about it.

Ah, that answers my earlier question, thanks. (:

With an initial glance, do you have any specific ideas as to how 
test_pkg_import.py should be changed to utilize importlib directly? I've been 
reading through the docs looking for ideas, but I'll admit that I'm not 
particularly experienced with using importlib.

--

___
Python tracker 
<https://bugs.python.org/issue37890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37890] Modernize several tests in test_importlib

2019-08-22 Thread Kyle Stanley


Kyle Stanley  added the comment:

> I would just read through the other tests files under test_importlib to see 
> how other tests were done.

Okay, I'll start with that and report back with any ideas for potential changes 
to test_pkg_import. Thanks.

--

___
Python tracker 
<https://bugs.python.org/issue37890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37585] Comparing PyDictValues does not give expected results

2019-08-23 Thread Kyle Stanley


Kyle Stanley  added the comment:

Reopening the issue for adding the documentation clarification, that comparing 
the values view of two dictionaries will always return false (as was suggested 
in the ML discussion 
https://mail.python.org/archives/list/python-...@python.org/message/K3SYX4DER3WAOWGQ4SPKCKXSXLXTIVAQ/).
 A month ago, I opened to PR to add the clarification to the documentation for 
dict.values(): https://github.com/python/cpython/pull/14954/files.

It probably went unnoticed because the issue had already been closed (which I 
hadn't realized at the time).

--
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue37585>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37585] Comparing PyDictValues does not give expected results

2019-08-23 Thread Kyle Stanley


Change by Kyle Stanley :


--
stage: resolved -> patch review

___
Python tracker 
<https://bugs.python.org/issue37585>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37934] Docs: Clarify NotImplemented use cases

2019-08-23 Thread Kyle Stanley


New submission from Kyle Stanley :

In the documentation for the NotImplemented constant 
(https://docs.python.org/3/library/constants.html#NotImplemented), the only use 
case mentioned is for binary special methods, (such as object.__eq__(other)). 
However, based on a conversation in a recent PR 
(https://github.com/python/cpython/pull/15327#discussion_r316561140), there 
seems be several other use cases as well.  It's quite useful to utilize when 
it's desired to express that a particular operation is not supported, without 
raising an error.

Expanding upon the use cases will provide readers an idea of other cases in 
which the constant could be useful. Also, the current wording could potentially 
come across as implying that the _only_ use case for the constant is for binary 
special methods, and as far as I'm aware, that is not the case.

--
assignee: docs@python
components: Documentation
keywords: easy
messages: 350333
nosy: aeros167, docs@python, eric.araujo, ezio.melotti, mdk, willingc
priority: normal
severity: normal
stage: needs patch
status: open
title: Docs: Clarify NotImplemented use cases
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37934>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

2019-08-23 Thread Kyle Stanley


Kyle Stanley  added the comment:

> python-dev likely isn't the right place.  That is a forum for more mature 
> ideas.

Agreed, that idea should be posted to python-list if they're having issues 
posting to python-ideas. Python-dev certainly wouldn't be appropriate for that.

--

___
Python tracker 
<https://bugs.python.org/issue37812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

2019-08-23 Thread Kyle Stanley


Change by Kyle Stanley :


--
type:  -> enhancement

___
Python tracker 
<https://bugs.python.org/issue37812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

2019-08-24 Thread Kyle Stanley


Kyle Stanley  added the comment:

> May I suggest directing your efforts towards fixing known bugs or 
> implementing requested features.  It isn't our goal to create more work for 
> one another

There frequently is value in improving code readability, as it can improve 
maintainability in the long term, but I definitely understand your point.

--

___
Python tracker 
<https://bugs.python.org/issue37812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34679] asyncio.add_signal_handler call fails if not on main thread

2019-08-24 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Skipping this call for non-main thread in proactor implementation makes sense.

How do we identify whether or not set_wakeup_fd() is being called from a 
non-main thread?

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue34679>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34679] asyncio.add_signal_handler call fails if not on main thread

2019-08-24 Thread Kyle Stanley


Kyle Stanley  added the comment:

> How do we identify whether or not set_wakeup_fd() is being called from a 
> non-main thread?

Never mind, I think I found the answer to my own question and tested a patch 
locally, I'll open a PR.

--

___
Python tracker 
<https://bugs.python.org/issue34679>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34679] asyncio.add_signal_handler call fails if not on main thread

2019-08-24 Thread Kyle Stanley


Change by Kyle Stanley :


--
keywords: +patch
pull_requests: +15163
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/15477

___
Python tracker 
<https://bugs.python.org/issue34679>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34679] asyncio.add_signal_handler call fails if not on main thread

2019-08-24 Thread Kyle Stanley


Change by Kyle Stanley :


--
keywords: +needs review -patch

___
Python tracker 
<https://bugs.python.org/issue34679>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34679] asyncio.add_signal_handler call fails if not on main thread

2019-08-25 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Kyle, thanks for the fix.
> I have basically the same change in my PR but with test and news note.

No problem, that works for me. I was mostly just trying to help with resolving 
some of the release blockers for 3.8b4.

--

___
Python tracker 
<https://bugs.python.org/issue34679>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23933] Struct module should accept arrays

2019-08-25 Thread Kyle Stanley


Change by Kyle Stanley :


--
nosy: +mark.dickinson, meador.inge

___
Python tracker 
<https://bugs.python.org/issue23933>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37934] Docs: Clarify NotImplemented use cases

2019-09-01 Thread Kyle Stanley


Kyle Stanley  added the comment:

Thanks for the feedback Vedran and Raymond.

> It is not the purpose of the docs to list use cases.  Mostly we say what 
> something does or how it is defined. As Vedran says, how people use it is 
> their own business.

The underlying issue here seems to be that I misunderstood the existing section 
to suggest to suggest binary special methods are the only use case, which is 
probably a good argument in favor of not listing additional use cases. This can 
be misinterpreted as suggesting that others are not recommended, and lead to 
further confusion.


First sentence of the NotImplemented documentation:

> Special value which should be returned by the binary special methods (e.g. 
> __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate that the 
> operation is not implemented with respect to the other type; may be returned 
> by the in-place binary special methods (e.g. __imul__(), __iand__(), etc.) 
> for the same purpose. 

Would it be viable to rephrase the existing section in a manner that explains 
the functional purpose of NotImplemented without revolving around its use case 
in binary special methods?

> Also, please be careful expanded the docs.  They quickly become a promise.

Good point, I may not have adequately considered that mentioning a use case 
turns into a promise for that functionality. This could easily result in a 
significant long-term maintenance cost.

--

___
Python tracker 
<https://bugs.python.org/issue37934>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37934] Docs: Clarify NotImplemented use cases

2019-09-01 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Would it be viable to rephrase the existing section in a manner that explains 
> the functional purpose of NotImplemented without revolving around its use 
> case in binary special methods?

To expand further upon this, here's an initial idea for improving the first 
sentence:

A special value used to indicate that an operation is not supported between 
specific types.

The section regarding it's usage in binary special methods could potentially 
remain. I'm thinking the main issue here (if there is one) is that the 
NotImplemented constant is defined _exclusively_ from a specific use case, 
rather than its general purpose.

--

___
Python tracker 
<https://bugs.python.org/issue37934>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37934] Docs: Clarify NotImplemented use cases

2019-09-02 Thread Kyle Stanley


Kyle Stanley  added the comment:

Thanks for the explanation.

> Of course, you might argue that _once Python has NotImplemented_, it can be 
> used elsewhere - but as I said, I don't think it should be encouraged.

Hmm, okay. My understanding of Raymond's explanation was more so "let's not 
encourage this because we don't want to guarantee the behavior" rather than 
"using it outside of binary operators shoudn't be encouraged".

Prior to Jerome's usage of it in ``fractions._as_integer_ratio()`` 
(https://github.com/python/cpython/pull/15327/files), I had not seen it used 
outside of special binary operators. I thought it was an interesting usage of 
NotImplemented, and the current phrasing of the documentation seemed to 
implicitly discourage it from being used in that manner.

--

___
Python tracker 
<https://bugs.python.org/issue37934>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18049] Re-enable threading test on macOS

2019-09-04 Thread Kyle Stanley


Kyle Stanley  added the comment:

Ronald, is it feasible that the changes made in 
https://github.com/python/cpython/pull/14748/files to THREAD_STACK_SIZE in 
Python/thread_pthread.h could be causing intermittent failures for the Azure 
macOS PR tests?

In a recent PR (https://github.com/python/cpython/pull/15651), 
test_pending_calls_race (test.test_concurrent_futures.ThreadPoolWaitTests) 
seemed to be timing out. See the build log for more details:  
https://dev.azure.com/Python/cpython/_build/results?buildId=49786&view=logs&j=18d1a34d-6940-5fc1-f55b-405e2fba32b1.

It doesn't appear to be causing consistent failures for Azure, but the latest 
commit to PR-15651 was only involving a formatting change to the news entry, so 
it should not have had any influence on the test that timed out. Prior to that 
commit, there were no CI failures.

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue18049>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37934] Docs: Clarify NotImplemented use cases

2019-09-04 Thread Kyle Stanley


Kyle Stanley  added the comment:

> As you say, we currently have only one usage of NotImplemented outside its 
> intended purpose. Maybe we should wait to see whether it becomes at least a 
> little bit more popular, before thinking about blessing it.

> I know at least 3 in CPython, so it's not so rare to use NotImplemented for 
> something else than binary operators


I'm thinking that it might be worthwhile to start a discussion about this on 
python-dev, to see what the others think about how the NotImplemented docs 
should be defined, and whether or not it should be used outside of binary 
operators.

Based on the feedback from Raymond and Vedran, I'm in agreement that it might 
not be the best idea in the long term to keep adding additional use cases to 
the documentation. But, it really doesn't make much sense to me that the 
constant is entirely defined around a single use case in the doc while we're 
using it for other purposes throughout CPython.

--

___
Python tracker 
<https://bugs.python.org/issue37934>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37245] Azure Pipeline 3.8 CI: multiple tests hung and timed out on macOS 10.13

2019-09-05 Thread Kyle Stanley


Kyle Stanley  added the comment:

It looks like the Azure macOS tests timed out again in the recently opened 
PR-15688. Specifically, for test_multiprocessing_spawn and test_functools (both 
of which also timed out in PR-15651, which Victor mentioned earlier):

0:26:41 load avg: 2.89 [418/419/1] test_multiprocessing_spawn crashed (Exit 
code 1) -- running: test_functools (14 min 38 sec)
Timeout (0:20:00)!

0:32:03 load avg: 3.17 [419/419/2] test_functools crashed (Exit code 1)
Timeout (0:20:00)!

Build logs: 
https://dev.azure.com/Python/cpython/_build/results?buildId=49868&view=logs&j=18d1a34d-6940-5fc1-f55b-405e2fba32b1

As far as I can tell, PR-15688 should have had no direct influence on 
test_multiprocessing_spawn or test_functools.

> Maybe macOS on Azure is running slower and we should just increase the 
> timeout?

> Yeah, I agree that increasing the timeout shouldn't be the answer here. 

Since this seems to be affecting multiple PRs, would it be appropriate to 
attempt to increase the timeout duration as a temporary fix and open an issue 
for further investigation on the cause of the intermittent slowdown on those 
tests?

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue37245>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

2019-09-06 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Any plan to reapply my change, or fix it?

I can try to help with this. I'm not the most familiar with the internals of 
asyncio, but I think it would provide a good learning experience.

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue34037>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

2019-09-08 Thread Kyle Stanley


Change by Kyle Stanley :


--
pull_requests: +15390
pull_request: https://github.com/python/cpython/pull/15735

___
Python tracker 
<https://bugs.python.org/issue34037>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

2019-09-08 Thread Kyle Stanley


Kyle Stanley  added the comment:

I've opened PR-15735 which applies the same functionality as Victor's PR-13786, 
but adds the public getter and setter methods (for both AbstractEventLoop and 
BaseEventLoop) as requested by Andrew. 

Since this is still causing intermittent CI failures from test_asyncio, I think 
it's important to implement these changes in some form in the near future.

--
priority: normal -> high

___
Python tracker 
<https://bugs.python.org/issue34037>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37635] Using constant for whence arg in seek()

2019-09-14 Thread Kyle Stanley


Change by Kyle Stanley :


--
keywords: +patch
pull_requests: +15757
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16147

___
Python tracker 
<https://bugs.python.org/issue37635>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37635] Using constant for whence arg in seek()

2019-09-14 Thread Kyle Stanley


Kyle Stanley  added the comment:

Created GH-16147 for replacing the *from_what* argument with *whence* in the IO 
tutorial. 

I would like to consider following up on this with another PR that adds the IO 
constants `SEEK_SET`, `SEEK_CUR`, and `SEEK_END` to the tutorial. Those 
constants would be particularly useful for new users of the language, and would 
likely make the tutorial easier to understand for those who don't have prior 
experience with using `seek()`.

However, I figured that replacing *from_what* with *whence* would be 
significantly less controversial and easier to review, which is why they will 
be in separate PRs.

--

___
Python tracker 
<https://bugs.python.org/issue37635>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37635] Using constant for whence arg in seek()

2019-09-14 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Thanks you Kyle!

No problem, thanks for merging it Antoine!

What do you think of the followup PR to make use of the SEEK_* constants listed 
in the documentation? I think it would be useful to at least mention them in 
the tutorial, or even make use of them directly in the examples.

--

___
Python tracker 
<https://bugs.python.org/issue37635>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26360] Deadlock in thread.join on Python 2.7/macOS

2019-09-14 Thread Kyle Stanley


Kyle Stanley  added the comment:

> FWIW, I've confirmed again that the exact same script on the same machine 
> seems fine under Python 3.x. Given the imminent demise of Python 2, perhaps 
> this issue is just destined to be an unsolved historical oddity.

Since it doesn't seem to be occurring across every macOS platform, is specific 
to 2.7, and isn't proving to be an easy fix, it's probably not worthwhile to 
invest a significant amount of further time into the issue. Thanks for working 
on it though, at least we tried. (:

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue26360>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters failed on AMD64 Windows8.1 Refleaks 3.8

2019-09-15 Thread Kyle Stanley


Kyle Stanley  added the comment:

Upon digging through Modules/_xxsubinterpretersmodule.c, I noticed that on line 
2059, `PyInterpreterState_Delete(interp);` is commented out 
(https://github.com/python/cpython/blob/bf169915ecdd42329726104278eb723a7dda2736/Modules/_xxsubinterpretersmodule.c#L2059).
 

This was done when _xxsubinterpretersmodule.c was first added by Eric Snow 
(https://github.com/python/cpython/blob/bf169915ecdd42329726104278eb723a7dda2736/Modules/_xxsubinterpretersmodule.c#L2059),
 so it seems to have been done intentionally but I don't understand why.

Is this because `Py_EndInterpreter()` is supposed to shutdown the interpreter, 
so `PyInterpreterState_Delete()` isn't needed? If so, that still doesn't 
particularly explain why it was commented out. Perhaps Eric can elaborate.

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters failed on AMD64 Windows8.1 Refleaks 3.8

2019-09-15 Thread Kyle Stanley


Kyle Stanley  added the comment:

Note that I'm not particularly experienced with the c-api, I'm just trying to 
take a stab at understanding why the buildbot failure is occuring.

--

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters failed on AMD64 Windows8.1 Refleaks 3.8

2019-09-15 Thread Kyle Stanley


Kyle Stanley  added the comment:

Clarification: In the above comment when I was referring to the commit made by 
Eric Snow, I meant to link to 
https://github.com/python/cpython/commit/7f8bfc9b9a8381ddb768421b5dd5cbd970266190.

--

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters failed on AMD64 Windows8.1 Refleaks 3.8

2019-09-15 Thread Kyle Stanley


Kyle Stanley  added the comment:

Upon further reading of the documentation and some experimentation, it would 
definitely not make sense to call `PyInterpreterState_Delete` here (since we're 
only closing the sub-interpreter in the current thread), so that's not the 
issue. I still have no idea as to why it's commented out there, but that's 
besides the point.

I think I may have found the potential cause of the test failure though. 
Looking into `test_still_running()`, it's clear that the intention is for a 
RuntimeError to be raised if `interpreters.destroy()` is called on a currently 
running interpreter:

```
def test_still_running(self):
main, = interpreters.list_all()
interp = interpreters.create()
with _running(interp):
with self.assertRaises(RuntimeError):
interpreters.destroy(interp)
self.assertTrue(interpreters.is_running(interp))
```

However, within interp_destroy 
(https://github.com/python/cpython/blob/56a45142e70a1ccf3233d43cb60c47255252e89a/Modules/_xxsubinterpretersmodule.c#L2024),
 it is apparent that the RuntimeError is ONLY raised when attempting to destroy 
the current interpreter:

```
if (interp == current) {
PyErr_SetString(PyExc_RuntimeError,
"cannot destroy the current interpreter");
return NULL;
}
```

When attempting to destroy a running interpreter, NULL is returned without 
raising the RuntimeError: 

```
if (_ensure_not_running(interp) < 0) {
return NULL;
}
```

This was my first guess at a solution:

```
if (_ensure_not_running(interp) < 0) {
PyErr_SetString(PyExc_RuntimeError,
"cannot destroy a running interpreter")
return NULL;
}
```

But, within `_ensure_not_running()` 
(https://github.com/python/cpython/blob/56a45142e70a1ccf3233d43cb60c47255252e89a/Modules/_xxsubinterpretersmodule.c#L1842),
 a RuntimeError is supposed to be raised from:

```
if (is_running) {
PyErr_Format(PyExc_RuntimeError, "interpreter already running");
return -1;
}
```

Initially, I was unsure of the difference between `PyErr_SetString()` and 
`PyErr_Format()`, so I referred to the documentation. `PyErr_Format()` is 
similar, but it also returns NULL. If I'm not mistaken doesn't mean that the 
`return -1` within the `if (is_running)` bock is effectively ignored? If so, 
this would potentially explain the problem... 

I think the appropriate solution would be to replace `PyErr_Format()` with 
`PyErr_SetString()` within `_ensure_not_running()`. Also, I think it would be 
more useful to additionally raise the RuntimeError within `interp_destroy()` if 
the interpreter is running, to provide a more useful and specific error 
message. "cannot destroy a running interpreter" is far more useful for 
debugging purposes than a more generic "interpreter already running".

I plan on opening a PR to make these changes in the next few days. Let me know 
what you think Victor.

--

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters failed on AMD64 Windows8.1 Refleaks 3.8

2019-09-15 Thread Kyle Stanley


Kyle Stanley  added the comment:

Clarification: 
"If I'm not mistaken doesn't mean that the `return -1` within ..." 

Should instead be:
"If I'm not mistaken doesn't this mean that the `return -1` within ..."

(I am really looking forward to moving issue over to GitHub at some point. It's 
nice to be able to edit comments and properly format code blocks...)

--

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38154] test__xxsubinterpreters: random failures on AMD64 FreeBSD CURRENT Shared 3.x

2019-09-16 Thread Kyle Stanley


Kyle Stanley  added the comment:

I believe I found a potential fix, see 
https://bugs.python.org/issue37224?@ok_message=msg%20352516%20created%0Aissue%2037224%20message_count%2C%20messages%20edited%20ok&@template=item#msg352514.
 Should I attach the PR to that issue or this one?

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue38154>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters failed on AMD64 Windows8.1 Refleaks 3.8

2019-09-16 Thread Kyle Stanley


Kyle Stanley  added the comment:

Upon further consideration, I don't think this will address the issue. If the 
RuntimeError was not being raised, this failure would be consistent rather than 
intermittent. 

I think I have have gotten a bit mixed up, even if the return value of 
PyErr_Format is NULL, it would not implicitly return NULL upon being called and 
exit the function early (like a macro could) . I'm not experienced with 
programming in C, I only started learning it more recently (Python, Java, and 
C# have been my primary languages) when I started contributing to CPython in 
June.

Instead, I suspect this is likely a concurrency issue, where multiple threads 
are trying to access the same sub-interpreter during ``interp_destroy()``. The 
solution will likely involve finding the correct place for a lock. I'll 
continue to work on this and see if I can find a solution.

--

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

2019-09-19 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Thanks, Kyle!

No problem, and thanks for all of the help from Andrew, Yury, and Victor!

> IMHO it will make asyncio more reliable, especially for tests on the CI.

Awesome, that was my primary intention. (:

> If it becomes an issue in Python 3.9 (executor hangs forever),

Feel free to add me to the nosy list if you have to open an issue for it, I'd 
be glad to help out with this if it becomes an issue.

--

___
Python tracker 
<https://bugs.python.org/issue34037>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

2019-09-19 Thread Kyle Stanley


Kyle Stanley  added the comment:

> I'm one of the first to advocate to replace ugly macros with clean static 
> inline functions. Macros are evil and can be too easily misused.

As someone who has only more recently started learning the C-API (and C in 
general), I'm certainly in favor of replacing macros with functions when 
possible. I might be a bit biased, but it definitely makes the code a lot 
easier to understand (especially the macros with implicit returns). As long as 
there's not a significant performance loss and it doesn't introduce new 
complications, I don't see an issue with it.

>From my understanding, readability isn't as high of a priority in the C code, 
>but certainly there's some benefit to improving areas that are more difficult 
>to understand. The easier the code is to understand, the lower the overall 
>maintenance cost becomes.

Of course, this should certainly be done in moderation to reduce the 
introduction of unnecessary new bugs and the cost in reviewer time for more 
pressing concerns.

--

___
Python tracker 
<https://bugs.python.org/issue37812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters failed on AMD64 Windows8.1 Refleaks 3.8

2019-09-19 Thread Kyle Stanley

Kyle Stanley  added the comment:

Is there a currently reliable way of accessing the GIL functions within the 
sub-interpreters, without causing deadlock issues? I was trying to follow the 
advice in the documentation 
(https://docs.python.org/3/c-api/init.html?highlight=global%20interpreter%20lock#bugs-and-caveats).

"It is highly recommended that you don’t switch sub-interpreters between a pair 
of matching PyGILState_Ensure() and PyGILState_Release() calls."

But it seemed that any attempt to use any of the PyGIL* calls within 
``interp_destroy()`` in a meaningful way resulted in a deadlock, even if it was 
done away from the sub-interpreter switching.

My next idea would be to add a conditional check to see if the current thread 
has ownership of the GIL, and using ``PyEval_RestoreThread()`` to acquire it if 
it doesn't. This would be followed by releasing the GIL with 
``PyThreadState_Get()`` at the end of the function. I'll try experimenting with 
that idea next.

--

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38154] test__xxsubinterpreters: random failures on AMD64 FreeBSD CURRENT Shared 3.x

2019-09-20 Thread Kyle Stanley


Change by Kyle Stanley :


--
keywords: +patch
pull_requests: +15879
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16293

___
Python tracker 
<https://bugs.python.org/issue38154>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38154] test__xxsubinterpreters: random failures on AMD64 FreeBSD CURRENT Shared 3.x

2019-09-20 Thread Kyle Stanley


Kyle Stanley  added the comment:

For an updated explanation of the PR, see https://bugs.python.org/msg352835 or 
the comments in the PR itself.

--
stage: patch review -> 

___
Python tracker 
<https://bugs.python.org/issue38154>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

2019-09-20 Thread Kyle Stanley


Kyle Stanley  added the comment:

> You can find my email in Git, and I'm on Zulip and Discourse; and I'd be 
> happy to start or follow a thread in a forum you think appropriate.  Or if 
> you'd rather drop it entirely, that's fine too.

I think opening a thread in https://discuss.python.org/c/users to talk about 
deciding between the usage of functions and macros (discussing when each may be 
appropriate) would be beneficial to the community at large.

> The recommendation of the senior most developer is being ignored, and now two 
> developers are speaking in terms of strong belief systems and labeling 
> long-stable code as "evil".  This doesn't bode well and it makes it difficult 
> to conduct reasoned discourse.  

Apologies if I added to that, I certainly respect your judgement on this issue. 
Pretty much everyone involved in this discussion has more experience in working 
with the CPython C-API than I do, you most of all (as far I'm aware). My 
perspective was coming from someone attempting to understand it better, and 
explaining how those learning the C-API might find it confusing.

I don't find the code itself to be at all "evil", just potentially confusing. 
That long-stable code has provided a lot of benefit over the years, and I can 
definitely appreciate the effort that was put into writing it.

--

___
Python tracker 
<https://bugs.python.org/issue37812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37812] Make implicit returns explicit in longobject.c (in CHECK_SMALL_INT)

2019-09-21 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Is there an "aesthetic code cleanup" patch that's ever turned out well? ;-)

>From what I can tell, any remotely extensive aesthetic code patch I've seen 
>has been highly controversial. I think they can have value in some cases, but 
>the value relative to the potential cost may have been a bit overstated in 
>this particular case.

--

___
Python tracker 
<https://bugs.python.org/issue37812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-23 Thread Kyle Stanley


Change by Kyle Stanley :


--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-23 Thread Kyle Stanley


Kyle Stanley  added the comment:

> We have to document that Process objects use a third kind of stream object 
> that doesn't match either the old or new APIs, and how this one works

>From my perspective, this point would have the largest user learning cost due 
>to the stream object having a completely different API. As a result, I'm 
>significantly more in favor of adding the two new subprocess functions.

--

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38267] Add thread timeout for loop.shutdown_default_executor

2019-09-24 Thread Kyle Stanley


New submission from Kyle Stanley :

Currently, for the recently added coroutine `loop.shutdown_default_executor()`, 
the executor shutdown can wait indefinitely for the threads to join. Under 
normal circumstances, waiting on the threads is appropriate, but there should 
be a timeout duration in the situation that the threads unable to finish 
joining. 

The motivation for this was based on the comments from Andrew Svetlov and Yury 
Selivanov in GH-16284.

The original idea from Andrew was to add the timeout duration as a default for 
a new parameter in `asyncio.run()` and `loop.shutdown_default_executor()`. 
However, Yury would prefer for this to be defined as a constant instead and not 
as a parameter for `asyncio.run()` to avoid the creation of an excessive number 
of parameters to tweak for the user.

I will attach a PR that adds the constant and the parameter for 
`loop.shutdown_default_executor()`, which will passed as an argument to 
`thread.join()`.

--
messages: 353115
nosy: aeros167, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Add thread timeout for loop.shutdown_default_executor
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue38267>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38267] Add thread timeout for loop.shutdown_default_executor

2019-09-24 Thread Kyle Stanley


Change by Kyle Stanley :


--
keywords: +patch
pull_requests: +15941
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16360

___
Python tracker 
<https://bugs.python.org/issue38267>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-27 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Andrew, do you want me to submit a PR or you can do it?

Since this has been elevated to a release blocker, I wouldn't mind helping to 
revert this ASAP. I can open a PR to fix it today.

--

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-27 Thread Kyle Stanley


Kyle Stanley  added the comment:

> You'll need to be careful to only revert the new functions & the 
> asyncio.Stream class.

So far the trickiest part has proven to be the tests (specifically 
test_streams.py) and keeping the deprecation warning for passing explicit loop 
arguments. I've had to be careful to be certain that no tests were 
unintentionally removed.

> Due to proximity to the deadline, please be prepared that we might need to 
> abandon your pull request if it's not ready by Sunday morning

No problem, I'll make sure to allow anyone with write access (yourself and 
Andrew) to edit the PR I open directly to make any needed changes. That way, at 
least any progress I make on it can help reduce the amount of work for you two.

--

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-27 Thread Kyle Stanley


Kyle Stanley  added the comment:

Currently focusing on the Lib/asyncio/* and Lib/test/* changes. Working on doc 
changes next, but that should be significantly easier. 

In addition to 
https://github.com/python/cpython/commit/23b4b697e5b6cc897696f9c0288c187d2d24bff2
 (main commit from Andrew that added asyncio.Stream and new functions), I've 
also had to remove https://github.com/python/cpython/commit/4cdbc452ce3 (minor 
asyncio test change from Pablo) due to it causing issues with the other tests 
from deleting asyncio.StreamReader and asyncio.StreamWriter.

--

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-27 Thread Kyle Stanley


Change by Kyle Stanley :


--
keywords: +patch
pull_requests: +16024
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16445

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-27 Thread Kyle Stanley


Change by Kyle Stanley :


--
pull_requests: +16035
pull_request: https://github.com/python/cpython/pull/16455

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38164] polishing asyncio Streams API

2019-09-29 Thread Kyle Stanley


Kyle Stanley  added the comment:

This should no longer be a release blocker for 3.8 with the reversion of the 
new asyncio streaming API in GH-16455.

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue38164>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-29 Thread Kyle Stanley


Change by Kyle Stanley :


--
stage: commit review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-29 Thread Kyle Stanley


Kyle Stanley  added the comment:

> I've reverted the code. Andrew, would really appreciate if you could quickly 
> do a post commit review.

Oops, I'll reopen it.

--

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-29 Thread Kyle Stanley


Change by Kyle Stanley :


--
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38242] Revert the new asyncio Streams API

2019-09-29 Thread Kyle Stanley


Change by Kyle Stanley :


--
stage: resolved -> commit review

___
Python tracker 
<https://bugs.python.org/issue38242>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38164] polishing asyncio Streams API

2019-09-29 Thread Kyle Stanley


Kyle Stanley  added the comment:

Closed by the new asyncio stream API reversion in GH-16485 and GH-16482.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38164>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38351] Modernize email example from %-formatting to f-string

2019-10-02 Thread Kyle Stanley


Kyle Stanley  added the comment:

Welcome back from the OOOS break Mariatta!

> My question (and it's just that) is whether we've made a decision to prefer 
> one formatting syntax over others (outside of examples discussing the 
> formatting approaches themselves).

I agree that we should reach a consensus on the preferred string formatting 
style. However, there seems to be two separate questions here:

1) Should the code examples in the docs be updated to use f-strings?

2) Should ALL instances of the old string formatting be updated to use 
f-strings? This would affect every *.py; potentially leading to additional code 
churn, which adds clutter to the commit logs and git blame.

The first one is far less costly and has very minimal risk of breakage. The 
cost of updating every *.py to use f-strings is worth considering, but is 
significantly higher and has more potential consequences, especially for the 
regression tests. I'm personally in favor of updating the code examples first 
and discussing the second question in a python-dev thread due to the wide 
impact.

> If a decision is made to prefer one over others, it's worth making that 
> decision separately, and then using separate PRs to deal with updates to 
> different parts of the docs.

Once we reach a decision on the matter, I think this particular issue could 
serve as a great first PR for a new contributor to become familiar with the 
workflow, so it would be a good candidate for the "newcomer friendly" label. 
Most python users are well acquainted with string formatting. I wouldn't mind 
opening a PR to fix it myself, but I think that leaving it open for a new 
contributor to work on as an intro to the workflow would be far more beneficial.

Although there may be a benefit to use f-strings instead here, there's 
certainly no rush to have it completed in a short period of time. I would be in 
favor of having each PR address a single documentation file. This would help 
accelerate the review process and provide a valuable learning experience to a 
greater number of new contributors, in comparison to a single PR that updates 
every single code example in the docs.

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue38351>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38351] Modernize email example from %-formatting to f-string

2019-10-02 Thread Kyle Stanley


Kyle Stanley  added the comment:

> so it would be a good candidate for the "newcomer friendly" label

Never mind, just noticed this was already labeled as newcomer friendly. I only 
saw the "easy" label at first. (:

If consensus is reached for this, we can open a separate issue for addressing 
the other doc files, something along the lines of "Update code examples to use 
f-strings". As mentioned earlier I think it would be worth having each new 
contributor update all of the instances in a single *.rst in a PR, but it can 
be a single issue.

--

___
Python tracker 
<https://bugs.python.org/issue38351>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38356] test_asyncio: SubprocessThreadedWatcherTests leaks threads

2019-10-02 Thread Kyle Stanley


Kyle Stanley  added the comment:

I can try to work on fixing this.

--
nosy: +aeros167

___
Python tracker 
<https://bugs.python.org/issue38356>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38351] Modernize email example from %-formatting to f-string

2019-10-02 Thread Kyle Stanley


Kyle Stanley  added the comment:

> I definitely think we should not modify any code in the stdlib just to switch 
> to f-strings.

Does this also apply to updating code to use f-strings in an area that's 
already being modified for a functional purpose? 

I agree that that we shouldn't update stdlib code for the sole purpose of 
switching to f-strings, but if a function or method is already being changed 
for another purpose, I don't think updating the formatting to use f-strings is 
an issue. This would probably have to be decided on a case-by-case basis though.

--

___
Python tracker 
<https://bugs.python.org/issue38351>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38356] test_asyncio: SubprocessThreadedWatcherTests leaks threads

2019-10-02 Thread Kyle Stanley


Kyle Stanley  added the comment:

First I'll work on adding a new method. Here's a few potential names, ordered 
roughly by my preferences:

1) join_threads()

2) shutdown_threads()

3) shutdown_threadpool()

The first and second options are roughly equal, but I think join_threads() is a 
bit more specific to what this is doing.

The third option is because a group of threads could be considered a 
"threadpool", but ThreadedChildWatcher doesn't utilize a dedicated threadpool 
class of any form, it just uses an internal dict named `_threads` that maps 
process IDs to threads. So this name might be potentially misleading.

I'm also wondering whether or not this method should have a timeout parameter 
that defaults to None. I'm thinking we can probably leave it out for now, but I 
wanted to hear some other opinions on this.

For now, I'll be implementing this as a regular method, but I could make it a 
coroutine if that's desired. Admittedly, I don't enough have knowledge of the 
realistic use cases for ThreadedChildWatcher to know if it would be 
particularly useful to have an awaitable method to join the threads.

Another consideration is if we want this method to join the threads to be 
called in `ThreadedChildWatcher.close()`. I think it makes sense from a design 
perspective to join all of the threads when the close method is used. 

Plus, it would allow the method to be integrated with the tests better. 
Currently, the test uses the same `setUp()` and `tearDown()` for each of the 
different subprocess watchers. If it weren't called in the `close()` method, 
we'd have to add an `if isinstance(watcher, ThreadedChildWatcher)` conditional 
in `tearDown()`. So, I would prefer to have the method called from `close()`.

Finally, should this method be part of the public API, or just a private 
method? As mentioned earlier, I'm not overly aware of the realistic use cases 
for ThreadedChildWatcher. As a result, I don't know if it would be especially 
useful for users to call this method independently, especially if this were 
called from `close()` as I was suggesting earlier. 

After we reach a consensus on the implementation details and API design for the 
new method, I'll work on adding an entry in the documentation (if it should be 
public) and updating test_subprocess.SubprocessThreadedWatcherTests to utilize 
it.

--

___
Python tracker 
<https://bugs.python.org/issue38356>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38356] test_asyncio: SubprocessThreadedWatcherTests leaks threads

2019-10-02 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Another consideration is if we want this method to join the threads to be 
> called in `ThreadedChildWatcher.close()`.

An additional benefit of having the method called from `close()` is that it 
means we don't have to modify the tests directly. Also, ThreadedChildWatcher's 
implementation of `close()` does nothing at the moment.

--

___
Python tracker 
<https://bugs.python.org/issue38356>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38356] test_asyncio: SubprocessThreadedWatcherTests leaks threads

2019-10-02 Thread Kyle Stanley


Change by Kyle Stanley :


--
keywords: +patch
pull_requests: +16140
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16552

___
Python tracker 
<https://bugs.python.org/issue38356>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38351] Modernize email example from %-formatting to f-string

2019-10-02 Thread Kyle Stanley


Kyle Stanley  added the comment:

> For the most part, templating examples can be switched to the .format() style 
> but shouldn't be switched to f-strings.

Is there no specific use case for the older "%s" % sub template that .format() 
doesn't have?

> The former technique is still necessary if someone wants to move templates to 
> an external file

Interesting, I wasn't aware of that. This seems like it might be worth a brief 
mention in https://docs.python.org/3.9/library/stdtypes.html#str.format. 

> AFAICT, it will be around forever, so people still need to see some examples 
> of each.

To allow users to see examples of each, would you suggest that we should leave 
the existing .format() examples as is and have more recently created examples 
use f-strings? I'd be okay with this, as long as there's a balance of both 
everywhere (particularly in the tutorial).

Personally, I think that the f-string examples should be used more commonly 
used since they're generally more concise and readable. But, I can definitely 
understand the purpose of leaving the older ones around as long as they have a 
specific use case and are still utilized.

--

___
Python tracker 
<https://bugs.python.org/issue38351>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38344] activate.bat else needs to be on the same line as the if

2019-10-10 Thread Kyle Stanley


Change by Kyle Stanley :


--
stage: patch review -> commit review

___
Python tracker 
<https://bugs.python.org/issue38344>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38355] ntpath.realpath() fails on sys.executable

2019-10-10 Thread Kyle Stanley


Change by Kyle Stanley :


--
stage: backport needed -> commit review

___
Python tracker 
<https://bugs.python.org/issue38355>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38359] pyw.exe opens console window in Windows 10

2019-10-10 Thread Kyle Stanley


Change by Kyle Stanley :


--
stage: backport needed -> commit review

___
Python tracker 
<https://bugs.python.org/issue38359>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37224] test__xxsubinterpreters fails randomly

2019-10-14 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Kyle Stanley proposed a fix: PR 16293.

Note that I'm not confident about the fix I proposed in GH-16293, it was more 
of an idea to fix the intermittent failures more than anything. It definitely 
needs review from someone knowledgeable about sub-interpreters and the 
PyInterpreterState API. This isn't an area that I'm experienced in, but I am 
interested in it.

--

___
Python tracker 
<https://bugs.python.org/issue37224>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31387] asyncio should make it easy to enable cooperative SIGINT handling

2019-10-14 Thread Kyle Stanley


Change by Kyle Stanley :


--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue31387>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28533] Replace asyncore

2019-10-14 Thread Kyle Stanley


Change by Kyle Stanley :


--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue28533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27873] multiprocessing.pool.Pool.map should take more than one iterable

2019-10-22 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Is it still open? What else needs to be done?

Yes, this patch needs to be translated into a GitHub PR. See 
https://devguide.python.org/pullrequest/ for more information on our PR 
workflow if you're not already familiar with it. Since naught101 wrote a patch, 
we should have their name in "Co-Authored-By" of the final commit to the PR if 
they have a GitHub account. If not, just mention it in a comment to the PR that 
they originally wrote the patch. 

Feel free to @mention me when you open the PR so I can help to review it, 
although this will likely require a final approval from a maintainer of 
multiprocessing module before it can be merged.

--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue27873>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38323] asyncio: MultiLoopWatcher has a race condition (test_asyncio: test_close_kill_running() hangs on AMD64 RHEL7 Refleaks 3.x)

2019-10-24 Thread Kyle Stanley


Change by Kyle Stanley :


--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue38323>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38564] test_asyncio: test_run_coroutine_threadsafe_with_timeout() has a race condition

2019-10-25 Thread Kyle Stanley


Kyle Stanley  added the comment:

I can confirm Victor's method of reproducing the failure consistently, by using 
asyncio.sleep(1e-9) within `RunCoroutineThreadsafeTests.add()` instead of the 
current asyncio.sleep(0.05).

I also experimented with adjusting the sleep time, to figure out the "breaking 
point" where I could no longer run `./python -m test test_asyncio -m 
test_run_coroutine_threadsafe_with_timeout -v -F` without failures (within 
~10,000 tests). From my results, the lowest reliable value was 0.001. At 1e-4, 
I was able to consistently reproduce the failure reported above:

FAIL: test_run_coroutine_threadsafe_with_timeout 
(test.test_asyncio.test_tasks.RunCoroutineThreadsafeTests)
Test coroutine submission from a thread to an event loop
--
Traceback (most recent call last):
  File "/home/aeros/repos/cpython/Lib/test/test_asyncio/test_tasks.py", line 
3210, in test_run_coroutine_threadsafe_with_timeout
self.loop.run_until_complete(future)
AssertionError: TimeoutError not raised

The failure becomes increasingly consistent with lowered time as expected, but 
at 1e-5 I was able to repeatedly reproduce the failure within 10 iterations of 
the test. At 1e-4 it took around 3000 iterations before failing (across 
multiple runs).

--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue38564>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38323] asyncio: MultiLoopWatcher has a race condition (test_asyncio: test_close_kill_running() hangs on AMD64 RHEL7 Refleaks 3.x)

2019-10-25 Thread Kyle Stanley


Kyle Stanley  added the comment:

> I'm able to reproduce the issue locally using the command:
./python -m test test_asyncio 
--match=test.test_asyncio.test_subprocess.SubprocessMultiLoopWatcherTests.test_close_kill_running
 -v -F -j20 --timeout=30.0

I was unable to reproduce this locally on OS: Arch Linux 5.3.7 x86_64 and CPU: 
Intel i5-4460 from the latest commit (heads/master:96b06aefe2) within a 
reasonable number of tests (~1). I also tried differing number of jobs, up 
to 100. What OS were you able to reproduce it locally on using that 
configuration?

This is of course still an issue since it's still occurring on multiple 
buildbots intermittently and Victor was able to reproduce it locally, but I was 
not able to do so on my local setup. This will be significantly harder to debug 
without a reliable means of reproducing the failure.

--

___
Python tracker 
<https://bugs.python.org/issue38323>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38564] test_asyncio: test_run_coroutine_threadsafe_with_timeout() has a race condition

2019-10-25 Thread Kyle Stanley


Kyle Stanley  added the comment:

> IHMO a test must not depend on time.

I would agree, the tests should optimally not depend on time. My above comment 
was primarily to confirm that the failure was able to be consistently 
reproduced, along with the minimum conditions to do so.

Yury had been the one to change it to "asyncio.sleep(0.05)" from the previous 
"asyncio.sleep(1)" in the following commit: 
https://github.com/python/cpython/commit/abe9625eeb71e40f042ccfccfe6a4489a6dcdf35
 (Nov 13, 2015).

Perhaps he might have some insight or ideas as to how we could improve the test 
to use a more reliable means of synchronization that has been implemented since 
that change was made. I'll add him to the nosy list.

--
nosy: +yselivanov

___
Python tracker 
<https://bugs.python.org/issue38564>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecating MultiLoopChildWatcher

2019-10-25 Thread Kyle Stanley


New submission from Kyle Stanley :

Proposal: 
Deprecate the alternative process watcher implementation to 
ThreadedChildWatcher, MultiLoopChildWatcher.

Motivation:
The idea for this proposal came from a comment from Andrew Svetlov in GH-16552:
"I believe after polishing ThreadedChildWatcher we can start deprecation of 
watchers subsystem at all, it generates more problems than solves."

Although Andrew suggested the idea of deprecating the process watchers 
subsystem entirely, I think that it would be adequate to start with just 
deprecating MultiLoopChildWatcher and proceeding from there. This is because 
the other three watcher implementations are significantly more stable in 
comparison (based on number of issues), have less complex implementations, and 
have far more widespread usage across public repositories. I would not be 
opposed to deprecating the other alternative watchers as well, but 
MultiLoopChildWatcher likely has the most justification for removal.

Details:
The class MultiLoopChildWatcher has a fairly complex implementation, causes a 
number of issues, is fairly unstable, is rarely utilized in comparison with the 
rest of the watchers API. It seems to have become a significant burden on 
maintenance for asyncio development without providing a significant benefit to 
the vast majority of users. 

Current unresolved MultiLoopChildWatcher issues:
https://bugs.python.org/issue38323
https://bugs.python.org/issue38182
https://bugs.python.org/issue37573

(3 out of the 5 open process watcher issues are caused by MultiLoopChildWatcher)

GitHub code usage comparison:
MultiLoopChildWatcher: 
https://github.com/search?l=Python&q=MultiLoopChildWatcher&type=Code (20 
results)
ThreadedChildWatcher: 
https://github.com/search?l=Python&q=ThreadedChildWatcher&type=Code (77 
results) 
FastChildWatcher: 
https://github.com/search?l=Python&q=FastChildWatcher&type=Code (4,426 results)
SafeChildWatcher: 
https://github.com/search?l=Python&q=SafeChildWatcher&type=Code (7,007 results)
All of asyncio usage: https://github.com/search?l=Python&q=asyncio&type=Code 
(599,131 results)

Note that for the above results, it also includes matches in the CPython 
repository and for repositories that have exact copies of 
Lib/asyncio/unix_events.py. Also, ThreadedChildWatcher likely has significantly 
less results since it's already the default watcher returned by 
asyncio.get_child_watcher(), so there's less need to explicitly declare it 
compared to the others. There are of course private repositories and non-GitHub 
repositories this doesn't include, but it should provide a general idea of 
overall usage.

My experience in interacting with asyncio users is similar to the results. The 
process watchers subsystem seems to be minimally used compared to the rest of 
asyncio, with ThreadedChildWatcher likely being the most used as the default 
returned from `asyncio.get_child_watcher()`. I have not personally seen a 
realistic example of usage of MultiLoopChildWatcher outside of python/cpython, 
and all of the results returned on GitHub were copies of 
Lib/asyncio/unix_events.py or Lib/test/test_asyncio/test_subprocess.py.

I would be interested in working on this issue if it is approved, as I think it 
would provide a significant long term reduction in maintenance for asyncio; 
allowing us to focus on the improvement and development of other features that 
benefit a far larger audience.

--
components: asyncio
messages: 355372
nosy: aeros, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: Deprecating MultiLoopChildWatcher
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecating MultiLoopChildWatcher

2019-10-25 Thread Kyle Stanley


Change by Kyle Stanley :


--
assignee:  -> aeros
nosy: +vstinner
status: open -> pending

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecating MultiLoopChildWatcher

2019-10-25 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Didn't we just add MultiLoopChildWatcher in 3.8?

Yep, that's correct: 
https://docs.python.org/3/library/asyncio-policy.html#asyncio.MultiLoopChildWatcher

I wasn't aware of that, but it would explain the current lack of usage. I'm 
generally in favor of Andrew's idea to deprecate the watchers subsystem, but 
perhaps if we go through with that we should remove MultiLoopChildWatcher from 
3.8 (if it's not too late to do so) instead of deprecating it.

--

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecating MultiLoopChildWatcher

2019-10-25 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Speaking of watchers -- big +1 from me to drop them all at some point. I 
> would start as early as 3.9.

Yeah that was my initial plan, to start the deprecation in 3.9 and finalize the 
removal in 3.11. We might be able to make an exception for 
MultiLoopChildWatcher and just remove it from 3.8 though. 

I think it will be necessary to fix the issues in the near future if we want to 
keep MultiLoopChildWatcher in 3.8 (as they apply to 3.8 and 3.9). I would 
certainly not be ecstatic about the idea of removing something that was just 
recently added (similar to how we had to revert the new streaming changes due 
to API design), but I'm even less in favor of keeping something around that's 
not stable in a final release. 

Based on my investigation of https://bugs.python.org/issue38323, it seems like 
it will be a rather complex issue to solve.

--
status: open -> pending

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecate Process Child Watchers

2019-10-25 Thread Kyle Stanley


Change by Kyle Stanley :


--
title: Deprecating MultiLoopChildWatcher -> Deprecate Process Child Watchers

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecating MultiLoopChildWatcher

2019-10-25 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Kyle, why are you resetting the status to "Pending"?

That was an accident, I think I had that set already on the page and submitted 
my comment just after you did yours.

I'm going to change the title of the issue to "Deprecate Process Child 
Watchers". I started the proposal bit more conservatively because I thought 
that it might be easier to just deprecate MultiLoopChildWatcher at first. But 
seeing as it was just added in 3.8, I don't think it would make as much sense 
to deprecate that one by itself.

--

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecate Process Child Watchers

2019-10-25 Thread Kyle Stanley


Kyle Stanley  added the comment:

> I think FastChildWatcher and SafeChildWatcher should go, ThreadedChildWatcher 
> should be kept default and MultiLoopChildWatcher is an option where 
> ThreadedChildWatcher is not satisfactory.

Okay, I think I can understand the reasoning here. Do you think that 
FastChildWatcher and SafeChildWatcher could be deprecated starting in 3.9 and 
removed in 3.11? If so, I'd be glad to start working on adding the deprecation 
warnings and the 3.9 Whats New entries.

> MultiLoopChildWatcher problems can and should be fixed; there is nothing bad 
> in the idea but slightly imperfect implementation.

Yeah my largest concern was just that the current issues seem especially 
complex to fix and I interpreted your previous comment as "I'd like to 
deprecate all the process watchers in the near future". Thus, it seemed to make 
sense to me that we could start removing them rather than sinking time into 
fixing something that might be removed soon. 

By "slightly imperfect implementation", do you have any ideas for particularly 
imperfect parts of it that could use improvement? 

I feel that I've developed a decent understanding of the implementation for 
ThreadedChildWatcher, but after looking at the race conditions for 
MultiLoopChildWatcher in https://bugs.python.org/issue38323, I'll admit that I 
felt a bit lost for where to find a solution. Primarily because my 
understanding of the signal module is quite limited in comparison to others; 
it's not an area that I've strongly focused on.

--

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecate Process Child Watchers

2019-10-26 Thread Kyle Stanley


Kyle Stanley  added the comment:

> > If asyncio is only run from the main thread, FastChildWatcher is safe, fast 
> > and has low memory footprint, no?

> Unfortunately, no. FastChildWatcher is safe if you can guarantee that no code 
> executed in asyncio main thread AND thread pools spawn subprocesses

Am I misunderstanding something here or is this supposed to be 
"FastChildWatcher is safe if you can guarantee that no code executed *outside 
of* the asyncio main thread AND ..."? Alternatively, "FastChildWatcher is safe 
if you can guarantee that code *only* executed in the asyncio main thread". 
Both of the above have the same functional meaning. 

I think it was a typo, but I just wanted to make sure because the distinction 
from the original makes a functional difference in this case. 

Also, regarding the second part "thread pools spawn subprocesses", is that to 
say that subprocesses can only spawn within the thread pools? As in, 
FastChildWatcher becomes unsafe if subprocesses are spawned from anywhere else?

These answers may be fairly obvious to someone familiar working within the 
internals of FastChildWatcher, but it may not be overly clear to someone such 
as myself who has mostly just read through the documentation and looked over 
the implementation briefly. I'm only familiar with the internals of 
ThreadedChildWatcher.

--

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38591] Deprecate Process Child Watchers

2019-10-26 Thread Kyle Stanley


Kyle Stanley  added the comment:

> But it spawns a new Python thread per process which can be a blocker issue if 
> a server memory is limited.

I understand that there's *some* overhead associated with spawning a new 
thread, but from my impression it's not substantial enough to make a 
significant impact in most cases. Each individual instance of threading.Thread 
is only 64 bytes. Have you seen any recent cases where the server memory is 
limited enough for the memory cost associated with having to spawn an 
additional thread per subprocess becomes the limiting factor?

--

___
Python tracker 
<https://bugs.python.org/issue38591>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34790] Deprecate passing coroutine objects to asyncio.wait()

2019-10-26 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Actually, since Andrew also agrees that we need to deprecate passing 
> coroutines to wait(), I'll keep this issue open until we add an actual 
> DeprecationWarning in 3.8.

Since 3.8 has been released and the deprecation notice is in the 3.8 whatsnew 
document, should we implement the warning in Lib/asyncio/tasks.py? If so, I can 
open a PR. 

> PendingDeprecationWarning

Also, it's not clear to me if this should be a DeprecationWarning or 
PendingDeprecationWarning. The most recent message from Yury in the issue 
suggests a PendingDeprecationWarning, but the actual entry in the documentation 
(https://docs.python.org/3/library/asyncio-task.html?highlight=asyncio%20wait#asyncio.wait)
 seems like it might imply that it would be a DeprecationWarning:

> Deprecated since version 3.8: If any awaitable in aws is a coroutine, it is 
> automatically scheduled as a Task. Passing coroutines objects to wait() 
> directly is deprecated as it leads to confusing behavior.

--
nosy: +aeros
status: pending -> open

___
Python tracker 
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34790] Deprecate passing coroutine objects to asyncio.wait()

2019-10-27 Thread Kyle Stanley


Kyle Stanley  added the comment:

> I think we can add `DeprecationWarning` for 3.9.

If we add the deprecation warning just for 3.9, would the removal release also 
be pushed forward? 

> Honestly, we just missed the issue when were prepared for 3.8

Yeah that's definitely understandable, there were quite a number of major 
changes made in 3.8. It was the first time I saw a deprecation clearly 
documented in a final release that didn't have an associated deprecation 
warning. 

As for 3.8, I would like to update the whatsnew entry to add a link to this bpo 
issue if that's okay, as it does not at the moment. I recently included the 
entry in a PR that added entries for multiple significant changes for asyncio 
in the 3.8, but I forgot to include the bpo link (mostly because I found it in 
the documentation, rather than from the bpo issue).

--

___
Python tracker 
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34790] Deprecate passing coroutine objects to asyncio.wait()

2019-10-27 Thread Kyle Stanley


Kyle Stanley  added the comment:

> It was the first time I saw a deprecation clearly documented in a final 
> release that didn't have an associated deprecation warning. 

I want to clarify that this may be a more common occurrence than I'm realizing, 
I'm not entirely certain. Also it's not intended at all as a criticism, it just 
surprised me since it was the first time I saw it happen.

--

___
Python tracker 
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21002] _sre.SRE_Scanner object should have a fullmatch() method

2019-10-27 Thread Kyle Stanley


Change by Kyle Stanley :


--
nosy: +serhiy.storchaka

___
Python tracker 
<https://bugs.python.org/issue21002>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21002] _sre.SRE_Scanner object should have a fullmatch() method

2019-10-27 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Is anyone able to make a call on whether this issue should be closed, or 
> alternatively give some guidance on what work this issue should encompass?

Added Serhiy to the nosy list, since he's an active maintainer for the re 
module.

--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue21002>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34790] Deprecate passing coroutine objects to asyncio.wait()

2019-10-27 Thread Kyle Stanley


Kyle Stanley  added the comment:

Sounds good, I'll work on opening a PR.

--

___
Python tracker 
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34790] Deprecate passing coroutine objects to asyncio.wait()

2019-10-27 Thread Kyle Stanley


Kyle Stanley  added the comment:

> Regarding 3.8 release notes update -- not sure if it is needed flr docs-only 
> change.

I'm not certain if the entry is necessary; my main concern is just that it's 
already present in the 3.8 release notes/whatsnew without anywhere to look for 
further information. 

> In the current situation, we have so-called *soft deprecation*: bare 
> coroutines are deprecated in docs without any code change. This is perfectly 
> fine, we give our users extra time to prepare for changes.

So is something considered to be a "soft deprecation" if there is no code 
change? If so, I may have made a mistake by specifying a release in the entry I 
recently added for it (https://docs.python.org/3/whatsnew/3.8.html#deprecated):

"The explicit passing of coroutine objects to asyncio.wait() has been 
deprecated and will be removed in version 3.10."

I would propose to adjust this to:

"The explicit passing of coroutine objects to asyncio.wait() has been 
deprecated. (Contributed by Yury Selivanov in :issue:`34790`.)"

Then for 3.9 whatsnew, we could specify the release:

"The explicit passing of coroutine objects to asyncio.wait() has been 
deprecated and will be removed in 3.11."

Thoughts?

--

___
Python tracker 
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34790] Deprecate passing coroutine objects to asyncio.wait()

2019-10-28 Thread Kyle Stanley


Change by Kyle Stanley :


--
pull_requests: +16502
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16975

___
Python tracker 
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34790] Deprecate passing coroutine objects to asyncio.wait()

2019-10-28 Thread Kyle Stanley


Kyle Stanley  added the comment:

GH-16975 Is a simple fix for the asyncio.wait() whatsnew entry for 3.8. I'll 
implement the deprecation warning and add a 3.9 whatsnew entry in a separate 
PR, since those changes won't be backported.

--
stage: patch review -> 

___
Python tracker 
<https://bugs.python.org/issue34790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   >