Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?

2019-01-17 Thread Steve Dower
Eryk Sun's correct and useful answer which I find very disappointing and a great way to discourage contributions. We can, and should, do better, at least by thanking the person for their response before running down a barely related side track. On 17Jan.2019 2209, Steve Dower wrote: > For

Re: [Python-Dev] Source of truth for C-API

2019-01-22 Thread Steve Dower
On 22Jan.2019 1517, Victor Stinner wrote: > I'm not aware of any tool to automatically list the content of the C API. The shell script attached to https://bugs.python.org/issue23903 should be able to do it with different preprocessor values (we originally intended to detect inconsistencies in the

Re: [Python-Dev] CPython on Windows ARM32

2019-02-05 Thread Steve Dower
Just confirming for the list that I'm aware of this and supportive, but am not the dedicated support for this effort. I also haven't reviewed the changes yet, but provided nobody is strongly opposed to taking on a supported platform (without additional releases on python.org), I expect I'll do a b

Re: [Python-Dev] About the future of multi-process Python

2019-02-06 Thread Steve Dower
On 06Feb2019 0906, Antoine Pitrou wrote: For the record there are number of initiatives currently to boost the usefulness and efficiency of multi-process computation in Python. One of them is PEP 574 (zero-copy pickling with out-of-band buffers), which I'm working on. Another is Pierre Glaser's

Re: [Python-Dev] CPython on Windows ARM32

2019-02-06 Thread Steve Dower
On 06Feb2019 0054, Terry Reedy wrote: On 2/5/2019 10:10 PM, Zachary Ware wrote: I'm all for the first two changes (especially the second), and if 10 years of pledged corporate support for a new platform is the price we have to pay for them, I'm ok with that :). I would expect that the main qu

Re: [Python-Dev] CPython on Windows ARM32

2019-02-06 Thread Steve Dower
On 06Feb2019 1423, Christian Heimes wrote: Do you want to update Python 3.8 (master) only or also 3.7? I'm not strictly against updating 3.7. However we have traditionally kept the OpenSSL version of each branch stable. 1.1.1 comes with new features, stricter security settings and some ciphers re

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-13 Thread Steve Dower
On 13Feb2019 0820, Victor Stinner wrote: On my Windows VM, "python" is Python 3.7 :-) In virtual environments, "python" can also be Python 3 as well. I recall that I saw commands using "python" rather than "python3" in the *official* Python 3 documentation: see examples below (*). Problem: On Wi

[Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform

2019-02-14 Thread Steve Dower
As part of adding ARM32 support for Windows, we need to enable cross-compilation in distutils. This is easy enough, though it requires somehow getting the target platform as well as the current platform. Right now, the change at https://github.com/python/cpython/pull/11774 adds a get_target_pl

Re: [Python-Dev] Adding test.support.safe_rmpath()

2019-02-15 Thread Steve Dower
On 14Feb.2019 0948, Brett Cannon wrote: > On Thu, Feb 14, 2019 at 7:26 AM Giampaolo Rodola' > wrote: > Extra: an argument in favor of using tempfile.mkdtemp() instead of > TESTFN is parallel testing, but I think we're not using it. > > > With -j you can do para

Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform

2019-02-15 Thread Steve Dower
On 14Feb2019 1147, Gregory P. Smith wrote: To alleviate confusion long term I'd love it if we could deprecate the unqualified get_platform() API and point people towards always being explicit about get_target_platform() vs get_current_platform(). This is an option too, though it doesn't reduce

Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform

2019-02-16 Thread Steve Dower
On 16Feb.2019 0831, Nick Coghlan wrote: > On Sat, 16 Feb 2019 at 08:06, Steve Dower wrote: >> I'm inclined to say that nobody but us uses this API :) Does that make >> it seem more okay to "clarify" that it's returning target platform? > > I've alwa

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-16 Thread Steve Dower
On 16Feb.2019 1332, Antoine Pitrou wrote: > On Sat, 16 Feb 2019 11:15:44 +0100 > Jeroen Demeyer wrote: >> On 2019-02-16 00:37, Eric Snow wrote: >>> One thing that would help simplify changes >>> in this area is if PyInterpreterState were defined in >>> Include/internal. >> >> How would that help

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-18 Thread Steve Dower
On 18Feb.2019 1324, Jeroen Demeyer wrote: > Still, do we really need so many levels of API: > (1) stable API (with #define Py_LIMITED_API) > (2) public documented API > (3) private undocumented API (the default exposed API) > (4) internal API (with #define Py_BUILD_CORE) > > I would argue to fold

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-19 Thread Steve Dower
On 19Feb2019 0229, Jeroen Demeyer wrote: On 2019-02-19 04:04, Steve Dower wrote: On 18Feb.2019 1324, Jeroen Demeyer wrote: For a very concrete example, was it really necessary to put _PyTuple_ITEMS in (4)? That's used in _functoolsmodule.c. Especially given that the very si

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-19 Thread Steve Dower
On 19Feb2019 0555, Nick Coghlan wrote: I really don't want us to ever get into a situation where we're actively encouraging third party projects to define Py_BUILD_CORE. If we decide we do want to go down a path like that, I'd instead prefer to see us define something more like "Py_FRAGILE_API"

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-19 Thread Steve Dower
On 19Feb2019 1212, Stefan Behnel wrote: So, yeah, I'm happy with the status quo, and a bit worried about all the moving around of declarations and that scent of a sword of Damocles hanging over their potential confinement. IMHO, things should just be public and potentially marked as "unstable" to

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-19 Thread Steve Dower
On 19Feb2019 1141, Barry Warsaw wrote: Steve Dower wrote on 2/16/19 14:34:> This is mostly about being able to assign blame when things break, so I'm totally okay with extension modules that want to play with internals declaring Py_BUILD_CORE to get access to them (though I suspect th

Re: [Python-Dev] new binary wheels PEP idea

2019-02-19 Thread Steve Dower
And for what it's worth, most of the really active contributors from distutils-sig seem to prefer the "Packaging" category at https://discuss.python.org/ If you'd prefer to use Discourse, I'd suggest posting there first and also email distutils-sig with a link to the discussion. Otherwise, go

Re: [Python-Dev] Question - Bug Triage for 3.4 & 3.5

2019-02-20 Thread Steve Dower
On 20Feb.2019 0711, Stephane Wirtel wrote: > After discussion with Victor, my proposal will generate noise with the > ML, maybe for nothing. > > On 02/20, Stephane Wirtel wrote: >> Hi, >> >> As you know, Python 3.4 and 3.5 are in security mode and the EOL for >> these versions are respectively 201

Re: [Python-Dev] Making PyInterpreterState an opaque type

2019-02-21 Thread Steve Dower
Just letting everyone know that I'm intending to restart this discussion over in capi-sig, as I feel like I've got an informational-PEP worth of "vision", "ideas" and "direction" and nomenclature for our C API (*not* talking about a rewrite, but the principles we should be following now... and woul

Re: [Python-Dev] Add minimal information with a new issue?

2019-02-21 Thread Steve Dower
On 21Feb2019 1117, Barry Warsaw wrote: On Feb 21, 2019, at 10:34, Raymond Hettinger wrote: I think that anything that raises the cost of filing a bug report will work to our detriment. Ideally, we want the barriers to reporting to be as low as possible. `python -m reportbug` could make th

Re: [Python-Dev] "Good first issues" on the bug tracker

2019-02-21 Thread Steve Dower
On 21Feb2019 0557, Cheryl Sabella wrote: If you find anything you think is suitable, please add a comment with 'good first issue' and nosy me or Mariatta on it.  If you're unsure, then nosy us anyway.  It would be awesome to have too many issues to choose from than not enough.  If an issue isn'

Re: [Python-Dev] "Good first issues" on the bug tracker

2019-02-21 Thread Steve Dower
On 21Feb2019 1258, Cheryl Sabella wrote: I agree completely. We normally add the "Easy" or "Easy (C)" keywords to mark these (the latter for issues that involve C code), and these are collected under the "Easy issues" link at the left hand side of the tracker. Any reason

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-26 Thread Steve Dower
On 2/26/2019 1:20 PM, Gregory P. Smith wrote: For an OS distro provided interpreter, being able to restrict its use to only OS distro provided software would be ideal (so ideal that people who haven't learned the hard distro maintenance lessons may hate me for it). Such a restriction could be

Re: [Python-Dev] [RELEASE] Python 2.7.16

2019-03-05 Thread Steve Dower
bout clicking 'more info'.  I don't know the status of python.org 3.x downloads. Since Steve Dower put 3.7 on the Windows store, PSF must now be a known publisher.  Perhaps he can help make 2.7 'known'. SmartScreen should recognize it now that it's been downloade

Re: [Python-Dev] PEPs from non-core devs now need a sponsor

2019-03-05 Thread Steve Dower
On 05Mar2019 0620, James Edwards wrote: I have to say, this is sort of surprising for what seems like the first official action of the steering committee.  Are there really /that many /PEPs that a team that is now, what, 5x the size of the BFDL model is worried that they'll be able to keep up? 

Re: [Python-Dev] PEPs from non-core devs now need a sponsor

2019-03-05 Thread Steve Dower
On 05Mar2019 1245, Chris Angelico wrote: How much effort does it take to sponsor a PEP? I'm not a core dev, but I can help someone with the work of writing and publishing. So if, in that hypothetical situation, some (very busy) core dev were willing to say "yeah, go ahead, put my name on it", wou

Re: [Python-Dev] configparser: should optionxform be idempotent?

2019-03-08 Thread Steve Dower
I just have one thing to add to the discussion, which is this: https://youtu.be/hAnCiTpxXPg?t=6339 Yes, people actually read and interpret our documentation :) So discussions like this are probably a good thing in terms of getting the best descriptions in there, but if we use a specific techn

Re: [Python-Dev] Replacement for array.array('u')?

2019-03-22 Thread Steve Dower
On 22Mar2019 0433, Antoine Pitrou wrote: The question is: why would you use a array.array() with a Windows C API? I started replying to this with a whole lot of examples, and eventually convinced myself that you wouldn't (or shouldn't). That said, I see value in having a type for array/struc

Re: [Python-Dev] Removing PendingDeprecationWarning

2019-03-22 Thread Steve Dower
On 22Mar2019 1101, Brett Cannon wrote: On Fri, Mar 22, 2019 at 10:37 AM Inada Naoki > wrote: There might be some small troubles.  But it was small enough for Python minor versions, I think.  I don't think it's worth the cost to users. We can just choose t

Re: [Python-Dev] Replacement for array.array('u')?

2019-03-25 Thread Steve Dower
On 25Mar2019 0812, Martin (gzlist) wrote: On Fri, 22 Mar 2019 at 16:12, Steve Dower wrote: On 22Mar2019 0433, Antoine Pitrou wrote: The question is: why would you use a array.array() with a Windows C API? I started replying to this with a whole lot of examples, and eventually convinced

Re: [Python-Dev] Removing PendingDeprecationWarning

2019-03-27 Thread Steve Dower
On 27Mar2019 0324, Inada Naoki wrote: On Mon, Mar 25, 2019 at 10:11 PM Inada Naoki wrote: C, Rust, Java, Ruby, PHP, don't have PendingDeprecation. Programmers only need Deprecation. Why programmers need PendingDeprecation only in Python? Any comments about this? I want to document Pending

Re: [Python-Dev] New Python Initialization API

2019-03-27 Thread Steve Dower
On 27Mar2019 1048, Victor Stinner wrote: Since November 2017, I'm refactoring the Python Initialization code to cleanup the code and prepare a new ("better") API to configure Python Initialization. I just fixed the last issues that Nick Coghlan asked me to fix (add a pre-initialization step: done

Re: [Python-Dev] New Python Initialization API

2019-03-28 Thread Steve Dower
On 28Mar2019 0703, Victor Stinner wrote: In fact, there is already a PyConfig._init_config flag (currently named _PyCoreConfig._init_main) which only initializes Python up to the "core initialization" if set to 0. This parameter is private since it's unclear to me what should be the exact scope o

[Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-28 Thread Steve Dower
om/zooba/cpython/tree/pep-578-3.7/) is already getting some real use (though this will not be added to 3.7, unless people *really* want it, so the backport is just for reference). Cheers, Steve = PEP: 578 Title: Python Runtime Audit Hooks Version: $Revision$ Last-Modified: $Date$ Author: S

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-28 Thread Steve Dower
The implementation can be viewed as a PR at https://github.com/python/cpython/pull/12613 On 28Mar2019 1535, Steve Dower wrote: Hi all Time is short, but I'm hoping to get PEP 578 (formerly PEP 551) into Python 3.8. Here's the current text for review and comment before I sub

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Steve Dower
On 29Mar2019 0334, Christian Heimes wrote: On 28/03/2019 23.35, Steve Dower wrote: Audit Hook -- In order to observe actions taken by the runtime (on behalf of the caller), an API is required to raise messages from within certain operations. These operations are typically deep within

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Steve Dower
Thanks Christian for responding - I endorse and support all your comments. (I'd hoped that by explicitly saying "this is not a sandbox" it would avoid people thinking it was a sandbox, but apparently I would have been better just to avoid the keyword completely...) On 29Mar2019 0324, Christian

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Steve Dower
n have the definition of that "everything". On 29.03.2019 1:35, Steve Dower wrote: Hi all Time is short, but I'm hoping to get PEP 578 (formerly PEP 551) into Python 3.8. Here's the current text for review and comment before I submit to the Steering Council.

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-29 Thread Steve Dower
On 29Mar2019 1218, Christian Heimes wrote: On 28/03/2019 23.35, Steve Dower wrote: The ``importlib.util.open_for_import()`` function is a drop-in replacement for ``open(str(pathlike), 'rb')``. Its default behaviour is to open a file for raw, binary access. To change the behaviour a n

[Python-Dev] Strange umask(?)/st_mode issue

2019-03-29 Thread Steve Dower
Hi I'm trying to track down the cause of the failed Linux tests on this build (it's also been happening on random PR builds for the last day or two, but this is the first I've seen it happen on already merged code): https://dev.azure.com/Python/cpython/_build/results?buildId=40189 One of the

Re: [Python-Dev] Strange umask(?)/st_mode issue

2019-03-29 Thread Steve Dower
On 29Mar.2019 1731, Nathaniel Smith wrote: > That does sound strange. Oh good, it's not just me :) > How easily can you reproduce it? That majorly > effects how I would try to debug something like this... No idea. Looking at the builds on Pipelines again, it seems that all the builds since abou

Re: [Python-Dev] New Python Initialization API

2019-03-29 Thread Steve Dower
On 29Mar.2019 1830, Victor Stinner wrote: > The purpose of the PEP 587 is to have a working document so everyone > can look at the proposed API (stay focused to the API rather than > bothering with the implementation). IMHO it's now time to get more > people looking at the Python Initialization. >

Re: [Python-Dev] Strange umask(?)/st_mode issue

2019-03-29 Thread Steve Dower
On 29Mar.2019 1939, Cameron Simpson wrote: > Can you get a branch into your pipeline? Then you could just hack the > tarfile test with something quick and dirty like: > >    pid = os.getpid() >    system("strace -p %d 2>/path/to/strace.out &" % pid) >    time.sleep(2)   # get strace heaps of time

Re: [Python-Dev] Strange umask(?)/st_mode issue

2019-03-30 Thread Steve Dower
On 29Mar.2019 1944, Steve Dower wrote: > On 29Mar.2019 1939, Cameron Simpson wrote: >> Can you get a branch into your pipeline? Then you could just hack the >> tarfile test with something quick and dirty like: >> >>    pid = os.getpid() >>    system("strac

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-30 Thread Steve Dower
On 29Mar.2019 2020, Inada Naoki wrote: > I don't like adding more Python callback from low level. > > Python runtime is very complicated already, especially __del__, > shutdown process, and multi threading. Python callback from low level > is source of very difficult bugs always. Asynchronous ca

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-03-30 Thread Steve Dower
On 30Mar.2019 0747, Nick Coghlan wrote: > I like this PEP in principle, but the specific "open_for_import" name > bothers me a lot, as it implies that "importing" is the only situation > where a file will be opened for code execution. > > If this part of the API were lower down the stack (e.g. >

Re: [Python-Dev] New Python Initialization API

2019-03-30 Thread Steve Dower
Here is my first review of https://www.python.org/dev/peps/pep-0587/ and in general I think it's very good. There are some things I'd like to consider changing before we embed them permanently in a public API, and as I'm still keen to have the ability to write Python code for configuration (to repl

Re: [Python-Dev] Strange umask(?)/st_mode issue

2019-04-01 Thread Steve Dower
On 30Mar2019 1130, Gregory P. Smith wrote: I wouldn't expect it to be the case in a CI environment but I believe a umask can be overridden if the filesystem is mounted and configured with acls set?  (oh, hah, Ivan just said the same thing) Yep, it appears this is the case. The Pipelines team g

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-01 Thread Steve Dower
On 31Mar2019 0538, Christian Heimes wrote: I don't like the fact that the PEP requires users to learn and use an additional layer to handle native code. Although we cannot provide a fully secure hook for native code, we could at least try to provide a best effort hook and document the limitations

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-01 Thread Steve Dower
On 30Mar2019 0913, Steve Dower wrote: On 30Mar.2019 0747, Nick Coghlan wrote: I like this PEP in principle, but the specific "open_for_import" name bothers me a lot, as it implies that "importing" is the only situation where a file will be opened for code execution. If

Re: [Python-Dev] Strange umask(?)/st_mode issue

2019-04-01 Thread Steve Dower
On 01Apr2019 1535, Cameron Simpson wrote: On 01Apr2019 09:12, Steve Dower wrote: On 30Mar2019 1130, Gregory P. Smith wrote: I wouldn't expect it to be the case in a CI environment but I believe a umask can be overridden if the filesystem is mounted and configured with acls set?  (oh

Re: [Python-Dev] how to rerun the job “Azure Pipelines PR”?

2019-04-02 Thread Steve Dower
On 02Apr2019 0522, Karthikeyan wrote: Closing and re-opening the PR will trigger the CI run again that might help in this case but it will run all the jobs. Yes, I believe this is still the best way to re-run Pipelines jobs. For people with logins (not yet everyone in the GitHub org, but I hea

Re: [Python-Dev] PEP-582 and multiple Python installations

2019-04-02 Thread Steve Dower
On 02Apr2019 0817, Calvin Spealman wrote: (I originally posted this to python-ideas, where I was told none of this PEP's authors subscribe so probably no one will see it there, so I'm posting it here to raise the issue where it can get seen and hopefully discussed) Correct, thanks for posting

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-09 Thread Steve Dower
On 09Apr2019 0925, Victor Stinner wrote: This change makes the debug build ABI closer to the release build ABI, but I am not sure how to compare these two ABI. Technically, C extensions still need to be recompiled. What do you think? What are the other changes that would be required? And is th

Re: [Python-Dev] New Python Initialization API

2019-04-09 Thread Steve Dower
Thanks for the replies. Anything I don't comment on means that I agree with you :) On 05Apr2019 0900, Victor Stinner wrote: Honestly, I'm not sure that we really have to distinguish "user error" and "internal error". It's an old debate about calling abort()/DebugBreak() or not. It seems like mo

Re: [Python-Dev] New Python Initialization API

2019-04-09 Thread Steve Dower
On 05Apr2019 0912, Victor Stinner wrote: About PyPreConfig and encodings. [...] * ``PyInitError Py_PreInitialize(const PyPreConfig *config)`` * ``PyInitError Py_PreInitializeFromArgs( const PyPreConfig *config, int argc, char **argv)`` * ``PyInitError Py_PreInitializeFromWideArgs( const PyPreC

Re: [Python-Dev] New Python Initialization API

2019-04-09 Thread Steve Dower
On 05Apr2019 0922, Victor Stinner wrote: While there are supporters of an "isolated Python" (sometimes called "system python"), the fact that it doesn't exist in any Linux distribution nor on any other operating system (Windows, macOS, FreeBSD), whereas it's already doable in Python 3.6 with Py_I

Re: [Python-Dev] New Python Initialization API

2019-04-09 Thread Steve Dower
On 05Apr2019 0936, Victor Stinner wrote: For the PyMainConfig structure idea, I cannot comment at this point. I need more time to think about it. About the "path configuration" fields, maybe a first step to enhance the API would be to add the the following function: PyInitError PyConfig_Comput

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-10 Thread Steve Dower
On 10Apr2019 0401, Victor Stinner wrote: Le mar. 9 avr. 2019 à 22:16, Steve Dower a écrit : What are the other changes that would be required? I don't know. And is there another way to get the same functionality without ABI modifications? Py_TRACE_REFS is a double linked list of

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-10 Thread Steve Dower
On 10Apr2019 1109, Steve Dower wrote: On 10Apr2019 0401, Victor Stinner wrote: I think it's worthwhile if we can really get to debug and non-debug builds being ABI compatible. Getting partway there in this case doesn't seem to offer any benefits. Disabling Py_TRACE_REFS by defaul

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-10 Thread Steve Dower
On 10Apr2019 1227, Nathaniel Smith wrote: On Wed, Apr 10, 2019, 04:04 Victor Stinner > wrote: I don't think that I ever used sys.getobjects(), whereas many projects use gc.get_objects() which is also available in release builds (not only in debug builds).

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-11 Thread Steve Dower
On 11Apr2019 0228, Victor Stinner wrote: Le jeu. 11 avr. 2019 à 07:49, Serhiy Storchaka a écrit : 10.04.19 14:01, Victor Stinner пише: Disabling Py_TRACE_REFS by default in debug mode reduces the Python memory footprint. Py_TRACE_REFS costs 2 pointers per PyObject: 16 bytes on 64-bit platforms

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-11 Thread Steve Dower
On 10Apr2019 1917, Nathaniel Smith wrote: It sounds like --with-pydebug has accumulated a big grab bag of unrelated features, mostly stuff that was useful at some point for some CPython dev trying to debug CPython itself? It's clearly not designed with end users as the primary audience, given tha

Re: [Python-Dev] Removing PID check from signal handler

2019-04-12 Thread Steve Dower
On 12Apr.2019 0919, Jeroen Demeyer wrote: > The signal handler (that receives signals from the OS) in Python starts > with a check > >     if (getpid() == main_pid) > > Looking at the comments, the intent was to do a check for the main > *thread* but this is checking the *process* id. So this con

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-12 Thread Steve Dower
On 12Apr.2019 1643, Nathaniel Smith wrote: > On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: >> >> On 10Apr2019 1917, Nathaniel Smith wrote: > I don't know how many people use Py_TRACE_REFS, but if we can't find > anyone on python-dev who uses it then it mus

Re: [Python-Dev] No longer enable Py_TRACE_REFS by default in debug build

2019-04-15 Thread Steve Dower
On 12Apr2019 1819, Nathaniel Smith wrote: On Fri, Apr 12, 2019 at 5:05 PM Steve Dower wrote: On 12Apr.2019 1643, Nathaniel Smith wrote: On Thu, Apr 11, 2019 at 8:26 AM Steve Dower wrote: The very first question I asked was whether this would let us converge the ABIs, and the answer was &qu

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-15 Thread Steve Dower
On 15Apr2019 1344, Christian Heimes wrote: Hi Steve, (memory dump before I go to bed) Steve Grubb from Red Hat security pointed me to some interesting things [1]. For instance there is some work on a new O_MAYEXEC flag for open(). Steve came to similar conclusions like we, e.g. streaming code f

Re: [Python-Dev] PEP 578: Python Runtime Audit Hooks

2019-04-16 Thread Steve Dower
(and exclusive) handle/descriptor", so this feels like YAGNI. On 01/04/2019 18.31, Steve Dower wrote: In each case there should be associated audit events for tracking the intent (and interrupting at that point if it doesn't like the intended action), but for the simple case of &quo

Re: [Python-Dev] Concurrent.futures: no type discovery for PyCharm

2019-04-22 Thread Steve Dower
On 22Apr2019 1521, Andrew Svetlov wrote: I see the chicken and egg problem here. If we are talking about typing module usage -- typeshed is the type hints provider. If PyCharm doesn't want to use it -- it is not CPython problem. I think there is no need to change python code itself but used tool

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-22 Thread Steve Dower
On 22Apr2019 1822, Glenn Linderman wrote: Inada is now proposing a way to allow the coder to suggest a group of dictionaries that might benefit from the same gains, by preclassifying non-__dict__ slot dictionaries to do similar sharing. CSV reader is an exemplary candidate, because it creates

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-22 Thread Steve Dower
On 22Apr2019 1921, Steve Dower wrote: On 22Apr2019 1822, Glenn Linderman wrote: Inada is now proposing a way to allow the coder to suggest a group of dictionaries that might benefit from the same gains, by preclassifying non-__dict__ slot dictionaries to do similar sharing. CSV reader is an

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-22 Thread Steve Dower
On 22Apr2019 2143, Inada Naoki wrote: On Tue, Apr 23, 2019 at 11:30 AM Steve Dower wrote: Or possibly just "dict(existing_dict).update(new_items)". Do you mean .update accepts values tuple? I can't think it's Not sure what you were going to go on to say here, but why

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-22 Thread Steve Dower
On 22Apr2019 2119, Glenn Linderman wrote: While Inada's suggested DictBuilder interface was immediately obvious, I don't get how either copy or update would achieve the goal. Perhaps you could explain? Particularly, what would be the trigger that would make dict() choose to create a shared key

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-23 Thread Steve Dower
On 23Apr2019 0034, Glenn Linderman wrote: On 4/22/2019 10:59 PM, Steve Dower wrote: On 22Apr2019 2119, Glenn Linderman wrote: While Inada's suggested DictBuilder interface was immediately obvious, I don't get how either copy or update would achieve the goal. Perhaps you cou

Re: [Python-Dev] Proposal: dict.with_values(iterable)

2019-04-23 Thread Steve Dower
On 23Apr2019 0008, Inada Naoki wrote: On Tue, Apr 23, 2019 at 2:54 PM Steve Dower wrote: On 22Apr2019 2143, Inada Naoki wrote: On Tue, Apr 23, 2019 at 11:30 AM Steve Dower wrote: Or possibly just "dict(existing_dict).update(new_items)". Do you mean .update accepts values tupl

Re: [Python-Dev] Location of CI for Windows Embeddable Distro

2019-05-01 Thread Steve Dower
On 01May2019 1109, Gerald Wiltse wrote: I looked through all the windows builders here and could not find any which generate this package, can anyone tell me where the work happens? I also tried searching looking for scripts and CI files which create the package in all the relevant repos and c

Re: [Python-Dev] Location of CI for Windows Embeddable Distro

2019-05-01 Thread Steve Dower
On 01May2019 1305, Gerald Wiltse wrote: That does help, thanks so much for the fast response!  I now see how it's executed on Azure pipelines also, so that should be enough for me to reverse engineer the rest. Followup questions if you have time: 1. It seems that directory doesn't exist in the

Re: [Python-Dev] RFC: PEP 587 "Python Initialization Configuration": 2nd version

2019-05-13 Thread Steve Dower
On 10May2019 1832, Victor Stinner wrote: Hi, First of all, I just found an old issue that we will solved by my PEP 587 :-) Add Py_SetFatalErrorAbortFunc: Allow embedding program to handle fatal errors https://bugs.python.org/issue30560 Yes, this should be a feature of any redesigned embedding

Re: [Python-Dev] RFC: PEP 587 "Python Initialization Configuration": 2nd version

2019-05-13 Thread Steve Dower
it turns out that other people had different opinions. Cheers, Steve On 13May2019 1452, Victor Stinner wrote: )Le lun. 13 mai 2019 à 18:28, Steve Dower a écrit : My take: * all the examples are trying to be isolated from the system Python install (except Vim?) "Isolation" means di

Re: [Python-Dev] RFC: PEP 587 "Python Initialization Configuration": 2nd version

2019-05-14 Thread Steve Dower
On 10May2019 1832, Victor Stinner wrote: I studied code of applications embedding Python. Most of them has to decode bytes strings to get wchar_t* to set home, argv, program name, etc. I'm not sure that they use the "correct" encoding, especially since Python 3.7 got UTF-8 Mode (PEP 540) and C lo

Re: [Python-Dev] PEP 581 (Using GitHub issues for CPython) is accepted

2019-05-15 Thread Steve Dower
On 15May2019 0240, Paul Moore wrote: On Wed, 15 May 2019 at 09:51, Antoine Pitrou wrote: On Tue, 14 May 2019 18:11:14 -0700 Barry Warsaw wrote: As the BDFL-Delegate for PEP 581, and with the unanimous backing of the rest of the Steering Council, I hereby Accept this PEP. For future refer

Re: [Python-Dev] deprecation of abstractstaticmethod and abstractclassmethod

2019-05-15 Thread Steve Dower
On 15May2019 1248, Nathaniel Smith wrote: I don't care about the deprecation either way. But can we fix the individual decorators so both orders work? Even if it requires a special case in the code, it seems worthwhile to remove a subtle user-visible footgun. The classmethod and staticmethod

Re: [Python-Dev] RFC: PEP 587 "Python Initialization Configuration": 3rd version

2019-05-16 Thread Steve Dower
Thanks for adding your input, Gregory! It's much appreciated. I'll shuffle your comments around a bit, as I'd rather address the themes than each individual point. On 15May2019 2134, Gregory Szorc wrote: PyPreConfig_INIT and PyConfig_INIT as macros that return a struct feel weird to me. Speci

Re: [Python-Dev] RFC: PEP 587 "Python Initialization Configuration": 3rd version

2019-05-16 Thread Steve Dower
On 15May2019 1610, Victor Stinner wrote: Thanks to the constructive discussions, I enhanced my PEP 587. I don't plan any further change, the PEP is now ready to review (and maybe even for pronouncement, hi Thomas! :-)). My view is that while this is a fantastic PEP and the groundwork that alre

Re: [Python-Dev] RFC: PEP 587 "Python Initialization Configuration": 3rd version

2019-05-16 Thread Steve Dower
On 16May2019 0710, Thomas Wouters wrote: A couple of things are documented as performing pre-initialisation (PyConfig_SetBytesString, PyConfig_SetBytesArgv). I understand why, but I feel like that might be confusing and error-prone. Would it not be better to have them fail if pre-initialisation

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Steve Dower
On 16May2019 0647, Victor Stinner wrote: Le jeu. 16 mai 2019 à 12:57, Serhiy Storchaka a écrit : For unraisable hook, it's not hard to imagine other useful parameters that can be passed to the hook to provide more context about the exception. For example, right now we only pass one object. But

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Steve Dower
On 16May2019 0856, Steve Dower wrote: On 16May2019 0647, Victor Stinner wrote: Le jeu. 16 mai 2019 à 12:57, Serhiy Storchaka a écrit : For unraisable hook, it's not hard to imagine other useful parameters that can be passed to the hook to provide more context about the exception. For ex

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Steve Dower
On 16May2019 0902, Steve Dower wrote: Actually, if the default implementation prints the exception message, how is this different from sys.excepthook? Specifically, from the point of customizing the hooks. If I were going to replace unraisablehook to do something on specific exceptions, I&#

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Steve Dower
On 16May2019 1404, Victor Stinner wrote: Le jeu. 16 mai 2019 à 17:56, Steve Dower a écrit : I really like this API, and I agree with Victor that we don't really need more than the exception info. For future expansion, we can pass in a different exception, no? Sorry, I don't und

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Steve Dower
On 16May2019 1441, Victor Stinner wrote: Le jeu. 16 mai 2019 à 23:17, Steve Dower a écrit : You go on to say "pass an error message" and "keep repr(obj) if you want", but how is this different from creating an exception that contains the custom message, the repr of the ob

Re: [Python-Dev] Parser module in the stdlib

2019-05-16 Thread Steve Dower
On 16May2019 1548, Pablo Galindo Salgado wrote: > Will the folks using forks be happy to switch to the stdlib version? For example I can imagine that if black wants to process 3.7 input code while running on 3.6, it might prefer a parser on PyPI even if he stdlib version were public, since the

[Python-Dev] Python in next Windows 10 update

2019-05-21 Thread Steve Dower
Hi all Just sharing this here because I think it's important for us to be aware of it - I'm not trying to promote or sell anything here :) (Those who were at the language summit have seen this already.) In the next Windows 10 update that starts rolling out today, we (Microsoft) have added "p

Re: [Python-Dev] Python in next Windows 10 update

2019-05-21 Thread Steve Dower
On 21May2019 1621, MRAB wrote: Does it behave nicely with py.exe? This is still something of an open issue with the Store package for Python - py.exe doesn't look for the registry keys in a way that will find them (due to some very obscure compatibility quirks). The Store package does not i

Re: [Python-Dev] Python in next Windows 10 update

2019-05-22 Thread Steve Dower
On 22May2019 1237, Oscar Benjamin wrote: On Tue, 21 May 2019 at 21:32, Steve Dower wrote: In the next Windows 10 update that starts rolling out today, we (Microsoft) have added "python.exe" and "python3.exe" commands that are installed on PATH *by default* and will open th

Re: [Python-Dev] Python in next Windows 10 update

2019-05-22 Thread Steve Dower
On 22May2019 1309, Ivan Pozdeev via Python-Dev wrote: As someone whose job is to diagnose and fix problems with running software: Are there patches in your release? Do you provide corresponding sources and debug symbols for it? You can find the sources at https://github.com/python/cpython :)

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Steve Dower
On 23May2019 0542, Inada Naoki wrote: 1. perf shows 95% of CPU time is eaten by _PyObject_Free, not kernel space. 2. This loop is cleary hot: https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L1816-L1819 I can attach the process by gdb and I confi

Re: [Python-Dev] PEP 558: Defined semantics for locals()

2019-05-23 Thread Steve Dower
On 23May2019 0636, Nick Coghlan wrote: However, I think the PR does show that the proposed technique can be implemented without too much additional code complexity, and will hopefully be adaptable for all implementations that emulate the frame API at all. Much excitement! One of the things I l

Re: [Python-Dev] PEP 594: update 1

2019-05-23 Thread Steve Dower
On 23May2019 0947, Anders Munch wrote: Fra: Paul Moore [mailto:p.f.mo...@gmail.com]: A major version change serves as a heads up that something is going on and you need to check the consequences before upgrading. Python's backward compatibility policy allows breaking changes between versions

Re: [Python-Dev] Python in next Windows 10 update

2019-05-24 Thread Steve Dower
On 24May2019 0220, Baptiste Carvello wrote: Hello, Le 21/05/2019 à 22:30, Steve Dower a écrit : [...] * the Python 3.7 installed from the store will not auto-update to 3.8, but when 3.8 is released we (Microsoft) will update the redirect to point at it * if you pass arguments to the redirect

<    1   2   3   4   5   6   7   8   >