[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks
> On 13 May 2021, at 02:09, Mike Miller wrote: > > >> On 2021-05-11 16:12, Guido van Rossum wrote: >> On Tue, May 11, 2021 at 4:07 PM Gregory P. Smith > There's a difference between tracebacks dumped as plain text (utf-8) by >>traceback.print_exc() appearing on stderr or directed into log files and >>what can be displayed within a terminal. It is highly unusual to emit >>terminal control characters into log files. >> And yet it happens all the time. :-( Let's not risk that happening. > > > os.isatty() is helpful in that situation, Most tools that support colour output allow you to customise the colours and have a always-colour, never-colour, auto-colour option. Isatty() is useful for the auto. Barry > > -Mike > > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/QT2CTAPV7BVUHEUR6OY6DKWTNX6WM5MF/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/I5A6JSFVERDC6YG6CNHSJQTY4CDUTSY6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11
> On 24 Feb 2022, at 11:45, Victor Stinner wrote: > > Ok, let me try something simpler: > > "Python 3.11 and newer versions use C11 without optional features. The > public C API should be compatible with C++." > https://github.com/python/peps/pull/2309/files Should is often read as meaning optional when writing specs. Can you say “must be compatible with C++”. Barry > > Victor > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/C32XCK5WHMLJNV6RGLMN2XNBDWR4DI3V/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AKW6Y5TIHWIZKCSQD4I6GD4Q7GZQYUZ7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Need Help
> On 25 Feb 2022, at 21:47, Prasad, PCRaghavendra > wrote: > > > Hi Scott, Scott is my family name. > > Thanks for the reply > > Are you asking how to link python to an external libexpat instead of the > vendor expat inside python? > > >> yes, we have done for some of the external libs like OpenSSL, bzip2 but > >> libexpat was an internal module to python so how to link to the latest > >> expat lib/code without changing the python version was our doubt. > > Have you tried deleting libexpat 2.2.8 from the python source code and > replacing it with the libexpat 2.4.6 and then > compiling python? > > >> No, do you mean hear removing the files ( python\Modules\expat ) folder > >> and replacing the new files from libexpat 2.4.6. we didn’t do that > We didn’t know whether that is the right way of doing it and if there are any > incompatibilities to the python version (3.9.5) > > Are you concerned that you need fixes in the python code to support the 2.4 > version? > > >> Yes our application is running with python 3.9.5 and it internal contains > >> libexpat 2.2.8 that has security vulnerabilities > One way is to upgrade the python to the latest version where the libexpat > issues are fixed ( maybe 3.9.11). > > What is the best approach so that there will be no major issues. If I was doing this I would replace the libexpat code inside the python tree then compile python and see if that works without error. Take that python version and run the python test suite against it. If that passes then I would run my application’s test suite to ensure no regressions. Barry > > Thanks, > Raghu > > > > Internal Use - Confidential > From: Barry Scott > Sent: Saturday, February 26, 2022 3:08 AM > To: Prasad, PCRaghavendra > Cc: Python-Dev@python.org > Subject: Re: [Python-Dev] Need Help > > [EXTERNAL EMAIL] > > > > > On 25 Feb 2022, at 12:58, Prasad, PCRaghavendra > wrote: > > Hi All, > > we are using the python 3.9.5 version in our application. > > In 3.9.5 it is using libexpat 2.2.8 version, as part of the Black duck scan, > it is showing critical vulnerabilities in libexpat 2.2.8. > > (CVE-2022-22824 > CVE-2022-23990 > CVE-2022-23852 > CVE-2022-25236 > CVE-2022-22823) > > when there are any issues ( security issues ) in external modules like > OpenSSL, bzip2, and zlib we were able to get the latest code and build as it > is straightforward, but libexpat is an internal module to the python and we > don't see how we can upgrade libexpat alone in python 3.9.5 > > So is there a way we can build python (ex 3.9.5) which is already carrying > libexpat 2.2.8 so that it will link to the latest libexpat version (2.4.6 - > fixed security issues). > > Another solution when we searched over the net and from the mails what we > came to know is we need to wait for Python 3.9.11 where this will be linked > to libexpat 2.4.6. > > Any inputs on this will be helpful. > > Are you asking how to link python to an external libexpat instead of the > vendored expat inside python? > > Have you tried deleting libexpat 2.2.8 from the python source code and > replacing with the libexpat 2.4.6 and then > compiling python? > > Are you concerned that you need fixes in the python code to support the 2.4 > version? > > Barry > > > > > Thanks, > Raghu > > Internal Use - Confidential > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > [mail.python.org] > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/2JHZTKQVVYR67KQRIFF5XEMXDY3FZLMN/ > [mail.python.org] > Code of Conduct: http://python.org/psf/codeofconduct/ [python.org] > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LR2IF3PBVSMW4U5WLOOEV55RR47IM5WL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Add -P command line option to not add sys.path[0]
> On 27 Apr 2022, at 17:22, Victor Stinner wrote: > > Ok, you changed my mind and I added PYTHONDONTADDPATH0=1 env var. Example: Maybe the env var say what it is not adding rather then where it adds it. PYTHONDONTADDPWD=1 Barry ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RT5M26HLDMN4QD7XOYEUO5HTUYIF55ZA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: EPOLLEXCLUSIVE and selectors
> On 3 Jul 2022, at 00:42, David Gilman wrote: > > Hello, I'd like to bring up https://bugs.python.org/issue44951 / > https://github.com/python/cpython/pull/27819 to the mailing list's > consideration as it has idled for a bit. I would appreciate some > authoritative feedback on which API design choice is best. I'll also > recap the PR quickly: > > Motivation: > - There is community demand for EPOLLEXCLUSIVE in Python (see blog > posts in BPO issue) > - You can't do this with the existing stdlib code as > _BaseSelectorImpl.register() raises ValueError on non-READ/WRITE > constants. (This is why _PollLikeSelector.register() has two > variables, events and poller_events) > - It's not an invasive change. The Python API doesn't change much or > at all and the kernel's EPOLLEXCLUSIVE behavior was carefully designed > to be backwards compatible with using exclusive and non-exclusive > watches on the same file descriptor. > > I've got two approaches to this. The first extends the EpollSelector > class with a property to toggle setting the EPOLLEXCLUSIVE on newly > registered file descriptors: > > https://github.com/dgilman/cpython/commit/43174df5bd7a78eedf0692ebbe63a9b943248a74 > > The second introduces an entirely new sibling class, > EpollExclusiveSelector, that unconditionally sets it on registration: > > https://github.com/dgilman/cpython/commit/554a5bf9c16b6bd82ce47b2d0dd1833f2bdd31cb > > The first one was my first attempt but I am leaning towards the > second. It doesn't require any new API surface area. It also gets > integrated into the DefaultSelector logic, and even if that shouldn't > happen it's still easy to swap out your existing selector class for > the EpollExclusiveSelector class. > > Enabling EPOLLEXCLUSIVE by default: > >> From the research I did last year my understanding is that > EPOLLEXCLUSIVE is never a performance drawback on Linux, and in the > case of a server that gets fast traffic it's a dramatic improvement. > However, I have not done my own benchmarking (with say, gunicorn, > which uses the stdlib's selectors module). > > Note that EPOLLEXCLUSIVE does have one kernel API break: you can no > longer use EPOLL_CTL_MOD on an exclusive file descriptor. Python uses > the _MOD flag under the hood to implement epoll.modify(), which > results in EpollSelector.modify() throwing an OSError if you try to > modify an exclusive file descriptor. > > In the second PR I implemented a EpollExclusiveSelector.modify() which > unregisters and reregisters the file descriptor to get around the _MOD > behavior. This means no surprise crash when someone updates Python. > But there may be other subtle regressions here: the performance of > modify() is likely going to regress, and someone could have a > dependency on Python actually using _MOD. > > But as a rebuttal to those, I have a feeling that nobody really > depends on the performance of modify(), and even in the case where > someone does use it a lot it's likely for the data= path which is > completely unchanged here. I also am struggling to think of a place > where someone would care about the kernel-level changes between _MOD > and _ADD/_DEL but that might be my own lack of imagination or > knowledge of epoll techniques. > > Maybe a compromise is to ship EpollExclusveSelector for a release > without it being the default and bump it to the default after seeing > if anyone's turned up any incompatibilities.s This would benefit frameworks like twisted. I have seen the thundering herd because of this problem In my work. Barry > > -- > David Gilman > :DG< > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VB2BBUUJENMWPGXGFMON7UQW27ZOBIB7/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RLVYNSTSSH465JPWAVHEXJACDEE2W4G5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: RFC: expose ssl internals for use by ctypes/cffi
> On 30 Nov 2022, at 16:59, chris...@weinigel.se wrote: > > Hi all, > > for quite some time I've been working on a Python implementation of a > protocol called NTS which requires access to an API in OpenSSL which is not > provided by the Python ssl module. I added a patch for that which > unfortunately for me the maintainer did not want to accept. Some comments > were made of a possible future way to give more generic access to the openssl > library via ctypes/cffi but I have been unable to find more information about > that. I was home sick last week and decided to take a shot at it and have > built something that I feel is a bit ugly but does seem to work. I'd like to > some feedback on this approach. We use the pyOpenSSL to access APIs of openssl. No need to use ctypes. Barry > > My patches can be found on github, based on the Python 3.11 tag: > > https://github.com/python/cpython/compare/3.11...wingel:cpython:main > > Here's a short description of each patch on this branch: > > "bpo-37952: SSL: add support for export_keying_material" is my old patch > which adds the method I need to the ssl library just for reference. > > The other commits add the necessary infrastructure with some example code. > These commits are not ready for submission but hopefully they show what I > have in mind. > > "Add CRYPTO_DLL_PATH and SSL_DLL_PATH to the _ssl module. " > > This commit adds two constants to the "_ssl" C module with the paths to > libcrypto and libssl respectively. On Linux dladdr and on Windows > GetModuleHandle/GetModuleFilename are used on a symbol in each library to > find the path to the corresponding DLL. I've verified that this works Debian > Bulleye and on Windows 10 with Visual Studio 2017. I don't own a Mac so I > haven't been able to test this on macOS, but I believe dladdr is available on > modern macOS so it might work out of the box. With the paths it's possible > to use ctypes or cffi get a handle to these libraries. > > "Add API to get the address of the SSL structure" then adds an API to an > SSLSocket which returns the address of the corresponding "SSL" C structure. > This address can be used by ctypes/cffi. One would probably want to expose > SSL_CTX, SSL_SESSION and BIO too but I started with just SSL since that's > what my code needs right now. > > "Add a small test program" is a small test program that uses the > infrastructure from the two above commits to call C functions in > libssl/libcrypto using both ctypes and cffi. It's a bit ugly but hopefully > it's not too hard to understand. > > "Example of how to extend the ssl library using ctypes" is an example of how > a Python module that extends the SSL library using ctypes could look. First > get a handle to libssl using ctypes, set up ctypes with the correct API for > the export_keying_material function, wrap it in a more Pythonic function and > then extend SSLSocket with the new function. A simplified version looks like > this: > >import ssl, ctypes >ssl_lib = ctypes.CDLL(ssl._ssl.SSL_DLL_PATH) >ssl_lib.SSL_export_keying_material.argtypes = ( >ctypes.c_void_p, # SSL pointer >ctypes.c_void_p, ctypes.c_size_t, # out pointer, out length >ctypes.c_void_p, ctypes.c_size_t, # label buffer, label length >ctypes.c_void_p, ctypes.c_size_t, # context, context length >ctypes.c_int) # use context flag >ssl_lib.SSL_export_keying_material.restype = ctypes.c_int > >def SSL_export_keying_material(self, label, key_len, context = None): >c_key = ctypes.create_string_buffer(key_len) >c_label = ctypes.create_string_buffer(len(label)) >c_context = ctypes.create_string_buffer(context, len(context)) >if ssl_lib.SSL_export_keying_material( > self._sslobj.get_internal_addr(), >c_key, key_len, >c_label, len(label), >c_context, len(context), 1); >return bytes(c_key) > >ssl.SSLSocket.export_keying_material = SSL_export_keying_material > > There's a final commit "Expose more OPENSSL_ variables" which add some more > constants to the ssl module which expose the cflags and build information > from OpenSSL. This patch is not really necessary, but it might be a good > idea to compare these constants with the corresponding constants retrieved > using ctypes/cffi to make sure that exactly the same version of the openssl > library is used. > > Does this seem like a good idea? As I said, I feel that
[Python-Dev] Re: Some pattern annoyance
> On 2 Aug 2023, at 12:03, Christian Tismer-Sperling > wrote: > > Hi folks, > > I just used Structural Pattern Matching quite intensively and I'm > pretty amazed of the new possibilities. > > But see this code, trying to implement Mark Pilgrim's regex > algorithm for roman literals with SPM: > > With constants, I can write > >match seq: >case "M", "M", "M", "M", *r: >return 4 * 1000, r > > But if I want to use abbreviations by assignment, this is no longer > possible, and I have to write something weird like: > >M = "M" >match seq: >case a, b, c, d, *r if M == a == b == c == d: >return 4 * 1000, r > > So what is missing seems to be a notion of const-ness, which > could be dynamically deduced. Am I missing something? Try asking for help at https://discuss.python.org/ This list is not for help or ideas, also its basically dead. Barry > > -- > Christian Tismer-Sperling:^) tis...@stackless.com > Software Consulting : http://www.stackless.com/ > Strandstraße 37 : https://github.com/PySide > 24217 Schönberg : GPG key -> 0xFB7BEE0E > phone +49 173 24 18 776 fax +49 (30) 700143-0023 > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/5MKBWCSVYZKR3S7OVY6KBF6FE7WYB5LC/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IN472AS2RMBFSLNUKCSOGWKTD3EN2ZX4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Map errno==ETIME to TimeoutError
> On 24 May 2020, at 17:42, Eric V. Smith wrote: > > > Does anyone have an opinion on https://bugs.python.org/issue39673? It maps > ETIME to TimeoutError, in addition to the already existing ETIMEDOUT. > > http://man7.org/linux/man-pages/man3/errno.3.html says: > >ETIME Timer expired (POSIX.1 (XSI STREAMS option)). > >(POSIX.1 says "STREAM ioctl(2) timeout".) > >ETIMEDOUT Connection timed out (POSIX.1-2001). > > It seems like a reasonable change to me, but I'm not a subject matter expert > on STREAMS, or what other affect this might have. > > And if added to 3.10, should it be backported? I'd tend to say "no", because > of unknown impacts on existing code. > What kernel calls return this errno? What is the current behaviour of thoses kernel calls? Barry > Eric > > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/C7KK6VSGPQKPA5IUCZ2MHH7QNLP2Q5QX/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2HXW2ZE54CFDR6VEE7J7HPLDUNHEZCT4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 632: Deprecate distutils module
> On 7 Sep 2020, at 14:28, Stefan Krah wrote: > > On Mon, Sep 07, 2020 at 11:43:41AM +0100, Steve Dower wrote: >> Rest assured, I am very aware of air-gapped and limited network systems, as >> well as corporate policies and processes. Having distutils in the standard >> library does not help with any of these. > > Of course it helps. You can develop extensions without any third party > packages or install them. But you do not need distutils to write python extensions right? Barry > > Same situation if you are on mobile Internet or in a cafe without Internet > and you want to try something out. > > > Or if you moved and you don't have cable Internet yet. Or if you are in > a country where there is no cable Internet. > > > Air-gapped systems were just an illustration of the problem. I did not > anticipate that people would take it as the centerpiece of my arguments. > > > Stefan Krah > > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/FNRIPAE4SN33STYR2OUHOKFKPUPOSJMU/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/J2MVXV3DHJ35OFDIH5P7YMTDMVKWXCND/ Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-Dev] No beta2 tonight
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Jul 18, 2008, at 01:27 AM, Martin v. Löwis wrote: >> The Windows buildbots are not very happy, though. test_ssl and >> test_bsddb and constantly failing on both the trunk and py3k. I don't >> know much about either of these items (or Windows for that matter), so >> any help would be greatly appreciated. > >bsddb is in a very bad shape, as the 2.6 code hasn't been merged into >3k. I somewhat doubt that this gets resolved before the release, so >bsddb users might need to skip 3.0. We have no showstoppers and several green buildbots, so I'm going to make the releases tonight. Please, NO CHECKINS until I say so, or ping me on #python-dev. As for bsddb, we'll make a determination after beta3. If it's terminally busted for Python 3.0, so be it. Thanks everyone for working so hard at getting beta2 ready. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIf/xT2YZpQepbvXERAtmoAKCyUkejAFxFG10Bsn/CJjZfGy0B9QCeMO0z momJmXRLCdmxs84j2hXGrTY= =Y3wS -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7 patch levels turning two digit
On Jun 23, 2014, at 04:20 PM, Donald Stufft wrote: >At the risk of getting Guido to post his slide again, I still think the >solution to the old compiler is to just roll a 2.8 with minimal changes. No. It's not going to happen, for all the reasons discussed previously. Python 2.8 is not a solution to anything. If a new, incompatible compiler suite is required, why can't there just be multiple Windows downloads on https://www.python.org/download/releases/2.7.7/ ? Well, on reason is that you'd have to convince MvL or someone else to take over the work that would require, but that's gotta be *much* lighter weight than releasing a Python 2.8. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7 patch levels turning two digit
On Jun 23, 2014, at 05:15 PM, Donald Stufft wrote: >Normally when I see someone suggest that switching compilers >in 2.7.x is likely to be less work than releasing a 2.8 It normally >appears to me they haven’t looked at the impact on the packaging >tooling. Just to be clear, releasing a Python 2.8 has enormous impact outside of just the amount of work to do it. It's an exceedingly bad idea. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7 patch levels turning two digit
On Jun 23, 2014, at 05:28 PM, Donald Stufft wrote: >Can you clarify? What support guarantees will we make about Python 2.8? Will it be supported as long as Python 2.7? Longer? Will we now have two long-term support versions or change *years* of expectations that users should transition off of Python 2.7 onto Python 2.8? Will all the LTS policies for 2.7 (e.g. PEP 466) be retired for 2.7 and/or adopted completely into 2.8? What should Linux distros do? Should they support both 2.7 and 2.8 or begin the long and potentially arduous process of certifying and transitioning to 2.8? What about other operating systems and package managers, including commercial redistributors? Who is going to do the work to make sure patch are forward ported from 2.7 to 2.8? Who is going to be the 2.8 release manager? Will they be strong enough to reject any and all new features that wouldn't have already made it into 2.7 (due to the already approved, narrow exemptions)? Or will we open the flood gates to Just One More Little New Feature To Make It Easier To Port to Python 3? How will we manage the PR surrounding our backtracking on Python 2.8? How will we manage expectations that it's only released to support a new Windows compiler? Should non-Windows users just ignore it (much like the Python 1.6 releases were mostly ignored)? How do you know which tools, workflows, and processes that will break with a Python 2.8 release? What assumptions about 2.7 being EOL for Python 2 are baked into the ecosystems outside of core Python? I could probably go on, but I'm exhausted. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sum(...) limitation
On Aug 10, 2014, at 05:24 PM, Stephen J. Turnbull wrote:
>Actually ... if I were a fan of the "".join() idiom, I'd seriously
>propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
>deprecate "".join(string_iterable) in favor of "".sum(string_iterable)
>(with the same efficient semantics).
Ever since ''.join was added, there has been vague talk about adding a join()
built-in. If the semantics and argument syntax can be worked out, I'd still
be in favor of that. Probably deserves a PEP and a millithread community
bikeshed paintdown.
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] class Foo(object) vs class Foo: should be clearly explained in python 2 and 3 doc
On Aug 10, 2014, at 11:51 AM, Alexander Belopolsky wrote: >This is not true. An alternative is to put > >__metaclass__ = type > >at the top of your module to make all classes in your module new-style in >python2. I like this much better, and it's what I do in my own bilingual code. It makes it much easier to remove the unnecessary cruft when you drop the Python 2 support. -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 4000 to explicitly declare we won't be doing a Py3k style compatibility break again?
On Aug 16, 2014, at 07:43 PM, Guido van Rossum wrote: >(Don't understand this to mean that we should never deprecate things. >Deprecations will happen, they are necessary for the evolution of any >programming language. But they won't ever hurt in the way that Python 3 >hurt.) It would be useful to explore what causes the most pain in the 2->3 transition? IMHO, it's not the deprecations or changes such as print -> print(). It's the bytes/str split - a fundamental change to core and common data types. The question then is whether you foresee any similar looming pervasive change? [*] -Barry [*] I was going to add a joke about mandatory static type checking, but sometimes jokes are blown up into apocalyptic prophesy around here. ;) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 467: Minor API improvements for bytes & bytearray
I think the biggest API "problem" is that default iteration returns integers instead of bytes. That's a real pain. I'm not sure .iterbytes() is the best name for spelling iteration over bytes instead of integers though. Given that we can't change __iter__(), I personally would perhaps prefer a simple .bytes property over which if you iterated you would receive bytes, e.g. >>> data = bytes([1, 2, 3]) >>> for i in data: ... print(i) ... 1 2 3 >>> for b in data.bytes: ... print(b) ... b'\x01' b'\x02' b'\x03' There are no backward compatibility issues with this of course. As for the single-int-ctor forms, they're inconvenient and arguably "wrong", but I think we can live with it. OTOH, I don't see any harm by adding the .zeros() alternative constructor. I'd probably want to spell the .byte() alternative constructor .from_int() but I also don't think the status quo (or .byte()) is that much of a usability problem. The API churn problem comes about when you start wanting to deprecate the single-int-ctor form. *If* that part gets adopted, it should have a really long deprecation cycle, IMO. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 467: Minor API improvements for bytes & bytearray
On Aug 18, 2014, at 08:48 AM, Nick Coghlan wrote: >Calling it bytes is too confusing: > >for x in bytes(data): > ... > >for x in bytes(data).bytes() > >When referring to bytes, which bytes do you mean, the builtin or the method? > >iterbytes() isn't especially attractive as a method name, but it's far more >explicit about its purpose. I don't know. How often do you really instantiate the bytes object there in the for loop? -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 467: Minor API improvements for bytes & bytearray
On Aug 18, 2014, at 09:12 AM, Nick Coghlan wrote: >I'm talking more generally - do you *really* want to be explaining that >"bytes" behaves like a tuple of integers, while "bytes.bytes" behaves like >a tuple of bytes? I would explain it differently though, using concrete examples. data = bytes(...) for i in data: # iterate over data as integers for i in data.bytes: # iterate over data as bytes But whatever. I just wish there was something better than iterbytes. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 467: Minor API improvements for bytes & bytearray
On Aug 18, 2014, at 10:08 AM, Nick Coghlan wrote: >There's actually another aspect to your idea, independent of the naming: >exposing a view rather than just an iterator. I'm going to have to look at >the implications for memoryview, but it may be a good way to go (and would >align with the iterator -> view changes in dict). Yep! Maybe that will inspire a better spelling. :) Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 467: Minor API improvements for bytes & bytearray
On Aug 17, 2014, at 09:39 PM, Antoine Pitrou wrote: >> need for a special case for a single byte. We already have a perfectly >> good spelling: >> NUL = bytes([0]) > >That is actually a very cumbersome spelling. Why should I first create a >one-element list in order to create a one-byte bytes object? I feel the same way every time I have to write `set(['foo'])`. -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 4000 to explicitly declare we won't be doing a Py3k style compatibility break again?
On Aug 17, 2014, at 06:02 PM, Guido van Rossum wrote: >I'm unsure about what's the single biggest pain moving to Python 3. In the >past I would have said that it's for sure the bytes/str split (which both >the biggest pain and the biggest payoff). > >But if I look carefully into the soul of teams that are still on 2.7 (I >know a few... :-), I think the real reason is that Python 3 changes so many >different things, you have to actually understand your code to port it >(unlike with minor version transitions, where the changes usually spike in >one specific area, and you can leave the rest to normal attrition and >periodic maintenance). The latter is a good point, and sometimes it's a huge challenge to understand the code being ported. A good test suite (and dare I say, doctests :) help a lot with this. I've ported a ton of stuff, and failed at a few. I think all the little changes are mostly tractable, and we've assembled a pretty good stack of documents to help[*]. Sometimes a seemingly easy and mechanical port will produce odd failures, where more domain expertise needs to be brought to bear to get just the right bilingual invocation. But if the underlying code does not itself have a clear bytes/str distinction, then you're doomed. One of my failures was a Python binding for a large C++ project that deeply conflated data and text. Another was a pure Python library that essentially did the same. In both cases, I ended up in a situation where some core types could be neither str nor bytes without some part of the test suite failing miserably. Those are the types of projects that largely get left unported since it's much harder to justify the costs vs. benefits. Cheers, -Barry [*] https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: PEP 467: Minor API improvements for bytes & bytearray
On Aug 17, 2014, at 09:48 PM, Donald Stufft wrote: >from __future__ import bytesdoneright? :D Synonymous to: bytes = bytesdoneright or maybe from bytesdoneright import bytes :) -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 476: Enabling certificate validation by default!
On Aug 30, 2014, at 12:19 PM, M.-A. Lemburg wrote: >The reasoning here is the same as for hash randomization. There >are cases where you want to test your application using self-signed >certificates which don't validate against the system CA root list. > >In those cases, you do know what you're doing. The test would fail >otherwise and the reason is not a bug in your code, it's just >the fact that the environment you're running it in is a test >environment. Exactly. I have test cases where I have to load up a self-signed cert via .load_cert_chain() and in the good-path tests, I expect to make successful https connections. I also have test cases that expect to fail when: * I load bogus self-signed certs * I have an http server masquerading as an https server * I load an expired self-signed cert It certainly makes sense for the default to be the most secure, but other use cases must be preserved. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed schedule for 3.4.2
On Sep 08, 2014, at 10:44 AM, Alex Gaynor wrote:
>*Shifts uncomfortably* it looks like presently there's not a good way to
>change anything about the SSL configuration for urllib.request.urlopen. It
>does not take a `context` argument, as the http.client API does:
>https://docs.python.org/3/library/urllib.request.html#module-urllib.request
>and instead takes the cafile, capath, cadefault args.
>
>This would need to be updated first, once it *did* take such an argument,
>this would be accomplished by:
>
>context = ssl.create_default_context()
>context.verify_mode = CERT_OPTIONACERT_NONE
>context.verify_hostname = False
>urllib.request.urlopen("https://something-i-apparently-dont-care-much-about";,
>context=context)
There's probably an ugly hack possibility that uses unittest.mock.patch. ;)
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed schedule for 3.4.2
On Sep 09, 2014, at 08:20 AM, Nick Coghlan wrote: >We could actually make it an "official" hack: > >import urllib.request >urllib.request.urlopen = urllib.request._unverified_urlopen > >Or else the user can just change the code to call the unverified one >directly. > >All we'd have to do is keep the existing version that doesn't validate >certs properly around under the name "_unverified_urlopen". > >I like this for a few reasons: > >1. It doesn't get much easier than calling function A instead of function B >2. Monkeypatching lets you do a process global hack >3. The name tells you exactly why this is a bad idea >4. It's easy to grep for later after you fix your certs >5. The leading underscore acts as a strong "keep away" signal >6. The leading underscore makes it clear this function may not always be >available (e.g. Jython, older versions of Python) +1. This would also make it easy to mock in a context manager if you just wanted to ignore certs for a small section of code. -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding numbering to PEP 20, the Zen of Python
On Sep 18, 2014, at 08:46 PM, Tim Peters wrote: >See Revelation 22:18. Adding numbers is how heresy begins ;-) Thus Spake Timbot -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 394 - Clarification of what "python" command should invoke
On Sep 19, 2014, at 03:31 AM, Bohuslav Kabrda wrote: >as Fedora is getting closer to having python3 as a default, I'm being more >and more asked by Fedora users/contributors what'll "/usr/bin/python" invoke >when we achieve this (Fedora 22 hopefully). So I was rereading PEP 394 and I >think I need a small clarification regarding two points in the PEP: As Nick points out, I think that the Fedora and Debian ecosystems are aligned on what we *should* do, even if the PEP text needs updating. The way I look at it is that "/usr/bin/python" is user interface. Distributions are completely free to choose whichever Python they want for system scripts, and it's great to see that Fedora is well on their way to making Python 3 the default for system scripts. Debian is also making good progress, though we likely won't complete the transition until Jessie+1. But none of that matters for /usr/bin/python because system scripts will just shebang /usr/bin/python3 and nobody cares. Users almost certainly don't care as long as the script just keeps working, although if there are plugins, it will indirectly matter for them. Only the maintainers of those system scripts really care what the shebang is. For those system scripts, or other /usr/bin entries, that still require Python 2, I think they should already be shebanged /usr/bin/python2. OTOH, users have /usr/bin on their $PATH and they're just going to type 'python' so for them, it's ui, and it *does* matter. In that case I think /usr/bin/python should always point to /usr/bin/python2 which always points to the default Python 2 version (these days, for modern OS versions, almost always 2.7). I don't care how that "pointing" actually works. TL;DR: use #!/usr/bin/python3 for Python 3 system scripts, #!/usr/bin/python2 for Python 2 system scripts, and leave /usr/bin/python to invoke the default Python 2 version. >The important word in the second point is, I think, *whenever*. Trying to >apply these two points to Fedora 22 situation, I can think of several >approaches: >- /usr/bin/python will always point to python3 (seems to go against the first >mentioned PEP recommendation) >- /usr/bin/python will always point to python2 (seems to go against the >second mentioned PEP recommendation, there is no /usr/bin/python if python2 >is not installed) >- /usr/bin/python will point to python3 if python2 is not installed, else it >will point to python2 (inconsistent; also the user doesn't know he's >running and what libraries he'll be able to import - the system can have >different sets of python2-* and python3-* extension modules installed) >- there will be no /usr/bin/python (goes against PEP and seems just wrong) I think the second point is right. If /usr/bin/python2 is not installed, neither should there be a /usr/bin/python. The reasoning is aligned with my views above. Python 2 won't go away, so even if it is not installed *by default* (and I hope we get there in both Debian and Ubuntu), it is entirely reasonable to assume it will eventually get installed. Someday a user is going to install a script that is Python 2, and then they'll get /usr/bin/python2 installed for that script's shebang. At that point, they'll *also* get /usr/bin/python which of course points to /usr/bin/python2. If the user wants to invoke Python 3, it's not hard to type 'python3' and I think that's the message we should be spreading. That already seems pretty ingrained in user habits afaict. If there's general agreement on that, we can work on improving the PEP's wording to make that policy more clear. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 3.5 release schedule PEP
Hi Larry, I think we need a Python 3.5 Release Schedule PEP. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 394 - Clarification of what "python" command should invoke
On Sep 19, 2014, at 10:23 AM, Donald Stufft wrote: >My biggest problem with ``python3``, is what happens after 3.9. FWIW, 3.9 by my rough calculation is 7 years away. >I know Guido doesn’t particularly like two digit version numbers and it’s >been suggested on this list that instead of 3.10 we’re likely to move >directly into 4.0 regardless of if it’s a “big” change or not. > >If that is the case, then all of the user education, ui, etc around >``python3`` would then need to be again updated to ``python4`` *OR* we’d need >a ``python3`` bin which points to ``python4``. If there’s a call to action >for at some point moving ``python`` to invoke Python 3.x at some point then >hopefully at that point Python 4.x could just be ``python``. > >All of this assuming of course that 4.0 isn’t a major break like 3.0 and that >we do 4.0 instead of 3.10 as has been suggested. I seem to recall Guido saying that *if* there's a 4.0, it won't be a major break like Python 3, whatever that says about the numbering scheme after 3.9. Is 7 years enough to eradicate Python 2 the way we did for Python 1? Then maybe Python 4 can reclaim /usr/bin/python. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 394 - Clarification of what "python" command should invoke
On Sep 19, 2014, at 08:40 AM, Guido van Rossum wrote: >Until I say so. Which will happen in the distant future. I'm gonna hid your time machine keys so you didn't find them. -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] shebang policy, and pip
On Oct 08, 2014, at 08:20 AM, Donald Stufft wrote: >Some editors can use shebang lines to control syntax highlighting or linting >(mine for example will lint different for python2 vs python3 shebangs). Some editors can also use `# -*- foo -*-` comments to set up editing modes and there are other ways to specify which version linters to use. I generally avoid shebangs where they aren't needed, and between entry points and -m they rarely are these days. I find most uses are in smaller one-off scripts and such. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] isinstance() on old-style classes in Py 2.7
On Oct 21, 2014, at 10:13 AM, Guido van Rossum wrote: >For new code, and whenever you have an opportunity to refactor old code, >you should use new-style classes, by inheriting your class from object (or >from another class that inherits from object). One nice way to do this module-globally is to set: __metaclass__ = type at the top of your file. Then when you're ready to drop Python 2, it's an easy clean up. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] isinstance() on old-style classes in Py 2.7
On Oct 21, 2014, at 11:22 AM, Guido van Rossum wrote: >Hm. I've never been a fan of that. EIBTI and such... Yeah, I just hate seeing `class Foo(object)` in Python 3 and am too lazy to clean up every class definition. ;) YMMV! Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Move selected documentation repos to PSF BitBucket account?
On Nov 21, 2014, at 10:36 PM, Nick Coghlan wrote: >I'd been taking "must be hosted in PSF infrastructure" as a hard >requirement, but MAL pointed out earlier this evening that in the age >of DVCS's, that requirement may not make sense: if you avoid tightly >coupling your automation to a particular DVCS host's infrastructure, >then reverting back to self-hosting (if that becomes necessary for >some reason) is mostly just a matter of "hg push". > >If that "must be self-hosted" constraint is removed, then the obvious >candidate for Mercurial hosting that supports online editing + pull >requests is the PSF's BitBucket account. For the record, I object to moving *official* PSF resources to proprietary, closed-source infrastructure that we do not control or have access to[*]. As nice and friendly as BitBucket or any other code hosting source is today, there are many reasons why I think this is a bad idea for *official* branches. We are beholden to their policies and operations, which may not align with PSF policies or principles today or in the future. We will not be able to customize the experience for our own needs. We will not have access to the underlying resources should we need them for any purpose. We cannot take action ourselves if some problem occurs, e.g. banning an offensive user. You're right that in a world of dvcs, branches can be mirrored anywhere. For that reason, I have no problem allowing developers to use non-PSF controlled resources *unofficially* if it makes their work easier and doesn't conflict with their own principles. However, in such cases, I still believe that the official, master, blessed repositories remain on PSF controlled infrastructure. Surely that too is possible in the world of dvcs, right? Cheers, -Barry [*] Please note that I am not objecting to our use of lower-level resources donated by our generous sponsors. It's a fine line perhaps, but I have no problem with a wiki running on a VM hosted on some donated hardware, since we still have full access to the machine, the OS, and the application software. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Move selected documentation repos to PSF BitBucket account?
On Nov 23, 2014, at 04:49 PM, Nick Coghlan wrote: >Moving from self-hosted Mercurial repos to externally hosted Mercurial >repos is a low risk change. It reduces maintenance overhead and lowers >barriers to external contribution, both without alienating existing >contributors by forcing them to change their workflows. If those repos are externally maintained, what kind of assurances will people have that they are talking to the *official* repositories of the PSF owned assets? One of the problem IMHO of the democratization of branches that a dvcs provides is knowing when you are interacting with the official code of the project. In general, more democracy is better, but that needs to be balanced with verifiable reputation. Having branches hosted on python.org gives people that immediately. Having branches hosted on other domains means there's more uncertainty. Even aside from the "we should support open source" question (which I feel strongly about but acknowledge others have less allegiance to), it must be absolutely clear that there are repositories which we as the Python development community, bless. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Move selected documentation repos to PSF BitBucket account?
On Nov 23, 2014, at 08:55 AM, Guido van Rossum wrote: >- Moving from Hg to Git is a fair amount of one-time work For anyone seriously interested in this, even experimentally, I would highly suggest looking at Eric Raymond's reposurgeon code. You can google it or find it covered in the vast discussions of his conversion of the Emacs repository from bazaar to git. I don't know for a fact that it would handle an hg to git conversion, but it's the most likely candidate for something so complex. There's a lot of similarity in the history of the Emacs and Python repositories, having gone through many previous iterations of vcs's - in Python's case, at least cvs, svn, and hg. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Move selected documentation repos to PSF BitBucket account?
On Nov 23, 2014, at 03:59 PM, Nick Coghlan wrote: >The learning curve on git is still awful What I find so ironic is that git's model is beautifully simple, but its cli is abysmal, and its manpages are less than helpful. git-push(1) is over 650 lines and it's nearly impossible to dig out the most important bits. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github
On Nov 30, 2014, at 09:54 AM, Ian Cordasco wrote: >- Migrating "data" from GitHub is easy. There are free-as-in-freedom >tools to do it and the only cost is the time it would take to monitor >the process *Extracting* data may be easy, but migrating it is a different story. As the Mailman project has seen in trying to migrate from Confluence to Moin, there is a ton of very difficult work involved after extracting the data. Parsing the data, ensuring that you have all the bits you need, fitting it into the new system's schema, working out the edge cases, adapting to semantic differences and gaps, ensuring that all the old links are redirected, and so on, were all exceedingly difficult[*]. Even converting between two FLOSS tools is an amazing amount of work. Look at what Eric Raymond did with reposurgeon to convert from Bazaar to git. It's a good thing that your data isn't locked behind a proprietary door, for now. That's only part of the story. But also, because github is a closed system, there's no guarantee that today's data-freeing APIs will still exist, continue to be functional for practical purposes, remain complete, or stay at parity with new features. Cheers, -Barry [*] And our huge gratitude goes to Paul Boddie for his amazing amount of work on the project. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Joining the PEP Editors team
On Dec 01, 2014, at 03:54 PM, Chris Angelico wrote: >In response to Guido's call for volunteers, I'm offering myself as a >PEP editor. Who is in charge of this kind of thing? Who manages public >key lists etc? I can add you to the pep editors mailing list. Please send me a off-list message with your preferred email address. I'd prefer it if you GPG signed that message. See here for getting your SSH key registered such that you can make commits to the PEP repo. https://docs.python.org/devguide/faq.html#ssh Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (3.4): - Issue #22966: Fix __pycache__ pyc file name clobber when pyc_compile is
On Dec 02, 2014, at 06:44 AM, Jeremy Kloth wrote:
>This test is failing on the Windows buildbots due to the hard-coded
>path separator. Using `os.pathsep` should work assuming that
>importlib returns normalized paths.
Good catch, thanks, however os.path would be the one to use. Here's the patch
that should fix it. This passes for me on Ubuntu, but I don't have a Windows
machine to do a test build on atm, so I'll just commit this and see how the
buildbots handle it.
diff -r 8badbd65840e Lib/test/test_py_compile.py
--- a/Lib/test/test_py_compile.py Tue Dec 02 09:24:06 2014 +0200
+++ b/Lib/test/test_py_compile.py Tue Dec 02 11:27:16 2014 -0500
@@ -106,9 +106,13 @@
weird_path = os.path.join(self.directory, 'foo.bar.py')
cache_path = importlib.util.cache_from_source(weird_path)
pyc_path = weird_path + 'c'
+head, tail = os.path.split(cache_path)
+penultimate_tail = os.path.basename(head)
self.assertEqual(
-'/'.join(cache_path.split('/')[-2:]),
-'__pycache__/foo.bar.{}.pyc'.format(sys.implementation.cache_tag))
+os.path.join(penultimate_tail, tail),
+os.path.join(
+'__pycache__',
+'foo.bar.{}.pyc'.format(sys.implementation.cache_tag)))
with open(weird_path, 'w') as file:
file.write('x = 123\n')
py_compile.compile(weird_path)
signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github
On Dec 02, 2014, at 06:21 PM, Brett Cannon wrote: >Well, if I'm going to be the Great Decider on this then I can say upfront >I'm taking a pragmatic view of preferring open but not mandating it, >preferring hg over git but not ruling out a switch, preferring Python-based >tools but not viewing it as a negative to not use Python, etc. I would like >to think I have earned somewhat of a reputation of being level-headed and >so none of this should really be a surprise to anyone. I think it's equally important to describe what criteria you will use to make this decision. E.g. are you saying all these above points will be completely ignored, or all else being equal, they will help tip the balance? Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 481 - Migrate Some Supporting Repositories to Git and Github
On Dec 02, 2014, at 07:20 PM, Brett Cannon wrote: >No because only two people have said they like the experiment idea so >that's not exactly enough to say it's worth the effort. =) Plus GitHub >could be chosen in the end. Experimenting could be useful, although if the traffic is disproportionate (e.g. peps are updated way more often than devinabox) or folks don't interact with each of the repos, it might not be very representative. Still, I think it's better to get a visceral sense of how things actually work than to speculate about how they *might* work. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making it possible to accept contributions without CLA (was: My thinking about the development process)
On Dec 09, 2014, at 09:31 AM, Ben Finney wrote: >Rather, I'm asking what, specifically, necessitates this situation. > >What would need to change, for the PSF to accept contributions to the >Python copyrighted works, without requiring the contributor to do >anything but license the work under Apache 2.0 license? My understanding is that the PSF needs the ability to relicense the contribution under the standard PSF license, and it is the contributor agreement that gives the PSF the legal right to do this. Many organizations, both for- and non-profit have this legal requirement, and there are many avenues for satisfying these needs, mostly based on different legal and business interpretations. In the scheme of such things, and IMHO, the PSF CLA is quite reasonable and lightweight, both in what it requires a contributor to provide, and in the value, rights, and guarantees it extends to the contributor. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making it possible to accept contributions without CLA (was: My thinking about the development process)
On Dec 09, 2014, at 07:42 PM, Nick Coghlan wrote: >A more restricted CLA that limited the PSF's outgoing licence choices to >OSI approved open source licenses might address some of the concerns >without causing problems elsewhere, but the combination of being both >interested in core development and having a philosophical or personal >objection to signing the CLA seems to be genuinely rare. The CLA does explicitly say "Contributor understands and agrees that PSF shall have the irrevocable and perpetual right to make and distribute copies of any Contribution, as well as to create and distribute collective works and derivative works of any Contribution, under the Initial License or under any other open source license approved by a unanimous vote of the PSF board." So while not explicitly limited to an OSI approved license, it must still be "open source", at least in the view of the entire (unanimous) PSF board. "OSI approved" would probably be the least controversial definition of "open source" that the PSF could adopt. Cheers, -Barry pgp7O9fxtjjZZ.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition
On Dec 12, 2014, at 08:07 PM, Petr Viktorin wrote: >If anyone is wondering why their favorite Linux distribution is stuck with >Python 2 – well, I can only speak for Fedora, but nowadays most of what's >left are CPython bindings. No pylint --py3k or 2to3 will help there... It's true that some of these are tough. I tried and failed a few times to port Xapian to Python 3. The issue was opened upstream 6 years ago and it's still unresolved: http://trac.xapian.org/ticket/346 OTOH, I ported dbus-python to Python 3 and that worked out much better; we've had solid Python 3 bindings for several years now, which allowed us to port many important Debian/Ubuntu tools to Python 3 and more importantly, do all our new work in Python 3. With other big toolkits like GObject introspection working on Python 3, there's a lot you can do. IME, if the underlying model is string/bytes clean, then the C extension port can sometimes be easier than pure-Python, thanks to cpp games. D-Bus's model is pretty clean, Xapian I found to be not so much (it doesn't help that Xapian is C++ ;). We're actually not terribly far from switching Debian and Ubuntu's default to Python 3. On Debian, the big blocker is the BTS code (which uses SOAP) and on Ubuntu it's the launchpadlib stack. I hope to find time after Jessie to work on the former, and before 16.04 LTS to work on the latter. Not that I disagree that there's a long tail of code that would still benefit a significant population if it got ported to Python 3. By far Python 3 is a better language, with a better stdlib, so the work is worth it. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition
On Dec 13, 2014, at 12:29 AM, Donald Stufft wrote: >For what it’s worth, I almost exclusively write 2/3 compatible code (and >that’s with the “easy” subset of 2.6+ and either 3.2+ or 3.3+) and doing so >does make the language far less fun for me than when I was writing 2.x only >code. For myself, the way I'd put it is: With the libraries I maintain, I generally write Python 2/3 compatible code, targeting Python 2.7 and 3.4, with 2.6, 3.3, and 3.2 support as bonuses, although I will not contort too much to support those older versions. Doing so does make the language far less fun for me than when I am writing 3.x only code. All applications I write in pure Python 3, targeting Python 3.4, unless my dependencies are not all available in Python 3, or I haven't yet had the cycles/resources to port to Python 3. Writing and maintaining applications in Python 2 is far less fun than doing so in Python 3. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition
On Dec 14, 2014, at 10:14 AM, Nick Coghlan wrote: >Barry, Petr, any of the other folks working on distro level C extension >ports, perhaps one of you would be willing to consider an update to the C >extension porting guide to be more in line with Brett's latest version of >the Python level porting guide? It's probably at least worth incorporating the quick guide on the wiki into the howto: https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition
On Dec 16, 2014, at 02:15 PM, Skip Montanaro wrote: >While he doesn't explicitly say so, I got the distinct impression reading >his recent blog post that he supports one source, not forked sources. I've ported a fair bit of code, both pure-Python and C extensions, both libraries and applications. For successful library ports to Python 3 that need to remain Python 2 compatible, I would almost always recommend a single source, common dialect, no-2to3 approach. There may be exceptions, but this strategy has proven effective over and over. I generally find I don't need `six` but it does provide some nice conveniences that can be helpful. With something like tox running your test suite, it doesn't even have to be painful to maintain. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: Update PEP 1 to allow an explicit "Provisional" status for PEPs
On Dec 18, 2014, at 06:57 AM, Nick Coghlan wrote: >However, I'd be happier if we could communicate that status more >explicitly through the PEP process, especially as I think such a >capability would be useful more generally as we move towards >implementing metadata 2.0 and potentially other enhancements for pip >7+ next year. > >If folks are OK with this idea, I'll go ahead and make the appropriate >changes to PEP 1 and the PEP index generator. I'm also happy to file a >tracker issue, or write a short PEP, if folks feel making such a >change requires a little more formality in its own right. Hi Nick. What specific changes do you propose to PEP 1 and/or the PEP process? FWIW, if they are fairly simple, then I think a tracker issue with at least the PEP 1 authors nosied would be fine. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Any grammar experts?
On Jan 25, 2015, at 09:31 PM, R. David Murray wrote:
>> > > {*x for x in it}
>> > >
>> > > which is a set comprehension, while the other is a dict comprehension :)
>> > >
>> >
>> > That distinction doesn't bother me -- you might as well claim it's
>> > confusing that f(*x) passes positional args from x while f(**x) passes
>> > keyword args.
>> >
>> > And the varargs set comprehension is similar to the varargs list
>> > comprehension:
>> >
>> > [*x for x in it]
>> >
>> > If `it` were a list of three items, this would be the same as
>> >
>> > [*it[0], *it[1], *it[2]]
>>
>> I find all this unreadable and difficult to understand.
>
>I did too, before reading the PEP.
>
>After reading the PEP, it makes perfect sense to me. Nor is the PEP
>complicated...it's just a matter of wrapping your head around the
>generalization[*] of what are currently special cases that is going on
>here.
It does make sense after reading the PEP but it also reduces the readability
and instant understanding of any such code. This is head-scratcher code that
I'm sure I'd get asked about from folks who aren't steeped in all the dark
corners of Python. I don't know if that's an argument not to adopt the PEP,
but it I think it would be a good reason to recommend against using such
obscure syntax, unless there's a good reason (and good comments!).
Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Any grammar experts?
On Jan 26, 2015, at 10:55 AM, Ethan Furman wrote:
>In the your example
>
> from_env = {'a': 12}
> from_config = {'a': 13}
>
> f(**from_env, **from_config)
>
>I would think 'a' should be 13, as from_config is processed /after/ from_env.
>
>So which is it?
In the face of ambiguity, refuse the temptation to guess.
Cheers,
-Barry
pgpQ9ie65GEHm.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (no subject)
On Feb 09, 2015, at 07:46 PM, Neil Girdhar wrote: >Also, regarding calling argument order, not any order is allowed. Regular >arguments must precede other kinds of arguments. Keyword arguments must >precede **-args. *-args must precede **-args. However, I agree with >Antoine that PEP 8 should be updated to suggest that *-args should precede >any keyword arguments. It is currently allowed to write f(x=2, *args), >which is equivalent to f(*args, x=2). But if we have to add a PEP 8 admonition against some syntax that's being newly added, why is this an improvement? I had some more snarky/funny comments to make, but I'll just say -1. The Rationale in the PEP doesn't sell me on it being an improvement to Python. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
I'm just now catching up on this thread, so hopefully these thoughts are still relevant. IIRC, the PEP has been scaled back to mostly a convenience around packing an existing directory into a .pyz file, and (on Windows) adding an association for those as executable Python zip file. To the extent that I care about Windows, +1. I don't know exactly what the procedure would be to claim .pyz for *nix, e.g. updating /etc/mime.types, but I think the PEP should at least mention this. I think we want to get as official support for .pyz files on *nix as possible. The broader question of pex, pyzaa, etc. is an important one for Python, IMHO. Having a standard single-executable distribution story will help Python continue to compete on platforms that work beyond the distribution models we've come to think about. App models, and transactional systems such as Ubuntu Core, etc. are gaining mindshare and Python's traditional way of deploying applications doesn't really fit very well, and that may discourage developers from using Python. Whether Python itself wants to put a stake in the ground today for that is an open question, but adding support in Python for extension modules in zips (whether via save-to-disk or new operating system APIs) would be useful to explore. I happen to like pex, but what I really want is something like: $ pyvenv foo $ source foo/bin/activate $ pip install coolthing $ pyzip -o coolthing.pyz . $ deactivate $ ./coolthing.pyz This is outside the scope of PEP 441, but if anybody at Pycon wants to explore this further, I'm in. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 15, 2015, at 05:15 PM, Petr Viktorin wrote: >So I'd suggest `/usr/bin/env python3` for the default. I really don't want `/usr/bin/env anything`. In general, using /usr/bin/env is a fine tool for developers, but a terrible tool for deployment. It's just too easy for end users to break an application by mucking with their $PATH. In Debian/Ubuntu, we don't allow these shebangs in system scripts, and in fact the package build tools will rewrite shebangs to avoid them. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 16, 2015, at 04:16 PM, Paul Moore wrote: >So, the options I see: > >1. Stick with /usr/bin/env python >2. No shebang unless -p is specified >3. Unix users come up with a solution which is the same as the above >for Windows users, but which suits them better. #2 seems to me to be the most reasonable alternative. The resulting pyz files (built w/o -p) would still be explicitly executable, just like .py files. However, -p must be able to accept any number of strings, including "/usr/bin/env python3" if the user wants that. Probably the best thing to do (on *nix at least) is, if the path is absolute, use the given string verbatim. If the path is relative, search for the given executable on $PATH and use the first one found. If nothing is found, use what's given explicitly. I think that will give all the reasonable use cases. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 15, 2015, at 05:21 PM, Thomas Wouters wrote: >FWIW, we have a similar tool to 'pyzaa' at Google, although a lot older (it >has a pure-python implementation of zipimport, because it had to work with >Python 2.2 back in the day) that *does* support extension modules and other >shared libraries (by extracting them to disk on program startup.) We're >also working to replace that by loading the extension modules directly from >the archive (using dlopen_with_offset as described by ppluzhnikov in >https://sourceware.org/bugzilla/show_bug.cgi?id=11767, although it should >also be possible using dlopen_ehdr/dlopen_phdr), which requires that >extension modules are stored uncompressed (simple) and page-aligned >(harder, as the zipfile.ZipFile class doesn't directly support >page-aligning anything, but it turns out it's easy to hack around by >overriding _writecheck :P) And yes, we really are in a position to modify >glibc to make our life distributing Python applications easier, internally; >the old code involves shell and Python bootstrap code that tries very hard >to avoid race conditions and security problems, which is a pain to maintain. So, let's add this to zipimport so we can remove an unfortunate limitation and avoid everyone having to reinvent the wheel! Oh, and the dlopen extension sound pretty cool. Can't wait until Google gets that into glibc so us mere mortals can use it. Seems reasonable to add this to zipimport too, with whatever feature guards are necessary. You'll want to give your future self an opportunity to trot out the keys to the time machine. :) >It might be nice to consider those use-cases in pyzapp as well, especially >once the glibc feature is released. It requires some fairly big changes to >zipimport (I ended up rewriting the whole thing) but we can easily >opensource the code, if there's any interest at all. +1 >I'd be happy to discuss this in more detail at PyCon, at which time we should >be deploying code internally using all of this. +1 Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 16, 2015, at 11:21 PM, Paul Moore wrote: >Default Interpreter >--- > >The initial draft of this PEP proposed using ``/usr/bin/env python`` >as the default interpreter. The other reasonable alternative for a default shebang is sys.executable. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 17, 2015, at 08:58 PM, Paul Moore wrote: >But I would say that anything that is added to zipimport should be >cross-platform. Having support for C extensions in zipimport on Unix >only will just add another way in which Python applications can >inadvertantly be non-portable... (It should be possible to support >both Windows and Unix, though, as py2exe has the Windows side of >things covered so presumably the code used in py2exe could be ported >to zipimport). The actual "import an extension module" support in zipimport does need to be cross-platform, but it can work differently depending on the platform. For example, if extended-dlopen is available on your Linux machine, zipimport could just use that. For a traditional-dlopen-only machine, it would copy-to-filesystem. It would do whatever Windowsy thing makes sense over there too. I'm much less concerned about the cross-platform portability of the resulting pyz files. There's lots of reasons why an application built in (or for) my Linux machine might not work properly on your Windows machine, but FWIW if it were the only problem, doing a native rebuild on the target platform (or via supported cross-building) would be fine with me. Cheers, -Barry pgpD_dIRHk2Px.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 17, 2015, at 08:53 PM, Paul Moore wrote: >On 17 February 2015 at 18:56, Barry Warsaw wrote: >>>The initial draft of this PEP proposed using ``/usr/bin/env python`` >>>as the default interpreter. >> >> The other reasonable alternative for a default shebang is sys.executable. > >That's instantly non-portable. On my PC, it'd give >C:\Apps\Python34\python.exe. Which won't work on Unix, or on PCs that >use the standard install location, or when I upgrade to Python 3.5... But it doesn't matter (at least to me). The scenarios I want to support don't include building a pyz on my Linux machine, giving it to you, and having you able to run it natively on your Windows machine. There are lots of interesting use cases we could still support. I could build a single-file application and deploy it into my OS's system archives. I could deploy that thing into my platform's app store. I could hand it over to a colleague running an identical version of the OS. I could deploy it into my data center. I could install it on my phone. Don't discount sys.executable to quickly. :) Cheers, -Barry pgpmiI1N7GM6i.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 17, 2015, at 08:52 PM, Paul Moore wrote:
>I'm pretty sure that's the way the general feeling is going.
Though the more I think about it, the more I like sys.executable. :)
>> However, -p must be able to accept any number of strings, including
>> "/usr/bin/env python3" if the user wants that.
>
>The code simply writes
>'#!{}\n'.format(p_option).encode(sys.filesystemencoding()) to the file, so
>you can put whatever you want in. Given that it isn't the name of the Python
>executable, maybe the option should be --interpreter instead?
(Quoting out of order.)
>I'm not quite sure what you mean here, but maybe you're thinking that
>the -p option is the executable name rather than what gets put in the
>#! line directly? Let me know if it's not covered by what I've already
>said.
I was thinking it would be useful to mimic virtualenv's -p/--python option,
but I think that doesn't actually do the $PATH parsing, so maybe taking -p
verbatim is fine.
>Oh, and am I right that the shebang line should be encoded using the
>filesystem encoding on Unix? I know 99.999% of use cases will be
>ascii, but someone could have a Python interpreter in
>/home/léon/.local/bin/python...
Well, actually probably utf-8 in most cases, at least for Python 3 on *nix.
I'm not sure sys.getfilesystemencoding() is the right encoding, rather than
sys.getdefaultencoding(), if you're talking about the encoding of the shebang
line rather than the encoding of the resulting pyz filename.
Cheers,
-Barry
pgpwgqU3Yj0vR.pgp
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 17, 2015, at 08:44 PM, Paul Moore wrote: >I'd like to see a good single-file bundled application format for >Python (on Windows, there's py2exe, which is fantastic, but often I >simply don't *want* to bundle Python and the stdlib with my code). I agree, for deployment to environments where you can guarantee that the basic Python infrastructure is available. It may not be though, and perhaps we can relegate that to py2exe, cx_freeze, and friends, although I would say that having such support built into Python would be very cool, and of high value to our users. That's not to play down the usefulness of good executable zip support, pared of the stdlib and interpreter. OTOH, it would be nice to think ahead, where the same tool could generate both types of single-file executables. >Handling of C extensions in zipfiles isn't so much an executable zip >issue as a zipimport one. Agreed. >(Although promoting the executable zip format would help persuade projects to >*use* those features rather than assuming a filesystem). Donald Stufft has a >proposal on resource APIs for importlib that would help with this. Yep, which I support. :) pkg_resources.resource_filename() is the way to do that today and it abstracts away all the nastiness. Mostly agree with your other comments, though I haven't played with pyzzer and pex is quite nice, for the pared down use case, although it's not quite integrated with virtualenv (I've had some discussions with pex's author about that). OTOH, I think it *would* be useful to have this built into Python. We almost have too many choices, and may people I've talked to lately (including experienced Pythonistas) have trouble choosing the right tool. But that can come later; let's get some basic support into Python first, and a good start IMHO is executable zips (including support for extension modules) with the pkg_resources/importlib API additions. Cheers, -Barry pgp0pH78owXV6.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 17, 2015, at 01:25 PM, Ethan Furman wrote: >On 02/17/2015 01:13 PM, Barry Warsaw wrote: > >> There are lots of interesting use cases we could still support. I could >> build a single-file application and deploy it into my OS's system archives. >> I could deploy that thing into my platform's app store. I could hand it >> over to a colleague running an identical version of the OS. I could deploy >> it into my data center. I could install it on my phone. > >All those locations have Python installed in the same place? Even your >phone?? But of course! :) Cheers, -Barry pgpy0nxwsuCpI.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 17, 2015, at 09:52 PM, Paul Moore wrote: >I see no problem with either having zipapp develop into a much >fuller-featured solution, or having an externally developed tool >brought into the stdlib and replacing zipapp. I just don't think >there's anything mature enough for that role out there yet, and I >don't feel that we know enough yet to design the right solution from >scratch in the PEP. > >As you say, though, that can come later. If 3.5 includes PEP 441 and >Donald's proposed importlib additions, I'll be happy. If it also >includes zipimport support for C extensions I'll be ecstatic! (But I >suspect that one's more a 3.6 timeframe) Agreed! The only difference between executable zips and the full-featured self-contained app is the goop you put at the front of the file, right? :) Cheers, -Barry pgp8tGSl3jwSy.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 17, 2015, at 09:45 PM, Paul Moore wrote: >If Python zip applications got the ability to include binary extensions, they >would *definitely* not be portable (we don't want to go down the route of >wheel-like compatibility tags for an application file format). Agreed! >Currently, I'm trying to leave this aspect of the PEP open, so the PEP >describes what the format is and how it works, but doesn't try to imply >particular use cases. That way people can use it how they prefer, and the PEP >doesn't put people off with a lot of caveats. My current draft of the >documentation page for the zipapp module includes an example of setting a >shebang line, with some notes on issues to consider for a portable shebang >line if you propose distributing your application - but that's as far as I >want to go. I think that's totally reasonable. Admittedly, I was hijacking your PEP discussion for some pie-in-the-sky musing. :) Cheers, -Barry pgpLu9N2fmjsg.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 441 - Improving Python ZIP Application Support
On Feb 18, 2015, at 09:16 PM, Paul Moore wrote: >That covers mime types, but not file extensions, so it's not really >what *I* thought Barry was talking about. I was actually thinking about both. :) The Wikipedia page has several external links to collections of file extensions, but I don't know that there's *an* official registry. http://en.wikipedia.org/wiki/File_extension Picking FileInfo.com at random, and searching for a few .py* extensions, you find the usual suspects, plus a few others, but all are Python related, and there's no .pyz extension currently registered. >Also, I don't think reserving anything is something I, as an >individual (and specifically a non-Unix user) should do. It probably >should be handled by the PSF, as the process seems to need a contact >email address... Probably so. Seems like they could pick one or more of these registries and register pyz for this purpose. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for Pronouncement: PEP 441 - Improving Python ZIP Application Support
On Feb 24, 2015, at 08:20 PM, Paul Moore wrote: >(side note: --python/-p or --interpreter/-i?) and set the entry point, Both virtualenv and (I think) pex use --python/-p so that seems to be the overwhelming trend . >To modify an archive could be done using > >python -m zipapp old.pyz new.pyz [-p interpreter] > >Default is to strip the shebang (no -p option). There's no option to >omit the target and do an inplace update because I feel the default >action (strip the shebang from the existing file with no backup) is >too dangerous. You have to be careful about the case where old.pyz == new.pyz (e.g. either handling this case safely or complaining loudly) , but also you could handle it by using a .tmp file and renaming. E.g. old.pyz -> old.pyz.bak and old.pyz.tmp -> old.pyz. >3. What to call the "show the shebang line" option I don't know how useful this is, given that (on *nix at least) you can effectively do the same with head(1). Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for Pronouncement: PEP 441 - Improving Python ZIP Application Support
On Feb 25, 2015, at 07:33 PM, Paul Moore wrote: >The module defines two convenience functions: > > >.. function:: create_archive(directory, target=None, interpreter=None, >main=None) > > Create an application archive from *source*. The source can be any > of the following: I think you meant s/directory/source/ in the signature. Other than that, +1. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 488: elimination of PYO files
On Mar 07, 2015, at 12:30 PM, Scott Dial wrote: >As a packager, this PEP is a bit silent on it's expectations about what >will happen with (for instance) Debian and Fedora packages for Python. >My familiarity is with Fedora, and on that platform, we ship .pyc and >.pyo files (using -O for the .pyo). Is it your expectation that such >platforms will still distribute -O only? Or also -OO? In my world, all >of the __pycache__ directories are owned by root. On Debuntu, we don't create pyo files by default, although IIRC it's an option. The problem has been the collision between -O and -OO, so this PEP may open the door to us generating both optimization levels automatically. It's not something that's very high on my todo list, but if there's enough interest it probably wouldn't be difficult to add. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons
On Mar 18, 2015, at 03:46 PM, Orion Poplawski wrote: >We're starting a discussion in Fedora about setting the default shbang for >system python executables and/or daemons to python -s or python -Es (or ?). We've talked about this in Debian/Ubuntu land and the general consensus is that for Python 2, use -Es and for Python 3 use -I (which implies -Es). I'm not sure we're consistent yet in making sure our build tools install these switches in our shebangs, but I'm hoping after Jessie we can make some progress on that. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons
On Mar 18, 2015, at 05:31 PM, Victor Stinner wrote:
>Does it work to pass command line options to Python in the shebang?
Yes, but only one "word", thus -Es or -I.
We've often mused about whether it makes sense to have two Pythons on the
system. One for system scripts and another for users. System Python
('/usr/bin/spython' perhaps) would be locked down and only extensible by
system packages. On Debuntu that might mean no /usr/local on sys.path. It
would also have a much more limited set of installed packages, i.e. only those
needed to support system functionality.
/usr/bin/python2 and /usr/bin/python3 then would be user tools, with all the
goodness they currently have.
It's never gotten much farther than musings, but protecting the system against
the weird things people install would be a good thing. OTOH, this feels a lot
like virtual environments so maybe there's something useful to be mined there.
Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons
On Mar 18, 2015, at 02:44 PM, Toshio Kuratomi wrote: >Interesting, I've cautiously in favor of -s in Fedora but the more I've >thought about it the less I've liked -E. It just seems like PYTHONPATH is >analagous to LD_LIBRARY_PATH for C programs and PATH for shell scripting. >We leave both of those for local admins and users to affect the behaviour of >programs if they needed to. It is, and it isn't. It's different because you can always explicitly override the shebang line if needed. So if a local admin really needed to override $PYTHONPATH (though I can't come up with a use case right now), they could just: $ python3 -s /usr/bin/foo >Was there some discussion of -E specifically in Debian where it was >consciously decided that PYTHONPATH was not analagous or that the benefit >risk was different than for those other env vars? I'd have to go digging around the archives. It wasn't a recent discussion IIRC. Cheers, -Barry pgpYnskkCwUKH.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons
On Mar 18, 2015, at 05:51 PM, Donald Stufft wrote: >I’ve long wished that the OS had it’s own virtual environment. A lot of >problems seems to come from trying to cram the things the OS wants with the >things that the user wants into the same namespace. Yep, and those breakages can be difficult to debug. Comment #47 for Bug 123456: "Oh wait, you installed *what?!*" Cheers, -Barry pgpkruqPndrJS.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons
On Mar 19, 2015, at 11:15 AM, Toshio Kuratomi wrote: >I could see that as a difference. However, the environment variables >give users the ability to change things globally whereas overriding >the shebang line is case-by-case so it's not a complete replacement >of the functionality. You make some good points. I guess it's a trade-off between flexibility and a known secure execution environment. I'm not sure there's a right answer; different admins might have valid different opinions. Cheers, -Barry pgpniq0DKukw3.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] some minor questions about pep8
On Mar 20, 2015, at 08:53 PM, Guido van Rossum wrote: >FWIW, I think __version__, __author__ etc. were bad ideas. Almost nobody >manages these correctly. Note that the PEP 8 section starts with less than >an endorsement: "If you *have* to have Subversion, CVS, or RCS crud in your >source file, do it as follows." I tend to agree. Individual module version numbers are mostly useless, especially with today's hash-based vcses. That's different than package versions, and for which I really need to resurrect and update PEP 396. >That said, if an official answer is required, common sense would suggest >that __version__ should go before the imports. (I would put it before the >docstring too, except then the docstring wouldn't be a docstring any more. >Go figure.) And after __future__ imports too, right? Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] some minor questions about pep8
On Mar 20, 2015, at 09:59 PM, Ian Lee wrote: >shebang >docstring >imports >dunder assignments >other code... I generally put imports after dunders, except for __future__. I want the first screenful of code to contain them, and imports can be a few dozen lines in some modules. -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] some minor questions about pep8
On Mar 21, 2015, at 05:00 PM, Donald Stufft wrote: >I sort of think (though I haven’t completely convinced myself) that adding >something like __version__ to a package is a work around to the fact that >we don’t have an API that lets someone ask “what is the distribution and >version that this module/import-package came from”. I tend to agree. Having that would solve one of the big problems that lead to PEP 394 thinking, that of having to update version numbers in more than one place. ISTM the best place to do it is once in setup.py and let the metadata flow. The only downside is for doing in-tree development without 'installing' the package (e.g. absence of venv, tox, or similar). Cheers, -Barry pgpMGj2h7VWfU.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?
On Apr 03, 2015, at 02:56 AM, Larry Hastings wrote: >My Windows development days are firmly behind me. So I don't really have an >opinion here. So I put it to you, Windows Python developers: do you care >about GnuPG signatures on Windows-specific files? Or do you not care? They're not mutually exclusive, so why not do both? I think the advantage of being able to verify the files on any platform is useful. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Do we need to sign Windows files with GnuPG?
On Apr 04, 2015, at 02:41 PM, Steve Dower wrote: >"Relying only on Authenticode for Windows installers would result in a break >in technology w/r to the downloads we make available for Python, since all >other files are (usually) GPG signed" It's the "only" part I have a question about. Does the use of Authenticode preclude detached GPG signatures of the exe file? I can't see how it would, but maybe there's something (well, a lot of somethings ;) I don't know about Windows. If not, then what's the problem with also providing a GPG signature? Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 8 update
On Apr 06, 2015, at 06:08 PM, Guido van Rossum wrote: >I've taken the liberty of adding the following old but good rule to PEP 8 >(I was surprised to find it wasn't already there since I've lived by this >for ages): > > Be consistent in return statements. Either all return statements in a > function should return an expression, or none of them should. If any return > statement returns an expression, any return statements where no value is > returned should explicitly state this as return None, and an explicit > return statement should be present at the end of the function (if > reachable). +1 Odd synchronicity: Today I discovered an old interface that was documented as returning a "thing or None" but the latter was relying on implicit None return in some cases. Fixed of course in exactly the way PEP 8 now recommends. :) Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Surely "nullable" is a reasonable name?
On Apr 19, 2015, at 01:19 AM, Larry Hastings wrote:
>We should rename "types" to "accept". "accept" should takes a set of types;
>these types specify the types of Python objects the Clinic parameter should
>accept. For the funny pseudo-types needed in some Clinic declarations
>("buffer", "robuffer", and "rwbuffer"), Clinic provides empty class
>declarations so these behave like types too.
Having only followed the AC discussions tangentially, I have to say that the
above suggestion and the given examples make a lot more intuitive sense to me.
I had the same initial thought as Glenn regarding type annotations. It's fine
that they're separate concepts, but it's also helpful that Larry's suggestion
above seems to align them better.
Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 20, 2015, at 07:30 PM, Harry Percival wrote: >tldr; type hints in python source are scary. Would reserving them for stub >files be better? I think so. I think PEP 8 should require stub files for stdlib modules and strongly encourage them for 3rd party code. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Type hints -- a mediocre programmer's reaction
On Apr 21, 2015, at 01:34 PM, Steven D'Aprano wrote: >Putting the type information in a stub file is an exponentially more distant >fourth best, or to put it another way, *the worst* solution for where to put >type hints. Not only do you Repeat Yourself with the name of the parameter, >but also the name of the function (or method and class) AND module. The type >information *isn't even in the same file*, which increases the chance of it >being lost, forgotten, deleted, out of date, unmaintained, etc. All true, but the trade-off is the agility and ease of working on, reading, and understanding the stdlib, all of which IMHO will suffer if type hints are inlined there. What I don't want to have happen is for type hints to slowly infiltrate the stdlib to the point where no patch will be accepted unless it also has hints. I have the same gut reaction to this as RDM expressed a few posts back. One of the thing I love most about Python is its dynamic typing. I'm all for giving linter developers a hook for experimenting with their tools, I just don't care and I don't want to *have* to care. Maybe some day they will make it so compelling that I will care, but I want to be convinced first. So I think stub files in the stdlib are the right compromise today. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Apr 21, 2015, at 01:26 PM, Yury Selivanov wrote: >The updated version of the PEP should be available shortly at >https://www.python.org/dev/peps/pep-0492 and is also pasted in this email. There's a lot to like about PEP 492. I only want to mildly bikeshed a bit on the proposed new syntax. Why "async def" and not "def async"? My concern is about existing tools that already know that "def" as the first non-whitespace on the line starts a function/method definition. Think of a regexp in an IDE that searches backwards from the current line to find the function its defined on. Sure, tools can be updated but it is it *necessary* to choose a syntax that breaks tools? def async useful(): seems okay to me. Probably the biggest impact on the PEP would be symmetry with asynchronous with and for. What about: with async lock: and for async data in cursor: That would also preserve at least some behavior of existing tools. Anyway, since the PEP doesn't explicitly describe this as an alternative, I want to bring it up. (I have mild concerns about __a*__ magic methods, since I think they'll be harder to visually separate, but here the PEP does describe the __async_*__ alternatives.) Cheers, -Barry pgpxCIhVKHrXy.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Apr 23, 2015, at 02:02 PM, Yury Selivanov wrote: >To my eye 'async def name()', 'async with', 'async for' look >better than 'def async name()', 'with async' and 'for async'. >But that's highly subjective. Would you be willing to add this as an alternative to the PEP, under the "Why async def" section probably? As with all such bikesheds, Guido will pick the color and we'll ooh and ahh. :) Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Apr 24, 2015, at 11:17 PM, Steven D'Aprano wrote:
>It seems to me that tools that search for r"^\s*def\s+spam\s*\(" are
They would likely search for something like r"^\s*def\s+[a-zA-Z0-9_]+" which
will hit "def async spam" but not "async def".
Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On Apr 24, 2015, at 08:03 PM, Greg Ewing wrote: >Putting it at the end would seem least likely to >cause breakage: > >def useful() async: That's not bad IMHO. I wonder how crazy it is in the face of, ahem, function annotations. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A macro for easier rich comparisons
On Apr 28, 2015, at 11:13 AM, Victor Stinner wrote: >It would be nice to have a six module for C extensions. I'm quite sure >that many projects are already full of #ifdef PYTHON3 ... #else ... >#endif macros. Maybe encapsulating some of the recommendations here: https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef#Python_extension_modules (We really need to collect all this information in on place.) >> #define Py_RETURN_RICHCOMPARE(val1, val2, op) I think this macro would make a nice addition to the C API. It might read better as `Py_RETURN_RICHCOMPARE(val1, op, val2)`. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] How shall we conduct the Python 3.5 beta and rc periods? (Please vote!)
On May 12, 2015, at 10:38 AM, Larry Hastings wrote: >It doesn't. Workflows 0 and 2 mean no public development of 3.6 until after >3.5.0 final ships, as per tradition. I still think it's a good idea to focus primarily on 3.5 while it's in the beta/rc period, but if you want to allow for landings on what will be 3.6, then going with workflow 1 will be an interesting social experiment. I'm fine with any of them as long as the workflow is *well documented*, preferably in the devguide. Cheers, -Barry pgpr_x4ASMLvd.pgp Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553: Built-in debug()
On Sep 6, 2017, at 23:10, Terry Reedy wrote: > > Environmental variables are set to strings, not objects. It is not clear how > you intend to handle the conversion. The environment variable names a module import path. Without quibbling about the details of the syntax (because honestly, I’m not convinced it’s a useful feature), it would work roughly like: * The default value is equivalent to PYTHONBREAKPOINTHOOK=pdb.set_trace * breakpoint() splits the value on the rightmost dot * modules on the LHS are imported, then the RHS is getattr’d out of that * That’s the callable breakpoint() calls -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553: Built-in debug()
On Sep 7, 2017, at 07:46, Guido van Rossum wrote: > Without following all this (or the PEP, yet) super exactly, does this mean > you are satisfied with what the PEP currently proposes or would you like > changes? It's a little unclear from what you wrote. I’m also unsure whether Terry is good with the existing PEP or suggesting changes. > I think that's true for any IDE that has existing integrated debug > capabilities. However for every IDE I would hope that calling breakpoint() > will *also* break into the IDE's debugger. My use case is that sometimes I > have a need for a *conditional* breakpoint where it's just much simpler to > decide whether to break or not in Python code than it would be to use the > IDE's conditional breakpoint facilities. That certainly aligns with my own experience and expectations. I guess I’m a fellow old dog. :) -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553: Built-in debug()
On Sep 7, 2017, at 10:00, Christian Heimes wrote: > Setuptools' entry points [1] use colon between import and function, e.g. > "pdb:set_trace" would import pdb and then execute set_trace. The > approach can be augmented to allow calling a class method, too. > > So > > "package.module:myclass.classfunc" > > would do : > > from package.module import myclass > myclass.classfunc Yep, that’s how it's described in the PEP 553 open issue. I just didn’t include that complication in my response. -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 553 V2 - builtin breakpoint() (was Re: PEP 553: Built-in debug())
Thanks for all the great feedback folks! Here then is PEP 553 version 2. The notable changes are: * Change the name of the built-in from debug() to breakpoint() * Modify the signature to be breakpoint(*args, **kws) https://www.python.org/dev/peps/pep-0553/ Included below for convenience. Cheers, -Barry PEP: 553 Title: Built-in breakpoint() Author: Barry Warsaw Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2017-09-05 Python-Version: 3.7 Post-History: 2017-09-05, 2017-09-07 Abstract This PEP proposes adding a new built-in function called ``breakpoint()`` which enters a Python debugger at the point of the call. Additionally, two new names are added to the ``sys`` module to make the debugger pluggable. Rationale = Python has long had a great debugger in its standard library called ``pdb``. Setting a break point is commonly written like this:: foo() import pdb; pdb.set_trace() bar() Thus after executing ``foo()`` and before executing ``bar()``, Python will enter the debugger. However this idiom has several disadvantages. * It's a lot to type (27 characters). * It's easy to typo. The PEP author often mistypes this line, e.g. omitting the semicolon, or typing a dot instead of an underscore. * It ties debugging directly to the choice of pdb. There might be other debugging options, say if you're using an IDE or some other development environment. * Python linters (e.g. flake8 [1]_) complain about this line because it contains two statements. Breaking the idiom up into two lines further complicates the use of the debugger, These problems can be solved by modeling a solution based on prior art in other languages, and utilizing a convention that already exists in Python. Proposal The JavaScript language provides a ``debugger`` statement [2]_ which enters the debugger at the point where the statement appears. This PEP proposes a new built-in function called ``breakpoint()`` which enters a Python debugger at the call site. Thus the example above would be written like so:: foo() breakpoint() bar() Further, this PEP proposes two new name bindings for the ``sys`` module, called ``sys.breakpointhook()`` and ``sys.__breakpointhook__``. By default, ``sys.breakpointhook()`` implements the actual importing and entry into ``pdb.set_trace()``, and it can be set to a different function to change the debugger that ``breakpoint()`` enters. ``sys.__breakpointhook__`` then stashes the default value of ``sys.breakpointhook()`` to make it easy to reset. This exactly models the existing ``sys.displayhook()`` / ``sys.__displayhook__`` and ``sys.excepthook()`` / ``sys.__excepthook__`` hooks [3]_. The signature of the built-in is ``breakpoint(*args, **kws)``. The positional and keyword arguments are passed straight through to ``sys.breakpointhook()`` and the signatures must match or a ``TypeError`` will be raised. The return from ``sys.breakpointhook()`` is passed back up to, and returned from ``breakpoint()``. Since ``sys.breakpointhook()`` by default calls ``pdb.set_trace()`` by default it accepts no arguments. Open issues === Confirmation from other debugger vendors We want to get confirmation from at least one alternative debugger implementation (e.g. PyCharm) that the hooks provided in this PEP will be useful to them. Breakpoint bytecode --- Related, there has been an idea to add a bytecode that calls ``sys.breakpointhook()``. Whether built-in ``breakpoint()`` emits this bytecode (or gets peephole optimized to the bytecode) is an open issue. The bytecode is useful for debuggers that actively modify bytecode streams to trampoline into their own debugger. Having a "breakpoint" bytecode might allow them to avoid bytecode modification in order to invoke this trampoline. *NOTE*: It probably makes sense to split this idea into a separate PEP. Environment variable Should we add an environment variable so that ``sys.breakpointhook()`` can be set outside of the Python invocation? E.g.:: $ export PYTHONBREAKPOINTHOOK=my.debugger:Debugger This would provide execution environments such as IDEs which run Python code inside them, to set an internal breakpoint hook before any Python code executes. Call a fancier object by default Some folks want to be able to use other ``pdb`` interfaces such as ``pdb.pm()``. Although this is a less commonly used API, it could be supported by binding ``sys.breakpointhook`` to an object that implements ``__call__()``. Calling this object would call ``pdb.set_trace()``, but the object could expose other methods, such as ``pdb.pm()``, making invocation of it as handy as ``breakpoint.pm()``. Implementation == A pull request exists with the proposed implementation [4]_. Rejected alternatives ===
Re: [Python-Dev] PEP 553: Built-in debug()
On Sep 7, 2017, at 14:04, Fred Drake wrote: > > On Thu, Sep 7, 2017 at 4:52 PM, Terry Reedy wrote: >> Environmental variables tend to be a pain on Windows and nigh unusable by >> beginners. Leaving that aside, I see these problems with trying to use one >> for IDLE's *current* debugger. >> >> pdb is universal, in the sense of working with any python run with actual or >> simulated stdin and stdout. IDLE's idb is specific to working with IDLE. So >> one could not set an EV to 'idlelib.idb.start' and leave it while switching >> between IDLE and console. > > Would it work for IDLE to set the environment variable for the child process? That’s exactly how I envision the environment variable would be used. If the process being debugged is run in an environment set up by the IDE, this would be the way for the IDE to communicate to the subprocess under debug, how it should behave in order to communicate properly with the debugger. > The user certainly should not need to be involved in that. Right. > That doesn't address the issue of setting up the communications > channel before breakpoint() is called, but allows the breakpointhook > that gets used to work with whatever has been arranged. Right again! I think setting up the communication channel is outside the scope of this PEP. Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553: Built-in debug()
On Sep 7, 2017, at 13:52, Terry Reedy wrote: > pdb.set_trace is a public and stable interface. IDLE's is private and likely > to be initially unstable. I can imagine that the function that I would want > to bind to sys.__breakpoint__ would be a bound method To be pedantic, you’re not supposed to touch sys.__breakpointhook__ although like sys.__excepthook__ and sys.__displayhook__ they are not enforced to be read-only. Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553: Built-in debug()
On Sep 7, 2017, at 12:09, Fernando Perez wrote: >> The PEP has an open issue regarding breakpoint() taking *args and **kws, >> which would just be passed through the call stack. It sounds like you’d be >> in favor of that enhancement. > > If you go witht the `(*a, **k)` pass-through API, would you have a special > keyword-only arg called 'header' or similar? That seems like a decent > compromise to support the feature with the builtin while allowing other > implementations to offer more features. In any case, +1 to a pass-through > API, as long as the built-in supports some kind of mechanism to help the user > get their bearings with "you're here" type messages. I don’t think I want to specify what goes in *args or **kws, I just want to pass them straight through. The user will have to have some understanding of what debugger they are using and what arguments their breakpoint hook allows. I’ll see what it takes to add `header` to pdb.set_trace(), but I’ll do that as a separate PR (i.e. not as part of this PEP). Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553: Built-in debug()
On Sep 7, 2017, at 14:25, Barry Warsaw wrote: > > I’ll see what it takes to add `header` to pdb.set_trace(), but I’ll do that > as a separate PR (i.e. not as part of this PEP). Turns out to be pretty easy. https://bugs.python.org/issue31389 https://github.com/python/cpython/pull/3438 Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 552: deterministic pycs
On Sep 7, 2017, at 16:58, Gregory P. Smith wrote: > Input from OS package distributors would be interesting. Would they use this? I suspect it won’t be that interesting to the Debian ecosystem, since we generate pyc files on package install. We do that because we can support multiple versions of Python installed simultaneously and we don’t know which versions are installed on the target machine. I suppose our stdlib package could ship pycs, but we don’t. Reproducible builds may still be interesting in other situations though, such as CI machines, but then SOURCE_DATE_EPOCH is probably good enough. -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553 V2 - builtin breakpoint() (was Re: PEP 553: Built-in debug())
On Sep 7, 2017, at 18:12, Nick Coghlan wrote: > > Related to this is the suggestion that we make the default > sys.breakpointhook() a no-op, so that accidentally checking in calls > to breakpoint() won' t hang CI systems. > > Then folks that wanted to use the functionality would set > "PYTHONBREAKPOINTHOOK=pdb:set_trace" I’d rather do it the other way ‘round because I want it to Just Work for the average developer, and maintainers of CI or production systems should be able to fairly easily tweak their environments to noop breakpoint(). Although maybe we want a shortcut for that, e.g. PYTHONBREAKPOINTHOOK=0 or some such. (Note, I’m still not sure it’s worth supporting the environment variable, but I am interesting in collecting the feedback on it.) -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553: Built-in debug()
On Sep 7, 2017, at 16:19, Terry Reedy wrote: > I think breakpoint() should have a db= parameter so one can select a debugger > in one removable line. The sys interface is more useful for IDEs to change > the default, possible with other args (like breakpoints and colors) bound to > the callable. I’m skeptical about that. I think any particular user is going to overwhelmingly use the same debugger, so having to repeat themselves every time they want to enter the debugger is going to get tedious fast. I know it would annoy *me* if I had to tell it to use pdb every time I wrote `breakpoint()`, and I’m almost never going to use anything else. I’m also not sure what useful semantics for `db` would be. E.g. what specifically would you set `db` to in order to invoke idle or tkdb (‘gdb’ would be an unfortunate name I think, given the popular existing GNU Debugger ;). I don’t even know what useful thing I’d set `db` to to mean “invoke pdb”. Please don’t say “well, pdb will still be the default so you don’t have to set it to anything” because that does kind of miss the point. I also want to keep breakpoint() as generic as possible. I think doing so allows it to be a nice API into whatever interesting and sophisticated implementations folks can think up underneath it. So *if* you came up with a cool thing that interpreted `db` in some special way, there should be a very low barrier to providing that to your users, e.g.: * pip install myfancymetadebugger * put a snippet which sets sys.breakpointhook in your $PYTHONSTARTUP file * profit! That second item could be replaced with export PYTHONBREAKPOINTHOOK=myfancymetadebugger.invoke (hmm, does that mean the envar is getting more interesting?) > Breakpoint() should pass on other args. I strongly believe it should pass through *all* args. > A somewhat separate point: the name breakpoint() is slightly misleading, > which has consequences if it is (improperly) called more than once. While > breakpoint() acts as a breakpoint, what it does (at least in the default pdb > case) is *initialize* and start a *new* debugger, possibly after an import. > Re-importing a module is no big deal. Replacing an existing debugger with a > *new* one, and tossing away all defined aliases and breakpoints and Bdb's > internal caches, is. It is likely not what most people would want or expect. > I think it more likely that people will call breakpoint() multiple times > than they would, for instance, call pdb.set_trace() multiple times. Multiple calls to pdb.set_trace() is fairly common in practice today, so I’m not terribly concerned about it. There’s nothing fundamentally different with multiple calls to breakpoint() today. If we care, we can provide a more efficient/different API and make that the default. The machinery in PEP 553 can easily support that, but doing it is outside the scope of the PEP. > With a gui debugger, having one window go and another appear might be > additionally annoying. If the first window is not immediately GCed, having > two windows would be confusing. Perhaps breakpoint() could be made a no-op > after the first call. Your sys.breakpointhook could easily implement that, with a much better user experience than what built-in breakpoint() could do anyway. Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
