[Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
Hi, I'm still working on a ssl module PEP for 3.7 [1], but it's probably not going to be finished before beta 1 deadline. I have a bunch of fixes and improvements for the ssl module in queue, most of them require OpenSSL 1.0.2 features. The features are also present and working properly since LibreSSL 2.5.3 If we agree to drop support for OpenSSL 0.9.8 and 1.0.1, then I can land bunch of useful goodies like proper hostname verification [2], proper fix for IP address in SNI TLS header [3], PEP 543 compatible Certificate and PrivateKey types (support loading certs and keys from file and memory) [4], and simplified cipher suite configuration [5]. I can finally clean up _ssl.c during the beta phase, too. OpenSSL 1.0.1 is out of support since December 2016, 0.9.8 since 2015. These versions haven't received any security updates for more than a year! All major Linux and BSD distributions have at least 1.0.2 [6]. The only relevant exception is Ubuntu 14.04 LTS, because Travis CI is running 14.04. PR 3562 [7] contains a PoC to compile a custom build of OpenSSL on Travis. Builds are cached. Regards, Christian [1] https://github.com/tiran/peps/blob/sslmodule37/pep-.txt [2] https://bugs.python.org/issue31399 [3] https://bugs.python.org/issue32185 [4] https://bugs.python.org/issue18369 [5] https://bugs.python.org/issue31429 [6] https://gist.github.com/tiran/c5409bbd60a5f082f654d967add8cc79 [7] https://github.com/python/cpython/pull/3462 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On Sat, 13 Jan 2018 13:54:33 +0100 Christian Heimes wrote: > > If we agree to drop support for OpenSSL 0.9.8 and 1.0.1, then I can land > bunch of useful goodies like proper hostname verification [2], proper > fix for IP address in SNI TLS header [3], PEP 543 compatible Certificate > and PrivateKey types (support loading certs and keys from file and > memory) [4], and simplified cipher suite configuration [5]. I can > finally clean up _ssl.c during the beta phase, too. Given the annoyance of supporting old OpenSSL versions, I'd say +1 to this. We'll have to deal with the complaints of users of Debian oldstable, CentOS 6 and RHEL 6, though. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On 2018-01-13 14:23, Antoine Pitrou wrote: > On Sat, 13 Jan 2018 13:54:33 +0100 > Christian Heimes wrote: >> >> If we agree to drop support for OpenSSL 0.9.8 and 1.0.1, then I can land >> bunch of useful goodies like proper hostname verification [2], proper >> fix for IP address in SNI TLS header [3], PEP 543 compatible Certificate >> and PrivateKey types (support loading certs and keys from file and >> memory) [4], and simplified cipher suite configuration [5]. I can >> finally clean up _ssl.c during the beta phase, too. > > Given the annoyance of supporting old OpenSSL versions, I'd say +1 to > this. > > We'll have to deal with the complaints of users of Debian oldstable, > CentOS 6 and RHEL 6, though. It's more of an issue for Travis CI. The Python 3.7-dev target won't have a functional ssl module. Travis either has to update their build base to 16.04, provide a custom build of OpenSSL, or all packages have to use a container. [1] Christian [1] https://github.com/travis-ci/travis-ci/issues/5821#issuecomment-214452987 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On Sat, 13 Jan 2018 15:49:21 +0100 Christian Heimes wrote: > On 2018-01-13 14:23, Antoine Pitrou wrote: > > On Sat, 13 Jan 2018 13:54:33 +0100 > > Christian Heimes wrote: > >> > >> If we agree to drop support for OpenSSL 0.9.8 and 1.0.1, then I can land > >> bunch of useful goodies like proper hostname verification [2], proper > >> fix for IP address in SNI TLS header [3], PEP 543 compatible Certificate > >> and PrivateKey types (support loading certs and keys from file and > >> memory) [4], and simplified cipher suite configuration [5]. I can > >> finally clean up _ssl.c during the beta phase, too. > > > > Given the annoyance of supporting old OpenSSL versions, I'd say +1 to > > this. > > > > We'll have to deal with the complaints of users of Debian oldstable, > > CentOS 6 and RHEL 6, though. > > It's more of an issue for Travis CI. The Python 3.7-dev target won't > have a functional ssl module. Travis either has to update their build > base to 16.04, provide a custom build of OpenSSL, or all packages have > to use a container. [1] That's Travis-CI's problem. And hopefully they'll migrate to Ubuntu 16.04 soon (it's almost 2 years old...). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Deprecate PEP 370 Per user site-packages directory?
Hi, PEP 370 [1] was my first PEP that got accepted. I created it exactly one decade and two days ago for Python 2.6 and 3.0. Back then we didn't have virtual environment support in Python. Ian Bicking had just started to create the virtualenv project a couple of months earlier. Fast forward 10 years... Nowadays Python has venv in the standard library. The user-specific site-packages directory is no longer that useful. I would even say it's causing more trouble than it's worth. For example it's common for system script to use "#!/usr/bin/python3" shebang without -s or -I option. I propose to deprecate the feature and remove it in Python 4.0. Regards, Christian [1] https://www.python.org/dev/peps/pep-0370/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Sat, Jan 13, 2018, at 12:06, Christian Heimes wrote: > Hi, > > PEP 370 [1] was my first PEP that got accepted. I created it exactly one > decade and two days ago for Python 2.6 and 3.0. Back then we didn't have > virtual environment support in Python. Ian Bicking had just started to > create the virtualenv project a couple of months earlier. > > Fast forward 10 years... > > Nowadays Python has venv in the standard library. The user-specific > site-packages directory is no longer that useful. I would even say it's > causing more trouble than it's worth. For example it's common for system > script to use "#!/usr/bin/python3" shebang without -s or -I option. > > I propose to deprecate the feature and remove it in Python 4.0. Where would pip install --user put packages, and how would one run scripts that require those packages? Right now these things Just Work; I've never had to learn how to use virtual environments. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On 2018-01-13 19:04, Random832 wrote: > On Sat, Jan 13, 2018, at 12:06, Christian Heimes wrote: >> Hi, >> >> PEP 370 [1] was my first PEP that got accepted. I created it exactly one >> decade and two days ago for Python 2.6 and 3.0. Back then we didn't have >> virtual environment support in Python. Ian Bicking had just started to >> create the virtualenv project a couple of months earlier. >> >> Fast forward 10 years... >> >> Nowadays Python has venv in the standard library. The user-specific >> site-packages directory is no longer that useful. I would even say it's >> causing more trouble than it's worth. For example it's common for system >> script to use "#!/usr/bin/python3" shebang without -s or -I option. >> >> I propose to deprecate the feature and remove it in Python 4.0. > > Where would pip install --user put packages, and how would one run scripts > that require those packages? Right now these things Just Work; I've never had > to learn how to use virtual environments. I see two option: 1) "pip install --user" is no longer supported. You have to learn how to use virtual envs. It's really easy: "python3 -m venv path; path/bin/pip install package". 2) "pip install --user" automatically creates or uses a custom virtual (~/.pip/virtualenv-$VERSION/) and links entry points to ~/.local/bin. Christian ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Sat, 13 Jan 2018 19:18:41 +0100 Christian Heimes wrote: > On 2018-01-13 19:04, Random832 wrote: > > On Sat, Jan 13, 2018, at 12:06, Christian Heimes wrote: > >> Hi, > >> > >> PEP 370 [1] was my first PEP that got accepted. I created it exactly one > >> decade and two days ago for Python 2.6 and 3.0. Back then we didn't have > >> virtual environment support in Python. Ian Bicking had just started to > >> create the virtualenv project a couple of months earlier. > >> > >> Fast forward 10 years... > >> > >> Nowadays Python has venv in the standard library. The user-specific > >> site-packages directory is no longer that useful. I would even say it's > >> causing more trouble than it's worth. For example it's common for system > >> script to use "#!/usr/bin/python3" shebang without -s or -I option. > >> > >> I propose to deprecate the feature and remove it in Python 4.0. > > > > Where would pip install --user put packages, and how would one run scripts > > that require those packages? Right now these things Just Work; I've never > > had to learn how to use virtual environments. > > I see two option: > > 1) "pip install --user" is no longer supported. You have to learn how to > use virtual envs. It's really easy: "python3 -m venv path; path/bin/pip > install package". > 2) "pip install --user" automatically creates or uses a custom virtual > (~/.pip/virtualenv-$VERSION/) and links entry points to ~/.local/bin. Option 2 doesn't work, since the installed package then isn't known to the system Python. I'm not sure user site-packages adds a lot of complexity to Python, so I don't think it's worth breaking some people's usage. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On 13.1.2018 18:06, Christian Heimes wrote: Nowadays Python has venv in the standard library. The user-specific site-packages directory is no longer that useful. I would even say it's causing more trouble than it's worth. For example it's common for system script to use "#!/usr/bin/python3" shebang without -s or -I option. While I consider venvs easy and cool, this just moves the barrier for the users a little bit higher. We (Fedora Python SIG) are fighting users that run `sudo pip install` all the time (because the Interwebz are full of such instructions). The users might be willing to listen to "please, don't use pip with sudo, use --user instead". However, if you tell them "learn how to use a venv", they'll just stick with sudo. > I propose to deprecate the feature and remove it in Python 4.0. We would very much like to see --user the default rather than having it removed. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
Hello, On Sat, 13 Jan 2018 19:18:41 +0100 Christian Heimes wrote: [] > >> Nowadays Python has venv in the standard library. The user-specific > >> site-packages directory is no longer that useful. I would even say > >> it's causing more trouble than it's worth. For example it's common > >> for system script to use "#!/usr/bin/python3" shebang without -s > >> or -I option. > >> > >> I propose to deprecate the feature and remove it in Python 4.0. > > > > Where would pip install --user put packages, and how would one run > > scripts that require those packages? Right now these things Just > > Work; I've never had to learn how to use virtual environments. > > I see two option: > > 1) "pip install --user" is no longer supported. You have to learn how > to use virtual envs. It's really easy: "python3 -m venv path; > path/bin/pip install package". Easy for whom? C, Ruby, JavaScript users, random grandmas and grandpas? Please don't make innocent hate Python, and don't make developers who chose Python to develop software hate it for impossibility to provide decent user support for their software. -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
Hi! On Sat, Jan 13, 2018 at 06:06:16PM +0100, Christian Heimes wrote: > Hi, > > PEP 370 [1] was my first PEP that got accepted. I created it exactly one > decade and two days ago for Python 2.6 and 3.0. Back then we didn't have > virtual environment support in Python. Ian Bicking had just started to > create the virtualenv project a couple of months earlier. > > Fast forward 10 years... > > Nowadays Python has venv in the standard library. The user-specific > site-packages directory is no longer that useful. Can I disagree? > I would even say it's > causing more trouble than it's worth. For example it's common for system > script to use "#!/usr/bin/python3" shebang without -s or -I option. System scripts are run under user root which seldom has user-specific site-packages so why worry? > I propose to deprecate the feature and remove it in Python 4.0. > > Regards, > Christian > > [1] https://www.python.org/dev/peps/pep-0370/ Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Sat, Jan 13, 2018 at 07:18:41PM +0100, Christian Heimes wrote: > On 2018-01-13 19:04, Random832 wrote: > > On Sat, Jan 13, 2018, at 12:06, Christian Heimes wrote: > >> Hi, > >> > >> PEP 370 [1] was my first PEP that got accepted. I created it exactly one > >> decade and two days ago for Python 2.6 and 3.0. Back then we didn't have > >> virtual environment support in Python. Ian Bicking had just started to > >> create the virtualenv project a couple of months earlier. > >> > >> Fast forward 10 years... > >> > >> Nowadays Python has venv in the standard library. The user-specific > >> site-packages directory is no longer that useful. I would even say it's > >> causing more trouble than it's worth. For example it's common for system > >> script to use "#!/usr/bin/python3" shebang without -s or -I option. > >> > >> I propose to deprecate the feature and remove it in Python 4.0. > > > > Where would pip install --user put packages, and how would one run scripts > > that require those packages? Right now these things Just Work; I've never > > had to learn how to use virtual environments. > > I see two option: > > 1) "pip install --user" is no longer supported. You have to learn how to > use virtual envs. It's really easy: "python3 -m venv path; path/bin/pip > install package". I've learned virtual envs and use them every day. I also use ``pip install --user``. Different use cases. Virtual envs are for development, ``pip install --user`` for deployment. > Christian Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On 2018-01-13 20:08, Oleg Broytman wrote: > Hi! > > On Sat, Jan 13, 2018 at 06:06:16PM +0100, Christian Heimes > wrote: >> Hi, >> >> PEP 370 [1] was my first PEP that got accepted. I created it exactly one >> decade and two days ago for Python 2.6 and 3.0. Back then we didn't have >> virtual environment support in Python. Ian Bicking had just started to >> create the virtualenv project a couple of months earlier. >> >> Fast forward 10 years... >> >> Nowadays Python has venv in the standard library. The user-specific >> site-packages directory is no longer that useful. > >Can I disagree? > >> I would even say it's >> causing more trouble than it's worth. For example it's common for system >> script to use "#!/usr/bin/python3" shebang without -s or -I option. > >System scripts are run under user root which seldom has user-specific > site-packages so why worry? You'd be surprised how many tools and programs are using Python these days. You can easily break important user programs by installing a package with --user. Christian ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On Sat, Jan 13, 2018, 05:24 Antoine Pitrou, wrote: > On Sat, 13 Jan 2018 13:54:33 +0100 > Christian Heimes wrote: > > > > If we agree to drop support for OpenSSL 0.9.8 and 1.0.1, then I can land > > bunch of useful goodies like proper hostname verification [2], proper > > fix for IP address in SNI TLS header [3], PEP 543 compatible Certificate > > and PrivateKey types (support loading certs and keys from file and > > memory) [4], and simplified cipher suite configuration [5]. I can > > finally clean up _ssl.c during the beta phase, too. > > Given the annoyance of supporting old OpenSSL versions, I'd say +1 to > this. > +1 from me as well for the improved security. -Brett > We'll have to deal with the complaints of users of Debian oldstable, > CentOS 6 and RHEL 6, though. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Sat, Jan 13, 2018, 11:13 Miro Hrončok, wrote: > On 13.1.2018 18:06, Christian Heimes wrote: > > Nowadays Python has venv in the standard library. The user-specific > > site-packages directory is no longer that useful. I would even say it's > > causing more trouble than it's worth. For example it's common for system > > script to use "#!/usr/bin/python3" shebang without -s or -I option. > > While I consider venvs easy and cool, this just moves the barrier for > the users a little bit higher. We (Fedora Python SIG) are fighting users > that run `sudo pip install` all the time (because the Interwebz are full > of such instructions). The users might be willing to listen to "please, > don't use pip with sudo, use --user instead". However, if you tell them > "learn how to use a venv", they'll just stick with sudo. > > > I propose to deprecate the feature and remove it in Python 4.0. > > We would very much like to see --user the default rather than having it > removed. > I concur with Miro. On VS Code we rely on people installing the e.g. linter of their choosing. We have been moving people away from sudo installs to user installs to minimize polluting the system Python (and we are doing what we can to promote venvs by using them automatically when present, but we can only do so much). Basically the only way I see it being reasonable to drop user installs is if we move entirely to venvs for installs and I think that's probably too radical to work. -Brett > > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On 2018-01-13 19:57, Antoine Pitrou wrote: > On Sat, 13 Jan 2018 19:18:41 +0100 > Christian Heimes wrote: >> On 2018-01-13 19:04, Random832 wrote: >>> On Sat, Jan 13, 2018, at 12:06, Christian Heimes wrote: Hi, PEP 370 [1] was my first PEP that got accepted. I created it exactly one decade and two days ago for Python 2.6 and 3.0. Back then we didn't have virtual environment support in Python. Ian Bicking had just started to create the virtualenv project a couple of months earlier. Fast forward 10 years... Nowadays Python has venv in the standard library. The user-specific site-packages directory is no longer that useful. I would even say it's causing more trouble than it's worth. For example it's common for system script to use "#!/usr/bin/python3" shebang without -s or -I option. I propose to deprecate the feature and remove it in Python 4.0. >>> >>> Where would pip install --user put packages, and how would one run scripts >>> that require those packages? Right now these things Just Work; I've never >>> had to learn how to use virtual environments. >> >> I see two option: >> >> 1) "pip install --user" is no longer supported. You have to learn how to >> use virtual envs. It's really easy: "python3 -m venv path; path/bin/pip >> install package". >> 2) "pip install --user" automatically creates or uses a custom virtual >> (~/.pip/virtualenv-$VERSION/) and links entry points to ~/.local/bin. > > Option 2 doesn't work, since the installed package then isn't known to > the system Python. I see that as a benefit. User installed packages will no longer be able to break system-wide programs. These days a lot of packages are using setuptools' entry points to create console scripts. Entry point have no option to create a console script with -s or -I flag. On my system, only 40 out of 360 scripts in /usr/bin have -s or -I. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Sat, Jan 13, 2018 at 09:00:07PM +0100, Christian Heimes wrote: > On 2018-01-13 20:08, Oleg Broytman wrote: > > Hi! > > > > On Sat, Jan 13, 2018 at 06:06:16PM +0100, Christian Heimes > > wrote: > >> Hi, > >> > >> PEP 370 [1] was my first PEP that got accepted. I created it exactly one > >> decade and two days ago for Python 2.6 and 3.0. Back then we didn't have > >> virtual environment support in Python. Ian Bicking had just started to > >> create the virtualenv project a couple of months earlier. > >> > >> Fast forward 10 years... > >> > >> Nowadays Python has venv in the standard library. The user-specific > >> site-packages directory is no longer that useful. > > > >Can I disagree? > > > >> I would even say it's > >> causing more trouble than it's worth. For example it's common for system > >> script to use "#!/usr/bin/python3" shebang without -s or -I option. > > > >System scripts are run under user root which seldom has user-specific > > site-packages so why worry? > > You'd be surprised how many tools and programs are using Python these > days. Certainly not. I wrote or helped to write a lot of them myself. :-) > You can easily break important user programs by installing a > package with --user. Under root? Probably. Then don't do that -- or do not allow system Python to import user-specific site-packages (i.e., distinguish system Python from normal Python running under user root). But for a non-root user user-specific site-packages is quite a convenient thing. Please don't remove it. > Christian Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Sat, 13 Jan 2018 21:06:19 +0100 Christian Heimes wrote: > >> > >> I see two option: > >> > >> 1) "pip install --user" is no longer supported. You have to learn how to > >> use virtual envs. It's really easy: "python3 -m venv path; path/bin/pip > >> install package". > >> 2) "pip install --user" automatically creates or uses a custom virtual > >> (~/.pip/virtualenv-$VERSION/) and links entry points to ~/.local/bin. > > > > Option 2 doesn't work, since the installed package then isn't known to > > the system Python. > > I see that as a benefit. User installed packages will no longer be able > to break system-wide programs. I don't know if it's better or worse. I'm just saying it's not a migration path from user site-packages since it doesn't have the same semantics. > These days a lot of packages are using setuptools' entry points to > create console scripts. Entry point have no option to create a console > script with -s or -I flag. Perhaps that should be fixed. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
I’m generally +1, though I don’t see an easy migration path. Moving to a node.js project model might be feasible, but I suspect our real solution will end up having to be ensuring use of -s where it’s needed. Top-posted from my Windows phone From: Christian Heimes Sent: Sunday, January 14, 2018 4:09 To: python-dev@python.org Subject: [Python-Dev] Deprecate PEP 370 Per user site-packages directory? Hi, PEP 370 [1] was my first PEP that got accepted. I created it exactly one decade and two days ago for Python 2.6 and 3.0. Back then we didn't have virtual environment support in Python. Ian Bicking had just started to create the virtualenv project a couple of months earlier. Fast forward 10 years... Nowadays Python has venv in the standard library. The user-specific site-packages directory is no longer that useful. I would even say it's causing more trouble than it's worth. For example it's common for system script to use "#!/usr/bin/python3" shebang without -s or -I option. I propose to deprecate the feature and remove it in Python 4.0. Regards, Christian [1] https://www.python.org/dev/peps/pep-0370/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
Hi all, I'm a user who decided to read the mailing list and respond to argue against and maybe stop things I don't think will help users as much as you think.I think deprecating user site-packages is such a change. That venvs exist doesn't mean most or even many people use them, even though I'm sure you and everyone you know does. I couldn't find usage statistics on short notice, do you have any? As far as I understand it site-packages is the default location for all packages that are installed and as long as the packages that are installed aren't causing a conflict, there is no problem. I've never had a problem in six years of using python. So I'd really like a more detailed break down of the troubles the existence of site packages causes and of the up- and downsides removing it would bring, before anything is decided. Another point I'd like to raise is that even though it's a good idea to isolate programs with venvs for stability, development and sometimes maybe security, idk, the idea of having a shared pool of packages has benefits too, namely code reuse and their availability, for example when you're offline. Also I don't really know why you would think it's necessary to force every user into venvs. The applications and users that do need their environments are free to use one after all. Regards, Max ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On 2018-01-13 21:02, Brett Cannon wrote: > +1 from me as well for the improved security. Thanks, Brett! How should we handle CPython's Travis CI tests? The 14.04 boxes have OpenSSL 1.0.1. To the best of my knowledge, Travis doesn't offer 16.04. We could either move to container-based testing with a 16.04 container, which would give us 1.0.2 Or we could compile our own copy of OpenSSL with my multissl builder and use some rpath magic. In order to test all new features, Ubuntu doesn't cut it. Even current snapshot of Ubuntu doesn't contain OpenSSL 1.1. Debian Stretch or Fedora would do the trick, though. Maybe Barry's work on official test container could leveraged testing? Regards, Christian ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Sat, Jan 13, 2018 at 09:00:07PM +0100, Christian Heimes wrote: > You'd be surprised how many tools and programs are using Python these > days. You can easily break important user programs by installing a > package with --user. Or by writing a Python script and innocently giving it the same name as a system module. On the tutor@ and python-list@ mailing lists, it is very frequent for users to accidently break Python by accidently shadowing an installed module, e.g. installing xlrd and then calling their own script "xlrd.py". But I've never come across somebody breaking anything by installing a package with --user. I presume it must happen, but I would be surprised if it happens often enough to justify removing what is otherwise a useful piece of functionality. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On Sat, Jan 13, 2018 at 02:23:19PM +0100, Antoine Pitrou wrote: > On Sat, 13 Jan 2018 13:54:33 +0100 > Christian Heimes wrote: > > > > If we agree to drop support for OpenSSL 0.9.8 and 1.0.1, then I can land > > bunch of useful goodies like proper hostname verification [2], proper > > fix for IP address in SNI TLS header [3], PEP 543 compatible Certificate > > and PrivateKey types (support loading certs and keys from file and > > memory) [4], and simplified cipher suite configuration [5]. I can > > finally clean up _ssl.c during the beta phase, too. > > Given the annoyance of supporting old OpenSSL versions, I'd say +1 to > this. > > We'll have to deal with the complaints of users of Debian oldstable, > CentOS 6 and RHEL 6, though. It will probably be more work for Christian, but is it reasonable to keep support for the older versions of OpenSSL, but make the useful goodies conditional on a newer version? -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
I would suggest throwing this to -ideas, rather than just keeping it in -dev as there is a much wider community of users and usecases in -ideas. ... and -ideas will shoot it down because user installs are too useful. It is also my understanding that it is the desire of PyPA to eventually have pip default to --user when run outside of a virtualenv (to mitigate people running sudo pip). Eliminating user installs would change the situation from one where, with pip install --user, one can recoverably break their system to one, with sudo pip install, one can un-recoverably break their system. > -Original Message- > From: Python-Dev [mailto:python-dev-bounces+tritium- > list=sdamon@python.org] On Behalf Of Christian Heimes > Sent: Saturday, January 13, 2018 12:06 PM > To: python-dev@python.org > Subject: [Python-Dev] Deprecate PEP 370 Per user site-packages directory? > > Hi, > > PEP 370 [1] was my first PEP that got accepted. I created it exactly one > decade and two days ago for Python 2.6 and 3.0. Back then we didn't have > virtual environment support in Python. Ian Bicking had just started to > create the virtualenv project a couple of months earlier. > > Fast forward 10 years... > > Nowadays Python has venv in the standard library. The user-specific > site-packages directory is no longer that useful. I would even say it's > causing more trouble than it's worth. For example it's common for system > script to use "#!/usr/bin/python3" shebang without -s or -I option. > > I propose to deprecate the feature and remove it in Python 4.0. > > Regards, > Christian > > [1] https://www.python.org/dev/peps/pep-0370/ > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium- > list%40sdamon.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On Sat, Jan 13, 2018 at 4:34 PM Steven D'Aprano wrote: > On Sat, Jan 13, 2018 at 02:23:19PM +0100, Antoine Pitrou wrote: > > On Sat, 13 Jan 2018 13:54:33 +0100 > > Christian Heimes wrote: > > > > > > If we agree to drop support for OpenSSL 0.9.8 and 1.0.1, then I can > land > > > bunch of useful goodies like proper hostname verification [2], proper > > > fix for IP address in SNI TLS header [3], PEP 543 compatible > Certificate > > > and PrivateKey types (support loading certs and keys from file and > > > memory) [4], and simplified cipher suite configuration [5]. I can > > > finally clean up _ssl.c during the beta phase, too. > > > > Given the annoyance of supporting old OpenSSL versions, I'd say +1 to > > this. > > > > We'll have to deal with the complaints of users of Debian oldstable, > > CentOS 6 and RHEL 6, though. > > It will probably be more work for Christian, but is it reasonable to > keep support for the older versions of OpenSSL, but make the useful > goodies conditional on a newer version? > I don't think it is worth spending our limited engineering time supporting an unsupported library version. Leave that burden to stale distro maintainers who continue to choose dangerously stale software versions if they ironically want to use something as modern as 3.7 on top of an ancient set of libraries. +1 from me for requiring OpenSSL >= 1.0.2 in Python 3.7. -gps ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On Sat, Jan 13, 2018, 14:45 Christian Heimes, wrote: > On 2018-01-13 21:02, Brett Cannon wrote: > > +1 from me as well for the improved security. > > Thanks, Brett! > > How should we handle CPython's Travis CI tests? The 14.04 boxes have > OpenSSL 1.0.1. To the best of my knowledge, Travis doesn't offer 16.04. > We could either move to container-based testing with a 16.04 container, > which would give us 1.0.2 Or we could compile our own copy of OpenSSL > with my multissl builder and use some rpath magic. > > In order to test all new features, Ubuntu doesn't cut it. Even current > snapshot of Ubuntu doesn't contain OpenSSL 1.1. Debian Stretch or Fedora > would do the trick, though. > > Maybe Barry's work on official test container could leveraged testing? > My guess is we either move to containers on Travis, see if we can manually install -- through apt or something -- a newer version of OpenSSL, or we look at alternative CI options. -Brett > Regards, > Christian > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Sat, Jan 13, 2018, 17:27 Alex Walters, wrote: > I would suggest throwing this to -ideas, rather than just keeping it in > -dev > as there is a much wider community of users and usecases in -ideas. > > ... and -ideas will shoot it down because user installs are too useful. It > is also my understanding that it is the desire of PyPA to eventually have > pip default to --user when run outside of a virtualenv (to mitigate people > running sudo pip). Eliminating user installs would change the situation > from one where, with pip install --user, one can recoverably break their > system to one, with sudo pip install, one can un-recoverably break their > system. > Which suggests that doing away with user installs would mean making it so interpreter installs don't break your system, which would require Linux and distros and macOS to not publicly expose their system Python installs so people can't accidentally break them. I think if that were to occur then you could consider dropping user installs as people would be installing into their own interpreter. But until then I think the attraction/lack of knowledge for people will be high enough that user installs will be the most pragmatic solution we have. -Brett > > -Original Message- > > From: Python-Dev [mailto:python-dev-bounces+tritium- > > list=sdamon@python.org] On Behalf Of Christian Heimes > > Sent: Saturday, January 13, 2018 12:06 PM > > To: python-dev@python.org > > Subject: [Python-Dev] Deprecate PEP 370 Per user site-packages directory? > > > > Hi, > > > > PEP 370 [1] was my first PEP that got accepted. I created it exactly one > > decade and two days ago for Python 2.6 and 3.0. Back then we didn't have > > virtual environment support in Python. Ian Bicking had just started to > > create the virtualenv project a couple of months earlier. > > > > Fast forward 10 years... > > > > Nowadays Python has venv in the standard library. The user-specific > > site-packages directory is no longer that useful. I would even say it's > > causing more trouble than it's worth. For example it's common for system > > script to use "#!/usr/bin/python3" shebang without -s or -I option. > > > > I propose to deprecate the feature and remove it in Python 4.0. > > > > Regards, > > Christian > > > > [1] https://www.python.org/dev/peps/pep-0370/ > > > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium- > > list%40sdamon.com > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Jan 13, 2018, at 11:12, Miro Hrončok wrote: > > We would very much like to see --user the default rather than having it > removed. Very much +1. In Debian/Ubuntu we’ve carried patches to do exactly that for years, and I think our users have been very happy about it. -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Jan 13, 2018, at 12:06, Christian Heimes wrote: > These days a lot of packages are using setuptools' entry points to > create console scripts. Entry point have no option to create a console > script with -s or -I flag. On my system, only 40 out of 360 scripts in > /usr/bin have -s or -I. -I should be the default for system scripts; it’s not on Debian/Ubuntu though unfortunately. -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
One thing to note is that if getting Travis working with Python 3.7 is a pain, a huge number of libraries on PyPI probably just won't test against Python 3.7, which is not a great situation to be in. It's probably worth contacting Travis to give them a head's up and see how likely it is that they'll be able to support Python 3.7 if it requires a newer version of these libraries. On January 14, 2018 2:16:53 AM UTC, Brett Cannon wrote: >On Sat, Jan 13, 2018, 14:45 Christian Heimes, >wrote: > >> On 2018-01-13 21:02, Brett Cannon wrote: >> > +1 from me as well for the improved security. >> >> Thanks, Brett! >> >> How should we handle CPython's Travis CI tests? The 14.04 boxes have >> OpenSSL 1.0.1. To the best of my knowledge, Travis doesn't offer >16.04. >> We could either move to container-based testing with a 16.04 >container, >> which would give us 1.0.2 Or we could compile our own copy of OpenSSL >> with my multissl builder and use some rpath magic. >> >> In order to test all new features, Ubuntu doesn't cut it. Even >current >> snapshot of Ubuntu doesn't contain OpenSSL 1.1. Debian Stretch or >Fedora >> would do the trick, though. >> >> Maybe Barry's work on official test container could leveraged >testing? >> > >My guess is we either move to containers on Travis, see if we can >manually >install -- through apt or something -- a newer version of OpenSSL, or >we >look at alternative CI options. > >-Brett > > >> Regards, >> Christian >> ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.7: Require OpenSSL >=1.0.2 / LibreSSL >= 2.5.3
On Sun, Jan 14, 2018 at 02:16:53AM +, Brett Cannon wrote: > My guess is we either move to containers on Travis, see if we can manually > install -- through apt or something -- a newer version of OpenSSL OpenSSL 1.0.2 cannot be installed with apt on Trusty but I think it can be compiled from sources. > -Brett Oleg. -- Oleg Broytmanhttp://phdru.name/p...@phdru.name Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecate PEP 370 Per user site-packages directory?
On Jan 13, 2018 9:08 AM, "Christian Heimes" wrote: Hi, PEP 370 [1] was my first PEP that got accepted. I created it exactly one decade and two days ago for Python 2.6 and 3.0. I didn't know I had you to thank for this! Thanks Christian! This is one of the best features of the python software packaging ecosystem! I almost exclusively install into user site packages these days. It lets me pull in the latest version of software when I want it for everyday use and revert to what my system shipped with if the updates break something. It's let me I install libraries ported to python3 before my distro got stopping to packaging the updates. It's let me perform an install when I want to test my packages as my users might be using it without touching the system dirs. It's been a godsend! Fast forward 10 years... Nowadays Python has venv in the standard library. The user-specific site-packages directory is no longer that useful. I would even say it's causing more trouble than it's worth. For example it's common for system script to use "#!/usr/bin/python3" shebang without -s or -I option. With great power comes great responsibility... Sure, installing something into user site packages can break system scripts. But it can also fix them. I can recall breaking system scripts twice by installing something into user site packages (both times, the tracebacks rapidly lead me to the reason that the scripts were failing). As a counter point to that I can recall *fixing* problems in system scripts by installing newer libraries into site packages twice in the last two months. (I've also fixed system software by installing into user and then modifying that version but I do that less frequently... Perhaps only a couple times a year...) Removing the user site packages also doesn't prevent people from making local changes that break system scripts (removing the pre-configuration of user site packages does not stop honoring usage of PYTHONPATH); it only makes people work a little harder to place their overridden packages into a location that python will find and leads to nonstandard locations for these overrides. This will make it harder for people to troubleshoot the problems other people may be having. Instead of asking "do you have any libraries in .local in your tracebacks?" as an easy first troubleshooting step. Without the user site packages standard we'll be back to trying to determine which directories are official for the user's install and then finding any local directories that their site may have defined for overrides I propose to deprecate the feature and remove it in Python 4.0. Although I don't like the idea of system scripts adding -s and -l because it prevents me from fixing them for my own use by installing just a newer or modified library into user site packages (similar to how c programs can use overridden libraries via ld_library_path), it seems that if you want to prevent users from choosing to use their own libraries with system scripts, the right thing to do is to get changes to allow adding those to setuptools and distutils. Those flags will do a much more thorough job of preventing this usage than removing user site packages can. -Toshio ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
