Would it be better to have an (os, os_release, arch) to [site_packages_dirs] map within CPython or are the patches too varied to be replaced by a centralized map and a runtime conditional?
For development convenience, having writeable site-packages dirs before unwriteable site-packages is easy; But for security and production deployment, site-packages maybe shouldn't be writeable at all by default because that's doing it wrong? On Wed, Feb 24, 2021, 15:34 Christian Heimes <christ...@python.org> wrote: > On 24/02/2021 20.03, Christian Heimes wrote: > > On 24/02/2021 19.17, Steve Dower wrote: > >> On 2/24/2021 4:26 PM, Christian Heimes wrote: > >>> On 24/02/2021 15.16, Random832 wrote: > >>>> On Wed, Feb 24, 2021, at 06:27, Christian Heimes wrote: > >>>>> Separate directories don't prevent clashes and system breakage. But > >>>>> they > >>>>> provide an easy way to *recover* from a broken system. > >>>> > >>>> I think it could be turned into a way to prevent them by A) having > >>>> site-packages always take precedence over dist-packages [i believe > >>>> this is already the case] in normal usage and B) providing an option > >>>> to the interpreter, used by system scripts, to exclude site-packages > >>>> entirely from the path. > >>>> > >>>> Basically, site-packages would effectively be layered on top of "Lib > >>>> + dist-packages" in a similar way to how a venv is layered on top of > >>>> the main python installation - the inverse of the suggestion someone > >>>> else in the thread made for the system python to be a venv. This > >>>> wouldn't *exactly* be a venv because it wouldn't imply the other > >>>> things that entering a venv does such as "python" [and script names > >>>> such as pip] being an alias for the correct version of python, but it > >>>> would provide the same kind of one-way isolation, whereby the "system > >>>> environment" can influence the "normal environment" and not > >>>> vice-versa, in the same way that packages installed in the main > >>>> installation affect a venv [unless system-site-packages is disabled] > >>>> but the venv obviously has no effect on the main installati > >> on. > >>> > >>> Yes, you are describing one major aspect of my idea for a system Python > >>> interpreter. I'm happy to read that other users are coming to similar > >>> conclusions. Instead of an option I'd create a new executable to lock > >>> down additional things (e.g. isolated mode, code verification hook). A > >>> separate executable would also allow distros to provide a stripped down > >>> interpreter that does not cause bad user experience. > >> > >> I mean, this is _precisely_ what PEP 370 defines (including the "-s" > >> option and PYTHONNOUSERSITE env variable to provide that one-way > >> isolation). > >> > >> Is the problem that pip doesn't use it by default? Or that the distros > >> went and made patches for the runtime rather than patching pip? (For > >> Windows installs from the Store, where even admin rights can't do an > >> all-users package install, I added a new config file location for pip to > >> change this default, but a patch would also have worked.) > >> > >> Maybe we need an easier way to patch the location of user site packages? > >> I also had to do this for the Store install on Windows, and it's a > >> little bit of a hack... but maybe having an official recommendation > >> would help encourage distributors to use the mechanism? > > > > (sorry for terse reply, I'm heading out.) > > > > Linux distros want an additional layer so the can differentiate between > > Python packages installed by their package manager and Python packages > > installed with "sudo pip install". > > > > Python has two default site-package directories (from highest to lowest > > precedence): > > > > 1. ~/.local/lib/python3.9/site-packages > > 2. /usr/lib/python3.9/site-packages > > > > (1) is defined by site.getusersitepackages(), (2) site.getsitepackages() > > > > Linux distro have additional site-package directories. My Fedora system > > X86_64 system with multiarch has three additional candidates. > > > > 1. ~/.local/lib/python3.9/site-packages > > 2. /usr/local/lib64/python3.9/site-packages > > 3. /usr/local/lib/python3.9/site-packages > > 4. /usr/lib64/python3.9/site-packages > > 5. /usr/lib/python3.9/site-packages > > > > The "lib" directories are for pure Python packages whiel the "lib64" > > directories contain X86_64 native code extensions (aka shared > > libraries). The "/usr/lib*" directories are used for distro packages. > > The downstream patch [1] ensures that "sudo pip install" installs > > packages into "/usr/local/lib". AFAIK "/usr/local/lib64" is not used. > > > > Debian has a similar mechanism to provide > > "/usr/lib/python3/dist-packages" [2]. > > > > A hypothetical /usr/bin/system-python3 interpreter would only use (4) > > and (5) as site-package directories. A user-facing /usr/bin/python3 > > would use (1) to (5). > > > > Christian > > > > [1] > > > https://src.fedoraproject.org/rpms/python3.9/blob/rawhide/f/00251-change-user-install-location.patch > > [2] > > > https://www.debian.org/doc/packaging-manuals/python-policy/ch-python.html > > PS: Fedora patch [1] uses -s / PYTHONNOUSERSITE to exclude > "/usr/local/lib*" site-package directories. > > $ python3 -c 'import site; print(site.getsitepackages())' > ['/usr/local/lib64/python3.9/site-packages', > '/usr/local/lib/python3.9/site-packages', > '/usr/lib64/python3.9/site-packages', '/usr/lib/python3.9/site-packages'] > > $ python3 -s -c 'import site; print(site.getsitepackages())' > ['/usr/lib64/python3.9/site-packages', '/usr/lib/python3.9/site-packages'] > > > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/EKYM66AZYDSRLTD5O4UOGC7FXKBB2VVL/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GTSWQYNDLMDF5CH2WV2CTLD7QOZBWSVX/ Code of Conduct: http://python.org/psf/codeofconduct/
