Re: "Don't install on the system Python"
On Sat, 30 Nov 2019 20:42:21 -0800 (PST) John Ladasky wrote: > Long-time Ubuntu user here. > > For years, I've read warnings about not installing one's personal > stack of Python modules on top of the system Python. It is possible > to corrupt the OS, or so I've gathered. > This is nonsense as you presumably have no permission to change anything Python related in /usr. The only possiblity I can imagine is that you somehow screw up your personal Python related setting in your home directory tree. But I have never (in the short period of time I've been using Python) encountered anything like this. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: "Don't install on the system Python"
On 01Dec2019 09:29, Manfred Lotz wrote: On Sat, 30 Nov 2019 20:42:21 -0800 (PST) John Ladasky wrote: For years, I've read warnings about not installing one's personal stack of Python modules on top of the system Python. It is possible to corrupt the OS, or so I've gathered. This is nonsense as you presumably have no permission to change anything Python related in /usr. The only possiblity I can imagine is that you somehow screw up your personal Python related setting in your home directory tree. But I have never (in the short period of time I've been using Python) encountered anything like this. What is to be avoided: Some people run pip as root and install in the vendor/supplier controlled space. This can lead to various problems, as it can conflict with or simply vary the system installed packages. Provided the OP is using pip in its (modern default) "install in my home directory" mode, they should be fine. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: ModuleNotFoundError with click module
Tim Johnson wrote: > Using linux ubuntu 16.04 with bash shell. > Am retired python programmer, but not terribly current. > I have moderate bash experience. > > When trying to install pgadmin4 via apt I get the following error > traceback when pgadmin4 is invoked: > > Traceback (most recent call last): > File "setup.py", line 17, in > from pgadmin.model import db, User, Version, ServerGroup, Server, \ > File "/usr/share/pgadmin4/web/pgadmin/__init__.py", line 19, in > from flask import Flask, abort, request, current_app, session, url_for > File "/usr/local/lib/python3.7/site-packages/flask/__init__.py", line > 21, in > from .app import Flask > File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 34, > in > from . import cli > File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 25, in > > import click > ModuleNotFoundError: No module named 'click' > > > If I invoke python3 (/usr/local/bin/python3), version 3.7.2 and invoke > >>> import click > click is imported successfully. > > In this invocation, sys.path is: > ['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', > '/usr/local/lib/python3.7/lib-dynload', > '/home/tim/.local/lib/python3.7/site-packages', > '/usr/local/lib/python3.7/site-packages'] > > $PYTHONPATH is empty when the bash shell is invoked > > $PATH as follows: > /home/tim/bin:/home/tim/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin > > click.py can be found at > /usr/local/lib/python3.7/site-packages/pipenv/patched/piptools/ > in turn click.py imports click, presumably as the package, > which appears to be at > /usr/local/lib/python3.7/site-packages/pipenv/vendor/click > > Any number of settings of PYTHONPATH to the various paths above has > failed to resolve the ModuleNotFoundError > Same issues with attempting install from a virtual environment. > > Any help will be appreciated. > thanks > tim > I'm too lazy to look into the details of your paths -- I'd just make sure that click is installed with the same interpreter and user as pgadmin4, e. g. globally $ sudo /usr/local/bin/python3 -m pip install click $ sudo /usr/local/bin/python3 path/to/setup.py install # or whatever it takes to install pgadmin4 or (better) in a virtual environment $ /usr/local/bin/python3 -m venv whatever $ cd whatever $ . bin/activate $ pip install click $ python path/to/setup.py -- https://mail.python.org/mailman/listinfo/python-list
Re: "Don't install on the system Python"
On Sunday, December 1, 2019 at 12:47:43 AM UTC-8, Cameron Simpson wrote: > On 01Dec2019 09:29, Manfred Lotz <[email protected]> wrote: > >On Sat, 30 Nov 2019 20:42:21 -0800 (PST) > >John Ladasky <[email protected]> wrote: > >> For years, I've read warnings about not installing one's personal > >> stack of Python modules on top of the system Python. It is possible > >> to corrupt the OS, or so I've gathered. > > > >This is nonsense as you presumably have no permission to change > >anything Python related in /usr. > > > >The only possiblity I can imagine is that you somehow screw up your > >personal Python related setting in your home directory tree. But I have > >never (in the short period of time I've been using Python) encountered > >anything like this. > > What is to be avoided: Some people run pip as root and install in the > vendor/supplier controlled space. This can lead to various problems, as > it can conflict with or simply vary the system installed packages. > > Provided the OP is using pip in its (modern default) "install in my home > directory" mode, they should be fine. > > Cheers, > Cameron Simpson <[email protected]> The only thing I must install with pip is tensorflow-gpu. For everything else, I make use of the Ubuntu repositories. The Synaptic package manager installs packages (including Python modules) for all user accounts at the same time, which I like. When I installed tensorflow-gpu using pip, I was in fact frustrated because I couldn't figure out how to deploy it across multiple user accounts at one time. I ended up installing it three times, once in each account. You're suggesting that's actually preferred, at least when pip is performing the installation. OK, I will endure the repetition. -- https://mail.python.org/mailman/listinfo/python-list
Re: "Don't install on the system Python"
On Sun, 1 Dec 2019 01:33:50 -0800 (PST) John Ladasky wrote: > The only thing I must install with pip is tensorflow-gpu. For > everything else, I make use of the Ubuntu repositories. The Synaptic > package manager installs packages (including Python modules) for all > user accounts at the same time, which I like. > > When I installed tensorflow-gpu using pip, I was in fact frustrated > because I couldn't figure out how to deploy it across multiple user > accounts at one time. I ended up installing it three times, once in > each account. You're suggesting that's actually preferred, at least > when pip is performing the installation. OK, I will endure the > repetition. You can set up a system-wide virtualenv (for instance in /usr/local/lib/myenv) and use pip install as root to set up everything into that. All the normal users have to do then is prepend /usr/local/lib/myenv/bin to their PATH. After that, you have a system-wide consistent distribution of all your needed Python packages. You can then uninstall all python packages provided by the Linux distro which you don't need. At the moment it seems as if all you need to install locally with pip is tensorflow-gpu. This will change once some future version of tensorflow-gpu depends on newer versions of the system-provided packges. When that happens, pip will pull all those packages into the user's local venv, and it will have to do that individually for each user. BTW, it took me a long time to embrace Python's "virtualenv" concept because I had a hard time figuring out what it was and how it worked. Turns out that there is no magic involved, and that "virtual environment" is a misnomer. It is simply a full Python environment in a separate location on your system. Nothing virtual about it. -- https://mail.python.org/mailman/listinfo/python-list
Re: Pickle caching objects?
On Sun, Dec 01, 2019 at 12:26:15PM +1100, Chris Angelico wrote: I can't answer your question authoritatively, but I can suggest a place to look. Python's memory allocator doesn't always return memory to the system when the objects are freed up, for various reasons including the way that memory pages get allocated from. But it internally knows which parts are in use and which parts aren't. You're seeing the RSS go down slightly at some points, which would be the times when entire pages can be released; but other than that, what you'll end up with is a sort of high-water-mark with lots of unused space inside it. So what you're seeing isn't actual objects being cached, but just memory ready to be populated with future objects. Thank you and Richard for your responses, this makes perfect sense now. Cheers, -- José María (Chema) Mateos || https://rinzewind.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: "Don't install on the system Python"
On 11/30/19 9:42 PM, John Ladasky wrote: > Can anyone provide concrete examples of problems arising from > installing modules on top of the system Python? Am I courting > disaster? No you aren't. I've also never had any problems. I've installed many things into my root system Python installation with pip including PyQt5. It's just easier for me to have them in the system installation. I'm on a CentOS 7 box, which depends on Python 2 for a lot of system functions. I've moved to Python 3 now, so I mess with the system Python less and less. I understand that I probably should be using a virtualenv and pip installing into that, but I'm just too lazy. A couple of years ago I even managed to upgrade the system Python from 2.6 to 2.7 without any issues. I ended up making an RPM that neatly upgraded the system one. I also have upgraded the gtk2 bindings and the GTK2 library itself (again building RPMs), and everything worked fine, even the graphical centos utilities. The only problems I've ever heard of come from trying to manually remove stuff from Python that the system depended on. I've never heard of any problems installing additional modules. 90% of the time the module you need will be in the repositories, so no worries there at all. And not much to worry about for the rest. -- https://mail.python.org/mailman/listinfo/python-list
Re: ModuleNotFoundError with click module
On 12/1/19 12:26 AM, Peter Otten wrote: Tim Johnson wrote: Using linux ubuntu 16.04 with bash shell. Am retired python programmer, but not terribly current. I have moderate bash experience. When trying to install pgadmin4 via apt I get the following error traceback when pgadmin4 is invoked: Traceback (most recent call last): snipped ... File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 25, in import click ModuleNotFoundError: No module named 'click' If I invoke python3 (/usr/local/bin/python3), version 3.7.2 and invoke >>> import click click is imported successfully. ... I'm too lazy to look into the details of your paths -- I'd just make sure that click is installed with the same interpreter and user as pgadmin4, e. g. globally $ sudo /usr/local/bin/python3 -m pip install click $ sudo /usr/local/bin/python3 path/to/setup.py install # or whatever it takes to install pgadmin4 Like I said, I'm not current. Yikes. Now I have /usr/local/lib/python3.7/site-packages/clic-0.1.3.dist-info/ After I have my coffee I will attempt to proceed from there with whatever it takes to finalize thanks -- Tim tj49.com -- https://mail.python.org/mailman/listinfo/python-list
Re: tab replace to space 4 (rmlibre)
> Its just that I've just began to touch tkinter, and would like to know of > bug-related pitfalls before I waste energy on trying to figure out what I > did wrong. :-\ One thing which is not obvious or easy to debug: Text widgets have some kind of inefficiency related to really long lines that don't have line breaks. The result is that your cpu will be completely gobbled up if you have a sufficiently long line. I haven't run any tests to get an accurate view of what the breaking point is. But one the cores in my cpu was hold up at 100% when displaying many many thousands of characters without line breaks. After lots of debugging and optimization finding, I just happened to google for "cpu load issue with" each of the widgets I was using and discovered that this indeed was the cause of the cpu load. Swapping in line breaks at regular, known intervals quickly fixed this. tl;dr Google for "issue with tkinter " for the widgets you'll be using, or " with tkinter" if you're experiencing a particular issue and are not sure if it could be caused for some reason by tkinter. -- https://mail.python.org/mailman/listinfo/python-list
Re: "Don't install on the system Python"
On 12/1/19 10:41 AM, Michael Torrie wrote: > On 11/30/19 9:42 PM, John Ladasky wrote: >> Can anyone provide concrete examples of problems arising from >> installing modules on top of the system Python? Am I courting >> disaster? > No you aren't. I've also never had any problems. I've installed many > things into my root system Python installation with pip including PyQt5. > It's just easier for me to have them in the system installation. I'm > on a CentOS 7 box, which depends on Python 2 for a lot of system > functions. I've moved to Python 3 now, so I mess with the system Python > less and less. I understand that I probably should be using a > virtualenv and pip installing into that, but I'm just too lazy. > > A couple of years ago I even managed to upgrade the system Python from > 2.6 to 2.7 without any issues. I ended up making an RPM that neatly > upgraded the system one. I also have upgraded the gtk2 bindings and the > GTK2 library itself (again building RPMs), and everything worked fine, > even the graphical centos utilities. > > The only problems I've ever heard of come from trying to manually remove > stuff from Python that the system depended on. I've never heard of any > problems installing additional modules. 90% of the time the module you > need will be in the repositories, so no worries there at all. And not > much to worry about for the rest. My guess is that the issue is with some more complicated/esoteric packages. Especially if there are ones that don't maintain strict backwards compatibility, so that some packages using them have a maximum usable version as well as a minimum usable version. This can lead to troubles as some packages become incompatible because one needs a version greater than x, while another needs a version less than x. -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Re: "Don't install on the system Python"
On Sun, Dec 1, 2019 at 3:46 PM John Ladasky wrote: > > Long-time Ubuntu user here. > > For years, I've read warnings about not installing one's personal stack of > Python modules on top of the system Python. It is possible to corrupt the > OS, or so I've gathered. > > Can anyone provide concrete examples of problems arising from installing > modules on top of the system Python? Am I courting disaster? > I'm going to start by separating out two concepts that are often, but not always, the same. The "system Python" is the one that the OS depends on. On my Debian Stretch, that's /usr/bin/python3 (symlink to /usr/bin/python3.5). On older systems, that might be a Python 2.7 (or worse). This is the installation of Python that is managed by your OS package manager, and - more importantly - is the one that any OS-provided scripts will depend on. The "default Python" is the one you get when you type "python3" at the shell. Often this is the same as the system Python, but there's no requirement for this to be the case. On many of my systems, I compile and install a new build of Python periodically (usually from the master branch, so it's a pre-alpha), and I'm happy for that to take over the name "python3". Currently, on my main system (the aforementioned Debian Stretch), that's /usr/local/bin/python3 (a symlink to /usr/local/bin/python3.9). Since the system Python is managed by your OS package manager, you need to be careful about using pip to install packages into it. For instance, if I were to "/usr/bin/python3 -m pip install psycopg2", it would potentially conflict with "sudo apt install python3-psycopg2". You MAY be safe using pip to install something that isn't available in your package manager, but it's possible to run into dependency versioning conflicts. Be careful. But installing into your default Python, if it's not the system Python, is absolutely safe. Or rather, if it breaks anything, then it's not your fault - it's the fault of something depending on the system Python but not using an absolute shebang :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: ModuleNotFoundError with click module
On 12/1/19 12:26 AM, Peter Otten wrote: Tim Johnson wrote: Using linux ubuntu 16.04 with bash shell. Am retired python programmer, but not terribly current. I have moderate bash experience. When trying to install pgadmin4 via apt I get the following error traceback when pgadmin4 is invoked: Traceback (most recent call last): File "setup.py", line 17, in from pgadmin.model import db, User, Version, ServerGroup, Server, \ File "/usr/share/pgadmin4/web/pgadmin/__init__.py", line 19, in from flask import Flask, abort, request, current_app, session, url_for File "/usr/local/lib/python3.7/site-packages/flask/__init__.py", line 21, in from .app import Flask File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 34, in from . import cli File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 25, in import click ModuleNotFoundError: No module named 'click' If I invoke python3 (/usr/local/bin/python3), version 3.7.2 and invoke >>> import click click is imported successfully. In this invocation, sys.path is: ['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/home/tim/.local/lib/python3.7/site-packages', '/usr/local/lib/python3.7/site-packages'] $PYTHONPATH is empty when the bash shell is invoked $PATH as follows: /home/tim/bin:/home/tim/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin click.py can be found at /usr/local/lib/python3.7/site-packages/pipenv/patched/piptools/ in turn click.py imports click, presumably as the package, which appears to be at /usr/local/lib/python3.7/site-packages/pipenv/vendor/click Any number of settings of PYTHONPATH to the various paths above has failed to resolve the ModuleNotFoundError Same issues with attempting install from a virtual environment. Any help will be appreciated. thanks tim I'm too lazy to look into the details of your paths -- I'd just make sure that click is installed with the same interpreter and user as pgadmin4, e. g. globally $ sudo /usr/local/bin/python3 -m pip install click $ sudo /usr/local/bin/python3 path/to/setup.py install # or whatever it takes to install pgadmin4 OK. Now I have /usr/local/lib/python3.7/site-packages/Click-7.0.dist-info/ which holds the following files: INSTALLER LICENSE.txt METADATA RECORD top_level.txt WHEEL I haven't a clue as to how to proceed! Never seen this before ... Furthermore, google is offering me nothing conclusive. Where to go from here! -- Tim tj49.com -- https://mail.python.org/mailman/listinfo/python-list
Re: ModuleNotFoundError with click module
On 12/1/19 3:41 PM, Tim Johnson wrote: On 12/1/19 12:26 AM, Peter Otten wrote: Tim Johnson wrote: Using linux ubuntu 16.04 with bash shell. Am retired python programmer, but not terribly current. I have moderate bash experience. When trying to install pgadmin4 via apt I get the following error traceback when pgadmin4 is invoked: Traceback (most recent call last): File "setup.py", line 17, in from pgadmin.model import db, User, Version, ServerGroup, Server, \ File "/usr/share/pgadmin4/web/pgadmin/__init__.py", line 19, in from flask import Flask, abort, request, current_app, session, url_for File "/usr/local/lib/python3.7/site-packages/flask/__init__.py", line 21, in from .app import Flask File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 34, in from . import cli File "/usr/local/lib/python3.7/site-packages/flask/cli.py", line 25, in import click ModuleNotFoundError: No module named 'click' If I invoke python3 (/usr/local/bin/python3), version 3.7.2 and invoke >>> import click click is imported successfully. In this invocation, sys.path is: ['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/home/tim/.local/lib/python3.7/site-packages', '/usr/local/lib/python3.7/site-packages'] $PYTHONPATH is empty when the bash shell is invoked $PATH as follows: /home/tim/bin:/home/tim/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin click.py can be found at /usr/local/lib/python3.7/site-packages/pipenv/patched/piptools/ in turn click.py imports click, presumably as the package, which appears to be at /usr/local/lib/python3.7/site-packages/pipenv/vendor/click Any number of settings of PYTHONPATH to the various paths above has failed to resolve the ModuleNotFoundError Same issues with attempting install from a virtual environment. Any help will be appreciated. thanks tim I'm too lazy to look into the details of your paths -- I'd just make sure that click is installed with the same interpreter and user as pgadmin4, e. g. globally $ sudo /usr/local/bin/python3 -m pip install click $ sudo /usr/local/bin/python3 path/to/setup.py install # or whatever it takes to install pgadmin4 OK. Now I have /usr/local/lib/python3.7/site-packages/Click-7.0.dist-info/ which holds the following files: INSTALLER LICENSE.txt METADATA RECORD top_level.txt WHEEL I haven't a clue as to how to proceed! Never seen this before ... Furthermore, google is offering me nothing conclusive. Where to go from here! P.S. It looks like that directory is sort of a stub; regardless of my take on it I am no longer having the ModuleNotFoundError. Peter has a been a great help. Couldn't have done it without him. cheers -- Tim tj49.com -- https://mail.python.org/mailman/listinfo/python-list
Re: increasing the page size of a dbm store?
> Maybe port to SQLite? I would not choose dbm these days. After sparring with it a while, I tweaked the existing job so that it chunked things into dbm-appropriate sizes to limp through; for the subsequent job (where I would have used dbm again) I went ahead and switched to sqlite and had no further issues. I'm not sure if it's worth mentioning the issue in the docs for the dbm module so others don't bump against it. I'm not sure if the limit is on sum(size(key) for key in db) or the number of keys total. Just not the sort of thing I'd want someone to be depending on, unaware of the potential pitfalls. Thanks, -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: "Don't install on the system Python"
W dniu niedziela, 1 grudnia 2019 05:42:35 UTC+1 użytkownik John Ladasky napisał: > For years, I've read warnings about not installing one's personal stack of > Python modules on top of the system Python. It is possible to corrupt the > OS, or so I've gathered. > > Well, I've never heeded this advice, and so far nothing bad has happened to > me. I don't like Anaconda, or virtual environments in general. I don't like > heavyweight IDE's. I like to be able to type "python3" at the command prompt > and be sure what I'll be getting. I have multiple user accounts on a system > that I manage, and I want every user account to have access to the same > modules. > > Maybe the modules that I require are safe to install on the system Python, > I'm not sure. My must-haves are mostly scientific computing and data > management modules: Numpy, Scipy, Scikit-learn, Matplotlib, Pandas, > Biopython, and Tensorflow. I also use PyQt5 from time to time. > > Can anyone provide concrete examples of problems arising from installing > modules on top of the system Python? Am I courting disaster? I did not heard of such problems. But I have another warning. Ubuntu 18.4 uses Python 3.6. Do not try to install 3.7 or 3.8 as systemwide python3 version. It breaks some programs including standard terminal. -- https://mail.python.org/mailman/listinfo/python-list
