On Sun, Nov 17, 2019 at 11:34:08AM -0600, Bruce Dubbs via blfs-dev wrote: > On 11/17/19 8:39 AM, Pierre Labastie via blfs-dev wrote: > > Le 17/11/2019 à 11:45, Wayne Blaszczyk via blfs-dev a écrit : > > > Hi All, > > > > > > I have just noticed that with all my builds, there are __pycache__ > > > directories > > > being installed all over the place. > > > As an example, when I installed totem, it created /usr/bin/__pycache__/ > > > with a > > > bunch of files underneath it which are actual not related to totem. > > > I raised a bug but it was quickly dismissed. > > > https://gitlab.gnome.org/GNOME/totem/issues/375 > > > I know that totem is not part of BLFS anymore, but there are other > > > packages > > > creating the same sort thing. > > > Polkit for example has created files under > > > /usr/lib/gobject-introspection/giscanner/__pycache__/
Amusing, particularly the response "The work-around is to run the build as a normal user and install the files under a temporary directory before deploying them. That's what package managers do for example." I assume that means recommending that the cached pyc files should never be installed, and therefore every use of a module will not be able to gain any speedup from having the byte code already created ? More below. > > > Has anyone else come across this? Is this expected? > > > > > > > I think it is expected: each time a python program is interpreted, it is > > actually transformed to an intermediate binary (byte-compiled). Under > > certain > > conditions, this binary is saved into the __pycache__directory, so that the > > next time it is run, the binary is used, saving the time for byte-compiling. > > > > There is also a python module for byte-compiling all the files in a > > directory [1]. > > > > As a specific example, gedit has a post_install.py script, which does > > byte-compile all the python programs in /usr/lib/gedit/plugins. I think it > > is > > the same for totem (although creating __pycache__ directly in /usr/bin > > should > > be avoided, IMO). > > > > Pierre > > > > [1] https://docs.python.org/3/library/compileall.html > > > > Checking my system, I find that directory in the following locations. > > /usr/lib/peas-demo > /usr/lib/gobject-introspection/giscanner > /usr/lib/gedit/plugins > /usr/lib/python3.7 Actually there are many such directories in 3.7. I'm still on that version while I prepare for a major machine upgrade. > /usr/lib/python3.8 > /usr/share/gdb/python > /usr/share/gtk-doc/python/gtkdoc > > It's not common, but I wonder what happens if /usr is mounted read only. > You would think that these files would be in a more conventional place line > /var/cache. > > I did a quick search but couldn't find anything about changing the cache > location. I would think an environment variable or a configuration file > could control this. The only python specific environment variables I could > find are PYTHONPATH and PYTHONDONTWRITEBYTECODE. > > I did find these: > > https://stackoverflow.com/questions/3522079/changing-the-directory-where-pyc-files-are-created/57414308#57414308 > > https://stackoverflow.com/questions/16869024/what-is-pycache > > -- Bruce Yes, looks as if PYTHONDONTWRITEBYTECODE ought to suppress them. An update to that second link says that since 3.8 you can use an envvar to change the location for the annoying cache directories, but it seems to just link back to the first link. There ARE comments that pyc files have sometimes not been updated when a module has been modified, and also that keeping files in sync across multiple machines can cause problems with the cached versions, e.g. in tests, so apparently some Python programmers routinely don't cache the byte code. I suppose that on a modern x86_64 machine the time loss will be minimal. Ah, see https://bugs.python.org/issue33499 From msg319714 in that (June last year) | Merged as PYTHONPYCACHEPREFIX=path, -X pycache_prefix=path and sys.pycache_prefix :) See also the discussion before that about affecting all versions of python on the system (so, 2.7 and theoretically any earlier 3.x versions). NB I have no idea if this will do the job when system-wide packages are installed (the original request and the discussion seem to be for developers who want to cache (if at all) in ~/. ĸen -- Whilst all mushrooms are edible, the trick is to eat only those which will prove to be edible more than once. The Celebrated Discworld Almanak recommends you play safe and eat beans on toast. -- http://lists.linuxfromscratch.org/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
