[issue13420] newer() function in dep_util.py discard changes in the same second
New submission from David Amian : Hi, I had a problem making deb packages with distutils and cdbs. Looking for any solution, I found Issue 11933 and saw that the newer() method has been changed, using [ST_MTIME] instead of st_mtime. This commit is to solve Issue 10148 that describes a bug when copying files with python, it depends on the filesystem. In these changes, modification time of files are checked as a integer, discarding the millisec. It makes the issue. For example, I've a file that is modified with an installation prefix. CDBS runs the build step and after the install step. In build step, the prefix is '/usr/local' because the prefix argument isn't supported, immediately the install step runs with prefix='/usr', so the file that contains a variable with project path modified with the prefix is not copied by distutils, it runs in a different millisec but in the same second. Then when install my app, it crashes because the variable with project path is '/usr/local' instead of '/usr' and this variable is used in the source. To solve this problem I made some modifications on file_util.py and dep_util.py files, I revert the changes of the issue 11933, but rounding all times returned by stat() with two decimals. With that changes, distutils is not affected by the Issue 10148 and if any file is modified in the same second, but in different millisec, distutils will compare it and it wil copy the file. -- assignee: tarek components: Distutils files: fix_mtime.diff keywords: patch messages: 147794 nosy: d.amian, eric.araujo, tarek priority: normal severity: normal status: open title: newer() function in dep_util.py discard changes in the same second type: behavior versions: Python 2.7 Added file: http://bugs.python.org/file23716/fix_mtime.diff ___ Python tracker <http://bugs.python.org/issue13420> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13420] newer() function in dep_util.py discard changes in the same second
David Amian added the comment: sorry, I didn't explain well. I've a project, in the setup.py file, I've a function called update_prefix, that updates the 'path_project' variable with prefix arguments from setup.py If you runs setup.py with --prefix=/usr, then the file in 'projectname/projectnameconfig.py' is modified changing the 'path_project' variable from @PREFIX@+"share/projectname" to '/usr/share/projectname'. If you runs setup.py without prefix arguments, the prefix is '/usr/local'. Debuild which is the tool used to build debian package, in this case using CDBS, gives this output: it first runs: ... python setup.py build \ --build-base="/tmp/buildd/pdal2-0.3.1/./build" running build running build_py creating /tmp/buildd/pdal2-0.3.1/build creating /tmp/buildd/pdal2-0.3.1/build/lib.linux-i686-2.7 creating /tmp/buildd/pdal2-0.3.1/build/lib.linux-i686-2.7/pdal copying pdal/utils.py -> /tmp/buildd/pdal2-0.3.1/./build/lib.linux-i686-2.7/pdal copying pdal/pdalconfig.py -> /tmp/buildd/pdal2-0.3.1/./build/lib.linux-i686-2.7/pdal ... then it runs: ... cd . && \ python setup.py install \ --root="/tmp/buildd/pdal2-0.3.1/debian/pdal2/" \ --install-purelib=/usr/lib/python2.7/site-packages/ \ --prefix=/usr --no-compile -O0 --install-layout=deb running install running build running build_py running build_scripts It doesn't copy pdal/pdalconfig.py becouse the file in /tmp/buildd/pdal2-0.3.1/./build/lib.linux-i686-2.7/pdal have the same st_mtime in seconds than pdal/pdalconfig.py, but if you see de st_mtime, both are different in millisec and the file is different. File: «pdal/pdalconfig.py» Modify: 2011-11-16 11:55:52.665727826 +0100 File: «/tmp/buildd/pdal2-0.3.1/./build/lib.linux-i686-2.7/pdal/pdalconfig.py» Modify: 2011-11-16 11:55:52.0 +0100 but if you print the st_mtime of both files within newer() function you can check the file's timestamp are the same when they are really not: ST_MTIME-source-target-1321440952-1321440952 That is cause of the issue, that I explain in the early comment, I fixed it rounding to two decimals instead of rounding to integer. -- ___ Python tracker <http://bugs.python.org/issue13420> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13420] newer() function in dep_util.py discard changes in the same second
David Amian added the comment: 2011/11/18 Éric Araujo > > Éric Araujo added the comment: > > > I've a project, in the setup.py file, I've a function called > update_prefix, that updates the > > 'path_project' variable with prefix arguments from setup.py > > If you runs setup.py with --prefix=/usr, then the file in > 'projectname/projectnameconfig.py' is > > modified changing the 'path_project' variable from > > @PREFIX@+"share/projectname" > to > > '/usr/share/projectname'. If you runs setup.py without prefix arguments, > the prefix is '/usr/local'. > > Did you write a custom 'install' class or are you for example looking at > sys.argv to decide whether to run your function? If it’s not implemented > as a distutils command, it’s not surprising that it does not integrate well. > Really I've a function to update prefix based on sys.argv but i changed this using a custom class command and the same issue occurs > > > [...] > > That is cause of the issue, that I explain in the early comment, I fixed > it rounding to two decimals > > instead of rounding to integer. > > Okay. Can you make sure that this is the source of the problem, for > example by adding time.sleep(1) between build and install? > I puts a time.sleep(1) bewteen 'setup.py build' and 'setup.py install' and I got these results: *Without waiting: $debuild -us -uc [...] mkdir -p debian/python-module-stampdir cd . && \ python setup.py build \ --build-base="/tmp/pdal/pdal2-0.3.1/./build" running build running build_py copying pdal/pdalconfig.py -> build/lib.linux-i686-2.7/pdal running build_scripts [...] python setup.py install \ --root="/tmp/pdal/pdal2-0.3.1/debian/pdal2/" \ --install-purelib=/usr/lib/python2.7/site-packages/ \ -f --prefix=/usr [...] running build running build_py running build_scripts [...] $ The stats of files are: File: «pdal/pdalconfig.py» Modify: 2011-11-21 13:44:52.497971724 +0100 File: «build/lib.linux-i686-2.7/pdal/pdalconfig.py» Modify: 2011-11-21 13:44:52.0 +0100 It doesn't copy in the install step because st_mtimes in newer() are the same: ST_MTIME-source- 1321879492 ST_MTIME-target- 1321879492 *Waiting a second between build and install: $debuild -us -uc [...] mkdir -p debian/python-module-stampdir cd . && \ python setup.py build \ --build-base="/tmp/pdal/pdal2-0.3.1/./build" running build running build_py copying pdal/pdalconfig.py -> build/lib.linux-i686-2.7/pdal running build_scripts [...] python setup.py install \ --root="/tmp/pdal/pdal2-0.3.1/debian/pdal2/" \ --install-purelib=/usr/lib/python2.7/site-packages/ \ -f --prefix=/usr [...] running build running build_py copying pdal/pdalconfig.py -> build/lib.linux-i686-2.7/pdal running build_scripts [...] $ Obviously the file has been copied as st_mtimes in newer() differ by a second ST_MTIME-source- 1321879603 ST_MTIME-target- 1321879602 > > -- > > ___ > Python tracker > <http://bugs.python.org/issue13420> > ___ > -- ___ Python tracker <http://bugs.python.org/issue13420> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
