[issue12944] setup.py upload to pypi needs to work with specified files
New submission from Rene Dudfield : We need to specify files to upload to pypi. Otherwise we have to use the web interface. 'Regarding the "setup.py upload" command, this isn't very helpful because it will not upload a package that was already built - for example, a Windows package built on another host while I am trying to release from a Linux desktop. Since automatic package building across a build farm is the only way I can actually build packages for most platforms, this rules out use of the "upload" command.' See this post for more details: http://as.ynchrono.us/2011/09/releasing-python-software-is-tedious.html -- assignee: tarek components: Distutils messages: 143754 nosy: eric.araujo, illume, tarek priority: normal severity: normal status: open title: setup.py upload to pypi needs to work with specified files type: feature request ___ Python tracker <http://bugs.python.org/issue12944> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11685] possible SQL injection into db APIs via table names... sqlite3
New submission from Rene Dudfield : Hi, you can possibly do an SQL injection via table names (and maybe some other parts of queries). Tested with sqlite3, but maybe it affects others too. You can not do parameter substitution for table names, so people use normal python string formatting instead. If the table name comes from an untrusted source, then possibly an SQL injection could happen. cheers, -- messages: 132247 nosy: illume priority: normal severity: normal status: open title: possible SQL injection into db APIs via table names... sqlite3 type: security ___ Python tracker <http://bugs.python.org/issue11685> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11685] possible SQL injection into db APIs via table names... sqlite3
Rene Dudfield added the comment: Hello, because the sqlite3 package comes with python. -- ___ Python tracker <http://bugs.python.org/issue11685> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11685] possible SQL injection into db APIs via table names... sqlite3
Rene Dudfield added the comment: The bug in python is that you can not use parameter substitution to put the table names into the queries. So people are forced to use string substitution instead. -- ___ Python tracker <http://bugs.python.org/issue11685> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11685] possible SQL injection into db APIs via table names... sqlite3
Rene Dudfield added the comment: Hi, aaah, ok. It seems to require the use of a quote function. See http://www.sqlite.org/c3ref/mprintf.html However python does not seem to expose the function? I don't see how you can write safe queries using python without it. -- ___ Python tracker <http://bugs.python.org/issue11685> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14367] try/except block in ismethoddescriptor() in inspect.py, so that pydoc works with pygame in Python 3.2
Rene Dudfield added the comment: This can be closed. $ python3 -m pydoc -p 7464 Then view in browser: http://localhost:7464/pygame.html No error. -- nosy: +illume ___ Python tracker <https://bugs.python.org/issue14367> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11685] possible SQL injection into db APIs via table names... sqlite3
Rene Dudfield added the comment:
Hi,
Here is an article with people trying to find a solution:
http://stackoverflow.com/questions/6514274/how-do-you-escape-strings-for-sqlite-table-column-names-in-python
"The psycopg2 documentation explicitly recommends using normal python % or {}
formatting to substitute in table and column names."
Sqlalchemy uses a format_table method with their sql compiler to quote table
names for sqlite.
It's probably just sane to either use SQLalchemy, use ctypes to get at the
sqlite mprintf function, or perhaps look at the above stackoverflow article for
more solutions.
There is python code out there vulnerable to attack, that doesn't quote table
names correctly. Including at least one major python framework. Hopefully
people who care will follow some of the above links.
cheers,
--
___
Python tracker
<http://bugs.python.org/issue11685>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2751] Regression for executing packages
Rene Dudfield added the comment: Hi, note -m on packages still works with python3.0 and python3.1. It works by allowing a __main__.py file that gets called by -m. This is a really annoying regression for python2.6. It's a fairly wide spread feature too. I can't see any PEP for why this regression exists. For pygame we will probably work around it by using "python -m tests.main" or "python -m pygame.tests.run" which is ugly, but at least it'll work! cheers, -- nosy: +illume ___ Python tracker <http://bugs.python.org/issue2751> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2751] Regression for executing packages
Rene Dudfield added the comment: hello, thanks for the explanation of why it's that way. Any ideas of a work around? python2.5 has been out for ages now. Even if it was an accident, it's the behavior people expect, and it's still a regression. Also, why should it matter if a module is a package or a module? Note how pygame.tests has a type of module, and not of package: >>> import pygame.tests >>> type(pygame.tests) Even though it is a package, python calls its type a module. This has been true for a long time (at least as far back as python2.3). Because it's a regression, I think this bug should be reopened. To illustrate why it causes problems, here is part of the documentation mentioning the __main__. """ You can do a self test with: python -m pygame.tests Or with python2.6 do: python -m pygame.tests.__main__ See a list of examples... python -m pygame.examples Or with python2.6, python -m pygame.examples.__main__ """ It's twice as long, and I doubt anyone will remember the __main__ part. People used to running their programs with -m now have broken programs with python2.6. Having said all that, maybe there is a work around... One work around might be to make it into a module-module, not a package-module. Then have the module-module load the package-module into its namespace. I haven't tested that yet, but it might work. Will have to go through a round of testing to see how that works out. Will write back when I've found out the issues with that approach. cheers, -- ___ Python tracker <http://bugs.python.org/issue2751> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6964] import new fails
New submission from Rene Dudfield : python3.1 >>> import new Traceback (most recent call last): File "", line 1, in ImportError: No module named new 2to3-3.1 doesn't mention how to change it. -- components: 2to3 (2.x to 3.0 conversion tool) messages: 92974 nosy: illume severity: normal status: open title: import new fails type: behavior versions: Python 3.1 ___ Python tracker <http://bugs.python.org/issue6964> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6964] import new fails
Rene Dudfield added the comment: Hi, yes it does report a warning with 2.6, thanks. python2.6 -3 -c "import new" -c:1: DeprecationWarning: The 'new' module has been removed in Python 3.0; use the 'types' module instead. I guess it should be a TODO item with 2to3. cheers, -- ___ Python tracker <http://bugs.python.org/issue6964> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6968] numpy extensions to distutils... are a source of improvements for distutils
New submission from Rene Dudfield : Hi, numpy includes a numpy/distutils package which has a lot of goodies/fixes which might be able to be incorporated into the main distutils. Adding this note so distutils maintainers are aware of it. cheers, -- assignee: tarek components: Distutils messages: 92986 nosy: illume, tarek severity: normal status: open title: numpy extensions to distutils... are a source of improvements for distutils type: feature request ___ Python tracker <http://bugs.python.org/issue6968> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6269] threading documentation makes no mention of the GIL
Rene Dudfield added the comment: hello, CPU intensive programs can also benefit from the GIL if they use code which releases the GIL around the CPU intensive parts. Some parts of python do this, as do the numpy and pygame extensions amongst others. Another good, but separate, documentation patch would be to document which functions release the GIL. cheers, -- nosy: +illume ___ Python tracker <http://bugs.python.org/issue6269> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2399] Patches for Tools/msi
Rene Dudfield added the comment: Adding distutils to components for this old msi patch. These seem like useful things for those making msi packages with distutils. -- components: +Distutils nosy: +illume ___ Python tracker <http://bugs.python.org/issue2399> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7412] distutils install race condition
New submission from Rene Dudfield : hello! Pythons distutils has a race condition where it starts to copy files into the python path whilst installing. This is a race condition, since python programs can be importing the package whilst the package is being installed. It would be good for distutils to install things in an atomic manner. Where things can be installed, or not installed. Like, on unix by moving the files in from a temporary directory. This would also help reduce breakages. Since if a package breaks half way installing a package then the broken version will not over write the existing version. It's not a very serious problem, since most people don't install things on live important systems(but some do). It does make hard to diagnose problems though... which are good ones to fix. Most packaging tools fix the issues with the source installs (eg, using .deb installers). I've only looked at the source install, but I imagine other install methods might be affected too. os.rename mkdtemp http://docs.python.org/library/tempfile.html#tempfile.mkdtemp Perhaps files should be created in the /tmp first, and then moved in. /tmp/ can be in a different file system so a rename won't work in that case on some OSes. If you create the temp directory in the same directory as the directory to install. In that case rename doesn't work, using the temp file system(eg /tmp) and then copying to a temp directory in the site-packages directory, with finally a move into place. If copying to the site-packages before moving, then there could be left over temp files in the site-packages directory. So these would need to be considered, and cruft cleaned up. This is why using the normal temp directory would be better, since the temp files will be cleaned as normal policy on the system. setuptools/distribute writes the .egg-info at the end of the install. This should also use a move for install, for the same reasons. However, there might still be a separate race condition since that lives in a different directory. Since you can not atomically move two things at once( I think? ) the .egg-info will continue to be a race condition, unless both directories are moved into the same one. Just as moving packages into place will make the installs more robust, so will testing the packages before moving them into place. Some way to test the install before moving it into place would be good. Or at least trying to import the package in a sub process. Taking advantage of the test command could be an idea. Test before moving into place would give a better chance of good packages being used. Import before moving into place could also help. -- assignee: tarek components: Distutils messages: 95833 nosy: illume, tarek severity: normal status: open title: distutils install race condition type: security ___ Python tracker <http://bugs.python.org/issue7412> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7412] distutils install race condition
Rene Dudfield added the comment: Hi Tarek, moving a package into place right at the end is the best thing to do I think. It solves a couple common problems: - broken packages after an install is stopped half way for one of many common reasons. - old files left around will not be there, since they would be moved. numpy 1.4 breaks for example if installing over an old version first and not removing the old version. So did pygame in the past... and likely other packages. - race condition whilst installing (not so common, but happens). I'm not sure I did say something like 'this race condition can happen even if you copy the file in some temp dir, then move them to python.'. But now I see a problem... if the existing directory has files in it, you can not move another directory over the top of it atomically. I don't think there is a way to solve this completely, without changing the packaging system quite dramatically. However we can make it much more likely we will win the race!!! First trying to install into a temp directory, then if on the same file system, moving the old directory to a backup one, then moving the directory into place, finally removing the old directory. This would reduce the time of the race condition to a fraction of a second(0.01 seconds on my box) compared to many seconds on average, and longer for big packages with big files. Packages not installing correctly is a big problem for C/C++ packages more so than simply python packages. Fixing this will mean distutils becomes more robust, and also will stop a security issue. I tried a method similar to this pseudo code as a work around: python setup.py install --home=/tmp/apackagetmp move temp package into place... Here is a script I started on... for pygame installs on ubuntu karmic koala. http://rene.f0o.com/~rene/stuff/safer_install.py I thought I'd try it out as a separate thing first to try and get it right before trying a patch with distutils. There's still a bunch of issues with it, and it needs tests written. - when installing to a root place the permissions are not correct. They are the ones of the user it is ran as. ie, 'chmod -R root:staff dest' would probably fix it. - it would be good to extract the paths for various things from distutils... rather than guessing. For example the dist-packages directory, where the include files are etc etc. - I'm going to put in a test step. So it installs to the temp directory first, then tries to run the package tests before installing. The idea being that if all the tests past *then* install. This should make it more robust still. It will probably try as a separate process with "python -m apackage.tests.__main__" or something like that. - it always copies in all of the files. Which is good and bad. It means that there won't be old files, but it also means that it can't avoid work... like say only one file changes. - probably many other issues... cheers, ps. happy new year! -- ___ Python tracker <http://bugs.python.org/issue7412> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7677] distutils, better error message for setup.py upload -sign without identity.
New submission from Rene Dudfield : hi, When using setup.py upload --sign without --identity, gpg can give weird messages. Rather than this error message: """running upload gpg --detach-sign --local-user gn -a dist/pywebsite-0.1.18pre.tar.gz gpg: skipped "gn": secret key not available gpg: signing failed: secret key not available error: command 'gpg' failed with exit status 2 make: *** [upload] Error 1""" It might be nicer to also mention using --identity=, as well as a link to the documentation for --sign (url and also the help from "--help upload"). cheers, -- assignee: tarek components: Distutils messages: 97589 nosy: illume, tarek severity: normal status: open title: distutils, better error message for setup.py upload -sign without identity. type: behavior ___ Python tracker <http://bugs.python.org/issue7677> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5342] distutils removing old files, deleting unneeded old files from installed location.
New submission from Rene Dudfield : A common problem is removing old files from the installed location. eg. version 1 installs. site-packages/packagename/bla.so version 2 installs. site-packages/packagename/_bla.so site-packages/packagename/bla.py However, in version 2 if we install over the top of version 1, distutils does not remove bla.so. Which causes the package to break because bla.so will be used instead of bla.py distutils should be able to be given a list of old files to make sure are removed from the package. It should work with the various installers... msi, dmg etc, as well as when using setup.py install. I've seen this cause breakage with numerous packages. The solution is to hack some old file detection into each package, or tell users 'delete the old install first'. Neither of which is really nice. Specifying an old_files meta data should be able to help. -- assignee: tarek components: Distutils messages: 82586 nosy: illume, tarek severity: normal status: open title: distutils removing old files, deleting unneeded old files from installed location. type: feature request ___ Python tracker <http://bugs.python.org/issue5342> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
