Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #13156: revert changeset f6feed6ec3f9, which was only relevant for native
On Sun, Oct 16, 2011 at 6:10 AM, Terry Reedy wrote: > On 10/15/2011 7:47 AM, "Martin v. Löwis" wrote: >> >> This may be a done deal, but: no. If a patch is reverted, the NEWS entry >> that got in with it gets out again on reversal. The NEWS file >> is for users of the release; there is no point telling them that a >> change was made first, and than got undone. > > I was going to say the same thing, but ... > > If a change is released in x.y.z and reverted for release x.y.(z+k), then I > think both notices should be present in their respective sections. > > I checked the date on the original patch and it was before 3.2.1, so perhaps > it *was* released. Indeed, "was it released?" is the gating criteria for whether the old NEWS entry is removed or whether a new one is made. No release should ever remove a NEWS entry from an earlier release, *unless* the NEWS entry itself was a mistake (i.e. it refers to a change that wasn't actually part of a release). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On 13 October 2011 17:25, Éric Araujo wrote: >> My expectation would be that the user would type pysetup install >> some_binary_format_file.zip and have that file unpacked and all the >> "bits" put in the appropriate place. Basically just like installing >> from a source archive - pysetup install project-1.0.tar.gz - but >> skipping the compile steps because the compiler output files are >> present. > Yep. > >> That may need some extra intelligence in pysetup if it doesn't have >> this feature already [...] just unzip the bits into the right place, >> or something similar. > Yes. The bdist can be just like an sdist, but it contains compiled > files instead of C source files (maybe setuptools bdist_egg is just > that), then pysetup uses the setup.cfg file to find files and install > them at the right places. I have uploaded an initial (working, just needs test, docs and some testing and possibly tidying up) version of bdist_simple in the tracker (http://bugs.python.org/issue13189). Taking note of Martin's comments, it would be nice to at a minimum have a converter to take a bdist_simple and build bdist_msi/bdist_wininst from it (and possibly vice versa). I'm not sure how to go about that at this stage, but I'll work on it. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Jeremy Kloth gmail.com> writes: > That said, I have been working on a drop-in replacement for the current > bdist_wininst executable stub with the following features: [snip] > http://www.flickr.com/photos/67460826 N04/sets/72157627653603530/ [snip] Sounds interesting, but your flickr link did not work for me, even after I tried replacing "" with "@". Care to post a shortened link? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Martin v. Löwis v.loewis.de> writes: > In particular wrt. virtual environments: I see no need to actually > *install* files multiple times. It's rather sufficient that the > distributions to be installed are *available* in the virtual env after > installation, and unavailable after being removed. Actually copying > them into the virtual environment might not be necessary or useful. > > So I envision a setup where the MSI file puts the binaries into a place > on disk where pysetup (or whatever tool) finds them, and links them > whereever they need to go (using whatever linking mechanism might work). > For MSI in particular, there could be some interaction with pysetup, > e.g. to register all virtualenvs that have linked the installation, > and warn the user that the file is still in use in certain locations. > Likewise, automated download might pick an MSI file, and tell it not > to place itself into the actual Python installation, but instead into > a location where pysetup will find it. While it seems a little inelegant, copying might actually be simpler and less error-prone. Firstly, AFAIK you can't do true symlinks in relatively recent, widely-used versions of Windows like XP. Also, while some people use virtualenvs in a central location (such as virtualenvwrapper users), others have their envs under a project folder. I don't know that the complication of a centralised registry of virtual envs is necessarily a good thing. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Nick Coghlan gmail.com> writes: > Compilation can be a problem on Linux systems as well, so a platform > neutral format is a better idea. Just have a mechanism that allows > pysetup to create a bdist_msi from a bdist_simple. Similar, bdist_rpm > and bdist_deb plugins could be taught to interpret bdist_simple. I agree that a platform-neutral format is a good idea, but there might be other complications with binary formats, which I'm not sure we've considered. For example, if we're just bundling self-contained C extensions which just link to libc/msvcrt, that's one thing. But what if those extensions link to particular versions of other libraries? Are those referenced binaries supposed to be bundled in the archive, too? I don't know that the dependency language supported by packaging extends to these kinds of dependencies on external, non-Python components. If we leave it to the packager to include all relevant binary dependencies, I'm not sure how satisfactory that'll be - possibly, not very. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Éric Araujo netwok.org> writes: > [Vinay] > > A simple change to packaging will allow an archive containing a > > setup.cfg-based > > directory to be installed in the same way as a > > source directory. > Isn’t that already supported, as long as the tarball or zipfile contains > source files? In any case, it was intended to be, and there’s still > support code around. No, by which I mean - if you have a simple zip of a project directory# containing a setup.cfg, and run pysetup3 install , it fails to work in the same way as pysetup3 install where the is a recursive zip of . However, a two-line change enables this: http://goo.gl/pd51J > Correct. I’m still pondering whether I find the idea of registering > built files in setup.cfg as elegant or hacky :) We also have the other > ideas I wrote to choose from. On Linux, if we're building from source, of course we use the build_ext step to capture the built artifacts. However, how else could you do it on Windows, when you're not actually building? The built files could be named in the [extension:] section rather than the [files] section - the former means that you have to add code to deal with it, the latter is less elegant but would require less work to make it happen. > > 3. Ideally, the GUI should co-operate with venvs, by offering some > > form of browse facility. The command line does this automatically. > Will Windows users want a GUI to create venvs too? I don't think this is needed for venv creation, but having a "Find Other..." to locate an alternative Python in a virtual env doesn't seem too onerous for the user. Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
Paul Moore gmail.com> writes: > > On 13 October 2011 17:25, Éric Araujo netwok.org> wrote: > >> My expectation would be that the user would type pysetup install > >> some_binary_format_file.zip and have that file unpacked and all the > >> "bits" put in the appropriate place. Basically just like installing > >> from a source archive - pysetup install project-1.0.tar.gz - but > >> skipping the compile steps because the compiler output files are > >> present. > > Yep. > > > >> That may need some extra intelligence in pysetup if it doesn't have > >> this feature already [...] just unzip the bits into the right place, > >> or something similar. > > Yes. The bdist can be just like an sdist, but it contains compiled > > files instead of C source files (maybe setuptools bdist_egg is just > > that), then pysetup uses the setup.cfg file to find files and install > > them at the right places. > > I have uploaded an initial (working, just needs test, docs and some > testing and possibly tidying up) version of bdist_simple in the > tracker (http://bugs.python.org/issue13189). There's one area of pysetup3 functionality which I don't think has been discussed in this thread, though it's pertinent to Windows users. Namely, a completely declarative approach to installation locations will not satisfy all requirements. For example, if you want to install PowerShell scripts, the right way of doing that is to make Shell API calls at installation time to determine the correct location for the installing user. I have this working at present for a project, using pysetup's hook functionality; but any move to a completely passive archive format would lose this kind of flexibility. So, I think whatever archive format we end up with should provide exactly the same level of flexibility we currently get with pysetup3 for pure-Python projects, but extended to include binary deliverables. The simplest way of doing this is to register those binaries in setup.cfg, and to have hook code check for the correct dependencies in the run-time environment before actually installing (e.g. x86/x64/ARM/Python version dependencies). While it's not the slickest solution imaginable, it does allow for just about all use cases. This flexibility is more important under Windows than under Posix, because the installation locations under Posix conform much more closely to a standard (FHS) than anything you find in the Windows environment. Distribution build step = get all source, binary code and data defined in setup.cfg and in a single directory tree (containing the setup.cfg in the root of the tree), then zip that tree. Distribution installation step = unzip aforementioned zip, and run pysetup3 install . BTW, I believe that for virtual env installations, there's no need to provide integration with Add/Remove Programs. I find that the Add/Remove Programs dialog takes long enough to populate as it is :-( Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On 16 October 2011 22:32, Vinay Sajip wrote: > There's one area of pysetup3 functionality which I don't think has been > discussed in this thread, though it's pertinent to Windows users. Namely, a > completely declarative approach to installation locations will not satisfy all > requirements. For example, if you want to install PowerShell scripts, the > right > way of doing that is to make Shell API calls at installation time to determine > the correct location for the installing user. Interesting. That's not a use case that I have encountered, but I can see it could be an issue. I have been working on the basis that a bdist_simple format that matches the functionality of bdist_wininst is at least good enough for those projects that currently use bdist_wininst. The only potential issue right now is with postinstall scripts, which bdist_simple doesn't support. It would be easy enough to add, and indeed it may be possible to use existing packaging functionality already (I haven't looked into this area). > I have this working at present for > a project, using pysetup's hook functionality; but any move to a completely > passive archive format would lose this kind of flexibility. So, I think > whatever > archive format we end up with should provide exactly the same level of > flexibility we currently get with pysetup3 for pure-Python projects, but > extended to include binary deliverables. I don't disagree, but I'm struggling to see how that would be done. > The simplest way of doing this is to > register those binaries in setup.cfg, and to have hook code check for the > correct dependencies in the run-time environment before actually installing > (e.g. x86/x64/ARM/Python version dependencies). While it's not the slickest > solution imaginable, it does allow for just about all use cases. Can you give an example of a setup.cfg, that would work like this? Suppose I have two files, foo.py and bar.pyd, which are a pure-python module and a compiled C extension. How would I write a setup.cfg and lay out the directory structure to allow pysetup install to do the right thing? I tried to do this myself, but couldn't get it to work the way I expected. It may be I was just hitting bugs, but it felt to me like I was going at the problem the wrong way. > This > flexibility is more important under Windows than under Posix, because the > installation locations under Posix conform much more closely to a standard > (FHS) > than anything you find in the Windows environment. Agreed in principle, although in practice, most projects I've encountered have ended up working within bdist_wininst limitations, probably just as a matter of practicality. > Distribution build step = get all source, binary code and data defined in > setup.cfg and in a single directory tree (containing the setup.cfg in the root > of the tree), then zip that tree. > > Distribution installation step = unzip aforementioned zip, and run pysetup3 > install . Works for me. I'd like to see a bdist_xxx command to do the build step as you describe, if only to make it trivially simple for developers to produce binary distributions. Having to package stuff up manually is bound to put at least some developers off. If you can give me the example I mentioned above, I could work on modifying the bdist_simple code I posted to the tracker today to produce that format rather than my custom format based on bdist_wininst. For the installation step, you shouldn't even need to unzip, as pysetup3 can do the unpacking for you. > BTW, I believe that for virtual env installations, there's no need to provide > integration with Add/Remove Programs. I find that the Add/Remove Programs > dialog > takes long enough to populate as it is :-( Agreed. Personally, as I've said, I'm happy not to use Add/Remove even for system installations - pysetup list and pysetup remove do what I need without slowing down the Add/Remove list. But I accept that's not likely to be the view of many Windows users. Anyone using virtual envs, though, is probably by definition comfortable enough with command line tools to be willing to use pysetup3. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging and binary distributions for Python 3.3
On Sunday, October 16, 2011 02:24:58 PM Vinay Sajip wrote: > Jeremy Kloth gmail.com> writes: > > That said, I have been working on a drop-in replacement for the current > > > bdist_wininst executable stub with the following features: > [snip] > > > http://www.flickr.com/photos/67460826 N04/sets/72157627653603530/ > > [snip] > > Sounds interesting, but your flickr link did not work for me, even after I > tried replacing "" with "@". Care to post a shortened link? Hmm, clicking the link in the email works here. but just to be safe: http://goo.gl/pC48e Thanks -- Jeremy Kloth ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Modules of plat-* directories
Hi, I don't understand why we kept modules of the plat-* directories (e.g. Lib/plat-linux/CDROM.py). It looks like these modules are not used, except maybe some DL constants used by PyKDE4. Can't we move used constants to classic Python modules (e.g. the os module) and drop unused modules? These modules are not regenerated when Python is compiled, so I don't understand how values can be correct. For example, IN.INT_MAX is 2147483647, whereas it should be 9223372036854775807 on my 64-bit Linux. These values don't look reliable. I'm looking at these modules because Arfrever asked me to review a patch for h2py.py, a script to regenerate these modules: http://bugs.python.org/issue13032 He also suggested somewhere to regenerate these modules when Python is built. -- Python has builtin modules generated from C header files: - CDIO: sunos5 - CDROM: linux - DLFCN: linux, sunos5 - _emx_link: os2emx - grp: os2emx - IN: aix4, darwin, freebsd[45678], linux, netbsd1, os2emx, sunos5, unixware7 - pwd: os2emx - SOCKET: os2emx - STROPTS: sunos5, unixware7 - TYPES: linux, sunos5 CDROM/CDIO can be used to control the low-level CDROM API. CDROM is used by the cdsuite project: http://offog.org/code/cdsuite.html DLFCN is used by PyKDE4: sys.setdlopenflags(DLFCN.RTLD_NOW|DLFCN.RTLD_GLOBAL). I didn't know this sys function :-) (OS/2 platform is deprecated, see PEP 11.) The IN module is used by policykit to get INT_MAX (used to compute "MAX_DBUS_TIMEOUT = INT_MAX / 1000.0"). It was also used in SGI video demos of Python 1.4. STROPTS is not used. TYPES is used by other plat-* modules (IN, DLFCN, STROPTS). These modules contain non-working functions: def __STRING(x): return #x def __constant_le32_to_cpu(x): return ((__u32)(__le32)(x)) __STRING(x) returns None, "#x" is a comment in Python (in C, the preprocessor converts x to a string). Call __constant_le32_to_cpu() fails because __le32 is not defined. Even if __le32 was defined, the code doesn't look to be valid. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Modules of plat-* directories
Le lundi 17 octobre 2011 01:16:36, Victor Stinner a écrit : > For example, IN.INT_MAX is 2147483647, whereas it should > be 9223372036854775807 on my 64-bit Linux. Oops, wrong example: INT_MAX is also 2147483647 on 64 bits. I mean IN.LONG_MAX. IN.LONG_MAX is always 9223372036854775807 on Linux, on 32 and 64 bits systems. > [Arfrerever] also suggested somewhere to regenerate these modules > when Python is built. somewhere is here: http://bugs.python.org/issue12619 > DLFCN is used by PyKDE4: > sys.setdlopenflags(DLFCN.RTLD_NOW|DLFCN.RTLD_GLOBAL). I didn't know this > sys function :-) Because Python has a sys.setdlopenflags(), we should provide these constants in a regular module (sys or posix). I'm quite sure that PyKDE4 would accept to add a try/except ImportError: DLFCN is only used in one file, to get 2 constants. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
