[Python-Dev] Possibility of binary configuration mismatch
Hi All, I'm not sure there's anything you can do about this, but I thought I should alert the Python devs that it can happen... http://allmydata.org/trac/tahoe/ticket/704#comment:7 describes a situation where my macports-installed python25 had a pyOpenSSL egg installed in it by something other than macports (possibly by easy_install-2.5?) that was not compatible with the Python build. My hunch is that the pyOpenSSL had binaries compiled against a UCS4 Python, but I don't know for sure. Whatever did the installation of the bad egg was almost certainly being executed by the macports python25 because macports is installed in /opt/local, and nothing is likely to have installed it under that prefix by chance. In other words, this egg probably couldn't have been left over from some non-macports python installation. In fact, I haven't had any other version of Python2.5 installed on this machine. Very odd. I wonder if it makes sense to enhance the extension module system to record this kind of information so the problem can be diagnosed by the system? -- Dave Abrahams BoostPro Computing http://www.boostpro.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] urlparse.urlunsplit should be smarter about +
This is a bug report. bugs.python.org seems to be down.
>>> from urlparse import *
>>> urlunsplit(urlsplit('git+file:///foo/bar/baz'))
git+file:/foo/bar/baz
Note the dropped slashes after the colon.
--
Dave Abrahams Meet me at BoostCon: http://www.boostcon.com
BoostPro Computing
http://www.boostpro.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urlparse.urlunsplit should be smarter about +
At Sat, 08 May 2010 11:04:47 -0500,
John Arbash Meinel wrote:
>
> Stephen J. Turnbull wrote:
> > David Abrahams writes:
> > >
> > > This is a bug report. bugs.python.org seems to be down.
> > >
> > > >>> from urlparse import *
> > > >>> urlunsplit(urlsplit('git+file:///foo/bar/baz'))
> > > git+file:/foo/bar/baz
> > >
> > > Note the dropped slashes after the colon.
> >
> > That's clearly wrong, but what does "+" have to to do with it? AFAIK,
> > the only thing special about + in scheme names is that it's not
> > allowed as the first character.
>
> Don't you need to register the "git+file:///" url for urlparse to
> properly split it?
Yes. But the question is whether urlparse should really be so fragile
that every hierarchical scheme needs to be explicitly registered.
Surely ending with “+file” should be sufficient to have it recognized
as a file-based scheme
--
Dave Abrahams Meet me at BoostCon: http://www.boostcon.com
BoostPro Computing
http://www.boostpro.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pep 353: Py_ssize_t advice
Pep 353 advises the use of this incantation: #if PY_VERSION_HEX < 0x0205 typedef int Py_ssize_t; #define PY_SSIZE_T_MAX INT_MAX #define PY_SSIZE_T_MIN INT_MIN #endif I just wanted to point out that this advice could lead to library header collisions when multiple 3rd parties decide to follow it. I suggest it be changed to something like: #if PY_VERSION_HEX < 0x0205 && !defined(PY_SSIZE_T_MIN) typedef int Py_ssize_t; #define PY_SSIZE_T_MAX INT_MAX #define PY_SSIZE_T_MIN INT_MIN #endif (C++ allows restating of typedefs; if C allows it, that should be something like): #if PY_VERSION_HEX < 0x0205 typedef int Py_ssize_t; # if !defined(PY_SSIZE_T_MIN) # define PY_SSIZE_T_MAX INT_MAX # define PY_SSIZE_T_MIN INT_MIN # endif #endif You may say that library developers should know better, but I just had an argument with a very bright guy who didn't get it at first. Thanks, and HTH. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pep 353: Py_ssize_t advice
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams schrieb: >> #if PY_VERSION_HEX < 0x0205 >> typedef int Py_ssize_t; >> #define PY_SSIZE_T_MAX INT_MAX >> #define PY_SSIZE_T_MIN INT_MIN >> #endif >> >> I just wanted to point out that this advice could lead to library >> header collisions when multiple 3rd parties decide to follow it. I >> suggest it be changed to something like: >> >> #if PY_VERSION_HEX < 0x0205 && !defined(PY_SSIZE_T_MIN) > > Strictly speaking, this shouldn't be necessary. C allows redefinition > of an object-like macro if the replacement list is identical (for > some definition of identical which applies if the fragment is > copied literally from the PEP). > > So I assume you had non-identical replacement list? No: a. I didn't actually experience a collision; I only anticipated it b. We were using C++, which IIRC does not allow such redefinition c. anyway you'll get a nasty warning, which for some people will be just as bad as an error > Can you share what alternative definition you were using? > > In any case, I still think this is good practice, so I added it > to the PEP. Thanks, -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pep 353: Py_ssize_t advice
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: >> c. anyway you'll get a nasty warning, which for some people will be >> just as bad as an error > > Try for yourself. You get the warning only if the redefinition is not > identical to the original definition (or an object-like macro is > redefined as a function-like macro or vice versa). I'm confident that whether you get the warning otherwise is dependent both on the compiler and the compiler-flags you use. But this question is academic now, I think, since you accepted my suggestion. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Plea to distribute debugging lib
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > Dave Abrahams schrieb: >> The only problem here is that there appears to be a lag in the release of >> ActivePython after Python itself is released. >> >> Is there any chance of putting up just the debugging libraries a little >> earlier? > > I may be out of context here: what is the precise problem in producing > them yourself? Why do you need somebody else to do it for you? At the moment I have too weak a server to provide those files, but that will change very soon. All that said, the Python and ActiveState teams need to be aware of each and every Python release and go through a standard release procedure anyway, whereas -- except for this problem -- I would not. I'm willing to try to add it if that's what works, and of course it's easy for me to say, but I think it adds a lot more overhead for me than it would for the other two groups. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Plea to distribute debugging lib
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > I'm not sure whether you are requesting these for yourself or for > somebody else. If for somebody else, that somebody else should seriously > consider building Python himself, and publishing the result. I'm requesting it for the many Boost.Python (heck, all Python 'C' API) users who find it a usability hurdle when their first visual studio projects fail to work properly in the default mode (debug) just because they don't have the right Python libraries. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Plea to distribute debugging lib
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams schrieb: >>> I'm not sure whether you are requesting these for yourself or for >>> somebody else. If for somebody else, that somebody else should seriously >>> consider building Python himself, and publishing the result. >> >> I'm requesting it for the many Boost.Python (heck, all Python 'C' API) >> users who find it a usability hurdle when their first visual studio >> projects fail to work properly in the default mode (debug) just >> because they don't have the right Python libraries. > > And there is not one of them who would be willing and able to build > a debug release, and distribute that I don't know. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PCBuild8
Hi, I tried building with MS Visual Studio 2005 from PCBuild8/pcbuild.sln, and for me it fails miserably. The first major complaint comes when linking pythoncore, where the _init_types symbol can't be found. On the other hand, allowing MSVS to convert PCBuild/pcbuild.sln works okay. Am I missing something? Thanks, -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PCBuild8
on Mon Mar 05 2007, "Martin v. Löwis" writes: > David Abrahams schrieb: >> I tried building with MS Visual Studio 2005 from PCBuild8/pcbuild.sln, >> and for me it fails miserably. The first major complaint comes when >> linking pythoncore, where the _init_types symbol can't be found. On >> the other hand, allowing MSVS to convert PCBuild/pcbuild.sln works >> okay. Am I missing something? > > Yes, it doesn't work in Python 2.5 as released. This specific problem > has been fixed in the trunk and the 2.5 branch several months ago, > so I recommend to use either of these Okay, thanks. > (or use VS 2003 to build the released version). VS2005 seems to work just fine for the released version as long as I stay away from the PCBuild8/ stuff. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Finding the python library binaries (and docs)
I'm trying to find the Python library binaries associated with a given python executable. If I look at the help for sys.prefix and sys.exec_prefix import sys; help(sys) I see: prefix -- prefix used to find the Python library exec_prefix -- prefix used to find the machine-specific Python library This text is somewhat ambiguous, but from the fact that exec_prefix refers to the "machine-specific" Python library I'm led to guess that prefix's "the Python library" refers to the *.py* files that form most of the Python standard library, and "the machine-specific Python library" perhaps refers to the library binaries such as libpython25.a. However, neither of those interpretations seems to correspond to reality. My guess is that these constants correspond to the --prefix and --exec_prefix options to the configure script invocation that was used to build Python. What I'm really looking for, I think, is the directory corresponding to the --libdir option to configure, but I don't see that in sys. If I look at "configure --help," it claims the default libdir is EPREFIX/lib, so I suppose I could look in sys.exec_prefix+'/lib', but when I look at real installations I find only the shared libraries in that location. For the static libraries, I have to look in EPREFIX/lib/python$(version)/config/, and in fact there is a symlink there to the shared libs. So: 1. I think the documentation for sys and configure both need some updating 2. I'd like to know if there's an officially correct procedure for finding the library binaries associated with a Python executable. Thanks in advance, -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding the python library binaries (and docs)
on Tue Mar 06 2007, "Martin v. Löwis" wrote: > David Abrahams schrieb: >> I'm trying to find the Python library binaries associated with a given >> python executable. > > This really isn't a python-dev question; please use python-list > (news:comp.lang.python) instead. I wrestled with the right list for this one and determined that only the python devs would know the answers. Also I intended to propose that the information I'm looking for be added to sys as a standard attribute. > Please take a look at sys.path. No help at all; that is the module search path (i.e. for .py files), not for the Python library binaries. >> 1. I think the documentation for sys and configure both need some >> updating > > Would you like to work on a patch? This information can be readily > obtained from the Python source code. I'll consider it, once we get the original intention cleared up. There are lots of ways to interpret what the code actually does, and documentation that merely transcribes the code's logic will not be very useful. >> 2. I'd like to know if there's an officially correct procedure for >> finding the library binaries associated with a Python executable. > > Yes (although I'm not sure what a "library binary" is). I gave one example in my post: libpython25.a -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding the python library binaries (and docs)
on Tue Mar 06 2007, "Martin v. Löwis" wrote: > David Abrahams schrieb: >> on Tue Mar 06 2007, "Martin v. Löwis" wrote: >>> David Abrahams schrieb: >>>> I'm trying to find the Python library binaries associated with a given >>>> python executable. >>> This really isn't a python-dev question; please use python-list >>> (news:comp.lang.python) instead. >> >> I wrestled with the right list for this one and determined that only >> the python devs would know the answers. > > That absolutely cannot be the case. Python is open source, you have > *everything* you need to answer this question. That assumes this is one of those questions to which "use the source" is applicable. I think answering it requires some understanding of intention that's not explicit in the source. 'Course, it may be that the answer is "nobody knows." >>>> 2. I'd like to know if there's an officially correct procedure for >>>> finding the library binaries associated with a Python executable. >>> Yes (although I'm not sure what a "library binary" is). >> >> I gave one example in my post: libpython25.a > > Ah, ok. If you want to know where that is installed (officially), > check out what "make install" does. Also, ask yourself whether you know > a Python module that should know how to find it. distutils No help there. It has at best a haphazard method of looking for libraries, and never looks in a /config subdirectory on linux AFAICT. Also AFAICT the results are not tied to a particular Python executable. Looking at some code that works most of the time because the authors tried to deduce intention by looking at the Python source and existing installations isn't much better than trying to deduce intention by looking at the Python source and existing installations myself. > and freeze Not part of Python? -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] GCC version compatibility
Recently people testing Boost.Python with GCC on Linux have reported that the extensions being tested have to be compiled with exactly the same version of GCC as the Python they're being loaded into, or they get mysterious crashes. That doesn't correspond to my past experience; it has always been true that, as long as the compiler used to build Python and the one used to build the extension have compatible 'C' ABIs, we've been okay. Yes, if you were going to pass types like FILE* across the Python/C API, then you additionally need to be sure that the two compilers are using the same 'C' library. That said, none of the Boost.Python tests do that. I'm wondering if there has been a well-known recent change either in Python or GCC that would account for these new reports. Any relevant information would be appreciated. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Linux Python linking with G++?
Apparently Python on some linux distros is being linked by g++ rather than gcc, resulting in the C++ runtime library being linked into Python; this has bad consequences for compatibility between C++ extension modules and Pythons that have been built with different versions of GCC. Is this behavior intentional? --- Begin Message --- Topics: Re: Regressions in your Boost libraries as of 2005-07-06 Re: Regressions in your Boost libraries as of 2005-07-06 --- End Message --- --- Begin Message --- On Jul 6, 2005, at 9:04 PM, Ralf W. Grosse-Kunstleve wrote: > (Newer?) Python executables are linked with "g++" (instead of "gcc"), > which > brings in libstdc++.so. We had weird crashes when using a Python > compiled on a > machine with libstdc++.so.5 (Redhat 8.0) for building Boost.Python > extensions > on another machine which had libstdc++.so.6 (Gentoo 1.6.8 and Fedora > core 3, I > believe). > > To check for libstdc++ incompatibilities, run > > ldd /python On eddie (one of the trouble systems), this gives: libstdc++.so.5 => /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.5-20050130/libstdc++.so.5 (0xb7dd8000) for the Python installed on the system. > and, e.g., > > ldd /minimal_ext.so ... and this gives (for eddie's GCC 4.0.0): libstdc++.so.6 => /usr/lib/gcc/i686-pc-linux-gnu/4.0.0-alpha20050213/libstdc++.so.6 (0xb7f0d000) It looks like that's the problem, then. We have two libstdc++ versions around, hence the need to build Boost.Python with the same compiler version as Python. Bummer. I wonder... does Python even use C++? Should they just be linking with gcc? Doug ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost --- End Message --- --- Begin Message --- --- David Abrahams <[EMAIL PROTECTED]> wrote: > >> Doug, I know you've drawn that conclusion, but it really surprises me. > >> Generally speaking, I have been able to use any version of Python with > >> any compiler, provided Python was compiled with something having a > >> compatible 'C' ABI. > > > > I dunno what else to say. You're free to play around with things on our > > Linux machine (eddie); we have various flavors of GCC 3.3 and 3.4 > > available, with Pythons compiled with those plus the system GCC 3.3.5. > > GCC 2.95.3 (with or without STLport) works fine with the system Python > > (compiled with the system's GCC 3.3.5), but Boost.Python tests fail to > > run properly unless Python is recompiled with the same GCC 3.3.6 or > > 3.4.4. > > Well, I've done this numerous times on numerous machines. I wonder > what's up with eddie? Ralf, does this sound normal to you? (Newer?) Python executables are linked with "g++" (instead of "gcc"), which brings in libstdc++.so. We had weird crashes when using a Python compiled on a machine with libstdc++.so.5 (Redhat 8.0) for building Boost.Python extensions on another machine which had libstdc++.so.6 (Gentoo 1.6.8 and Fedora core 3, I believe). To check for libstdc++ incompatibilities, run ldd /python and, e.g., ldd /minimal_ext.so Look for lines like libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x002a95689000) HTH, Ralf __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ___ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost --- End Message --- -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
Sjoerd Mullender <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> Apparently Python on some linux distros is being linked by g++ rather >> than gcc, resulting in the C++ runtime library being linked into >> Python; this has bad consequences for compatibility between C++ >> extension modules and Pythons that have been built with different >> versions of GCC. Is this behavior intentional? > > Configure with --without-cxx to not use g++. Since there is an option > in configure, I assume it is intentional. O-kay... any idea what the rationale for this decision might be? -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Chaining try statements: eltry?
Tim Peters <[EMAIL PROTECTED]> writes: > I also suspect that if they weren't in the language already, a PEP to > introduce them would fail, because > > still_looking = True > some loop: > if found it: > still_looking = False > break > if still_looking: > # what would have been in the "else" clause > > is clear and easy to write without it. Oh, that's wierd. I didn't know there were "else" clauses for loops, but I would've expected the other semantics. That is, either the loop terminates normally, "else:" whatever. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
Skip Montanaro <[EMAIL PROTECTED]> writes: > >> Configure with --without-cxx to not use g++. Since there is an > >> option in configure, I assume it is intentional. > > Dave> O-kay... any idea what the rationale for this decision might be? > > I believe it's so that people can link in libraries written in C++ and have > them initialized properly. Can you give specifics? What do you mean by "link in?" Do you mean "statically link into the Python interpreter," or something else? Boost.Python is a library written in C++ and I've never had trouble using it with a Python executable... until I ran into a Python that was linked with libstdc++! -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GCC version compatibility
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> I'm wondering if there has been a well-known recent change either in Python >> or GCC that would account for these new reports. Any relevant >> information would be appreciated. > > So what about the theory that it may be that different versions of > libstdc++ get linked? That's been confirmed. > Python is linked with g++ if configure thinks this is necessary Right. The question is, when should configure "think it's necessary?" > and the g++ used to link the extension might be different. > > I'd like to see a backtrace of one such mysterious crash. I don't have it, but ldd confirms that the crash happens when the versions of libstdc++ in python and in the extension module are different. A C++ exception thrown from the extension module into the Boost.Python library to which it is linked (both compiled and linked with the same g++) without passing through any of Python's code (of course) will cause a crash unless Python is using the same libstdc++ as everything else, or unless Python isn't linked with libstdc++. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
Skip Montanaro <[EMAIL PROTECTED]> writes: > >> I believe it's so that people can link in libraries written in C++ > >> and have them initialized properly. > > Dave> Can you give specifics? What do you mean by "link in?" Do you > Dave> mean "statically link into the Python interpreter," or something > Dave> else? > > Probably not. I'm not a C++ guy. My understanding is that global (maybe > static?) C++ objects need the help of C++'s version of crt0 to get properly > initialized at program start. Yes. > If there is some library with such objects that happens to get > wrapped and dynamically linked into a Python interpreter Whoa there. How would that ever happen under ordinary circumstances? Doesn't Python's makefile control what gets linked to Python? > that was linked with a regular C linker (and thus had a C crt0), > that initialization wouldn't happen. Right, and linking would fail, IIRC. It seems to me that if someone decides to build a wacky Python executable that links in C++ code directly, they can afford to use a configure option. There's no need to punish all the other writers of C++ extension modules out there by tying the default build of Python to a particular version of libstdc++. > Dave> Boost.Python is a library written in C++ and I've never had > Dave> trouble using it with a Python executable... until I ran into a > Dave> Python that was linked with libstdc++! > > Sorry, I can't help. I'm just recounting my remembering of the > reasons for C++ linkage. Personally, I avoid C++ as much as I > can... If there's someone around here who is responsible for this change and knows its real rationale, can (s)he please tell me what it is? If not, can we please change things back so Python doesn't get linked to the C++ runtime by default? Thanks, -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> Apparently Python on some linux distros is being linked by g++ rather >> than gcc, resulting in the C++ runtime library being linked into >> Python; this has bad consequences for compatibility between C++ >> extension modules and Pythons that have been built with different >> versions of GCC. Is this behavior intentional? > > It's as Skip says. According to the C++ standard, a "C++ program" is > one where all translation units are written in C++. While most platforms > offer interoperability between C and C++ in the sense that C libraries > can be linked into C++ programs, interoperability in the other direction > is not always supported, i.e. C programs may not be able to use C++ > libraries. This is the specific case you are talking about: Python is > written in C, yet the extension might be written in C++. The C++ standard doesn't cover interoperability with 'C', or dynamic linking (we on the C++ committee are working to fix that, but for now that is the case) and it doesn't cover dynamic loading without linking, which is what happens when Python loads an extension written in C++. You can't appeal to the standard for the rules about what needs to be done. All this stuff is entirely implementation-dependent. > Now, on some of these implementations, things can be fixed by writing > main() as a C++ translation unit, and compiling it with the C++ > compiler. Then, Python itself is a C library to this C++ program, and > the extension modules are C++ libraries. Then everything works fine. C++ extension modules work fine even when main() is a 'C' program on Linux/GCC. The only reason that main would have to be a C++ program on this platform and compiler is if there were C++ translation units _linked_ into it (not loaded as in with dlopen). Since whoever writes the Makefile for Python also controls what's being linked into it, there's no reason to speculate and make Python a C++ program based on what might be linked in. > To support this, main() must be a C++ program. Python compiles main > using the C++ compiler if configure thinks this is necessary. > > Doing so is necessary for example on systems using the G++ collect2 > mechanism for static initializers: compiling main with g++ leads > to a call to __main(), which is then created by collect2. Extension modules that get loaded with dlopen don't get their static initializers run by __main() anyway. > configure thinks that using CXX for linking is necessary if compiling > a program using CXX and linking it using CC fails. That might be the right thing to do for some programs, but AFAICT that's the wrong logic to use for Python. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
Jeff Epler <[EMAIL PROTECTED]> writes:
> If we change the linker back to gcc, not g++, will it work if extension
> module 1 gets linked with libstdc++ A and ABI Q, and extension module 2
> gets linked with libstdc++ B and ABI Z?
Yes, unless they are using sys.setdlopenflags to force symbols to be
shared across these extension modules. That's a very intentional act
and should (obviously?) only be undertaken when the extension modules
share an ABI.
> What if a built-in module is written in C++, as it might be for some
> people embedding C++?
"Embedding" usually refers to embedding a _Python_ interpreter in a
program written in some language other than Python. But I think
that's what you meant (just correct me if I'm wrong).
> (this will force use of g++ as the linker, right?)
Yes.
> Don't these cases matter too?
Yes. But the 2nd case is not one in which the Python executable is
being built. The person building a program that embeds Python can
control how (s)he does linking.
> Assuming they can fail now, how will changing the use of CXX as the
> linker fix them?
I don't understand the question.
> Jeff
> PS The Python 2.3 and Python 2.4 binaries installed on my Fedora Core
> machine don't list libstdc++ in `rpm -q --requires python' or `ldd
> /usr/bin/python'. I don't see a patch that would change Python's
> behavior in the SRPM, though. I wonder what the difference is between
> my FC2 and the other systems...
I don't know; the ones we happen to be testing are Debian ("sarge," I
think).
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >>>configure thinks that using CXX for linking is necessary if compiling >>>a program using CXX and linking it using CC fails. >> >> >> That might be the right thing to do for some programs, but AFAICT >> that's the wrong logic to use for Python. > > Why do you say that? Python compiles Modules/ccpython.cc as the main > function, using the C++ compiler, and then tries to link it somehow. > On some systems (including some Linux installations), linking will > *fail* if linking is done using gcc (instead of g++). So we *must* > link using g++, or else it won't link at all. This is starting to feel tautological, or self-referential, or something. If by ccpython.cc you mean http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/dist/src/Modules/ccpython.cc well of *course* linking will fail. You have to compile that file as C++ program since it uses extern "C" which is only legal in C++ . But AFAICT, in a normal build of the Python executable, there's no reason at all for main to be a C++ function in the first place. Unless, of course, I'm missing something. So if I am missing something, what is it? -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GCC version compatibility
[Christoph, please keep the python-dev list in the loop here, at least until they get annoyed and decide we're off-topic. I think this is crucial to the way they package and deliver Python] Christoph Ludwig <[EMAIL PROTECTED]> writes: > On Thu, Jul 07, 2005 at 06:27:46PM -0400, David Abrahams wrote: >> "Martin v. Löwis" <[EMAIL PROTECTED]> writes: >> >> > David Abrahams wrote: >> >> I'm wondering if there has been a well-known recent change either in >> >> Python >> >> or GCC that would account for these new reports. Any relevant >> >> information would be appreciated. > [...] >> > Python is linked with g++ if configure thinks this is necessary >> >> Right. The question is, when should configure "think it's necessary?" > > Just to add to the confusion... I encountered the case that configure decided > to use gcc for linking but it should have used g++. (It is Python > PR #1189330 <http://tinyurl.com/dlheb>. This was on a x86 Linux system with > g++ 3.4.2.) > > Background: The description of --with-cxx in the README of the > Python 2.4.1 source distribution made me think that I need to > configure my Python installation with > --with-configure=/opt/gcc/gcc-3.4.2/bin/g++ if I plan to use C++ > extensions built with this compiler. (That was possibly a > misunderstanding on my part, AFAICT, yes. > but Python should build with this option anyway.) > > configure set `LINKCC=$(PURIFY) $(CC)'. The result was that make failed when > linking the python executable due to an unresolved reference to > __gxx_personality_v0. I had to replace CC by CXX in the definition of LINKCC > to finish the build of Python. > > When I looked into this problem I saw that configure in fact builds a test > executable that included an object file compiled with g++. If the link step > with gcc succeeds then LINKCC is set as above, otherwise CXX is > used. Obviously, on my system this test was successful so configure decided > to link with gcc. However, minimal changes to the source of the test program > caused the link step to fail. It was not obvious to me at all why the latter > source code should cause a dependency on the C++ runtime if the original > code does not. My conclusion was that this test is fragile and should be > skipped. Sounds like it. I have never understood what the test was really checking for since the moment it was first described to me, FWIW. > If Python is built with --with-cxx then it should be linked with CXX > as well. U betcha. > I gather from posts on the Boost mailing lists that you can import > Boost.Python extensions even if Python was configured > --without-cxx. Yes, all the tests are passing that way. > (On ELF based Linux/x86, at least.) That leaves me wondering > > * when is --with-cxx really necessary? I think it's plausible that if you set sys.dlopenflags to share symbols it *might* end up being necessary, but IIRC Ralf does use sys.dlopenflags with a standard build of Python (no --with-cxx)... right, Ralf? > * what happens if I import extensions built with different g++ versions? Will >there be a conflict between the different versions of libstdc++ those >extensions depend on? Not unless you set sys.dlopenflags to share symbols. It's conceivable that they might conflict through their shared use of libboost_python.so, but I think you have to accept that an extension module and the libboost_python.so it is linked with have to be built with compatible ABIs anyway. So in that case you may need to make sure each group of extensions built with a given ABI use their own libboost_python.so (or just link statically to libboost_python.a if you don't need cross-module conversions). HTH, -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> Unless, of course, I'm missing something. So if I am missing >> something, what is it? > > You are missing something, and I can only repeat myself. Some systems > require main() to be compiled as C++, or else constructors may not work > (and perhaps other things fail as well). Yes, and that becomes important in programs that have constructors. I.e., C++ programs. The Python executable is not such a program, except for one C++ file: ccpython.cc. There is no reason that file couldn't be rewritten as a pure 'C' file and any need for Python to be linked with G++ would disappear. > The configure option --with-cxx (documented as "enable C++ support") > make Python C++ options What are "Python C++ options?" > work on such systems. It is automatically enabled if a C++ compiler > is found. > > There is configure auto-detection for what linker is used when > ccpython.o becomes main(). > > This is the state of the things as it is. In what way would you like to > see that state changed? I would like the Python executable never to be linked (or compiled either) by g++ unless that is explicitly requested by the person invoking configure or make. > I could personally accept if ccpython and --with-cxx would be dropped > entirely (i.e. deliberately breaking systems which require it); I don't believe any systems require it. I realize you have said otherwise, but after years of working with Boost.Python I'm very familiar with the issues of dynamic linking and C/C++ interoperability on a wide variety of platforms, and I'm not convinced by your assertion. If such a system exists, it should be easy for someone to point me at it, and show that something breaks. > I just notice the irony of history: ccpython.cc was originally > introduced to better support C++ extension modules - now it might > get removed for the very same reason (to better support C++ > extension modules). Indeed. I was amazed to read in the log that it was introduced in 1995 for that reason. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GCC version compatibility
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >>>When I looked into this problem I saw that configure in fact builds a test >>>executable that included an object file compiled with g++. If the link step >>>with gcc succeeds then LINKCC is set as above, otherwise CXX is >>>used. Obviously, on my system this test was successful so configure decided >>>to link with gcc. However, minimal changes to the source of the test program >>>caused the link step to fail. It was not obvious to me at all why the latter >>>source code should cause a dependency on the C++ runtime if the original >>>code does not. My conclusion was that this test is fragile and should be >>>skipped. >> >> >> Sounds like it. I have never understood what the test was really >> checking for since the moment it was first described to me, FWIW. > > I'll describe it once more: *If* a program is compiled with the C++ > compiler, is it *then* possible to still link it with the C compiler? > This is the question this test tries to answer. Okay, I understand that. What I have never understood is why that should be an appropriate thing to test for the Python executable. There's no reason to compile any of Python with a C++ compiler. >>>If Python is built with --with-cxx then it should be linked with CXX >>>as well. >> >> U betcha. > > Wrong. The test was introduced in response to complaints that Python > unnecessarily links with libstdc++ on some Linux systems. Apparently it still does. > On these Linux systems, it was well possible to build main() with a > C++ compiler Nobody would need to build Python's main() with a C++ compiler, if you'd just comment out the 'extern "C"'. > and still link the entire thing with gcc. Since main() doesn't use > any libstdc++ functionality, and since collect2/__main isn't used, > one would indeed expect that linking with CXX is not necessary. It isn't. >>>(On ELF based Linux/x86, at least.) That leaves me wondering >>> >>> * when is --with-cxx really necessary? >> >> >> I think it's plausible that if you set sys.dlopenflags > > This has no relationship at all. --with-cxx is much older than > sys.dlopenflags. It is used on systems where main() must be a > C++ program for C++ extension modules to work (e.g. some Linux > systems). I have tested Boost.Python and C++ extension modules on a wide variety of Linux systems, and have seen this phenomenon. Everyone who is testing it on Linux is finding that if they build Python --without-cxx, everything works. And, yes, the mechanisms at the very *core* of Boost.Python rely on static initializers being run properly, so if there were anything wrong with that mechanism the tests would be breaking left and right. I think either the ELF Linux loader changed substantially since 1995, or whoever introduced this test was just confused. C++ extension modules get their static initializers run when they are loaded by dlopen (or, strictly speaking, sometime between then and the time their code begins to execute) which happens long after main or __main are invoked. The executable doesn't know about these extension modules until they are loaded, and when it loads them it can't get its hooks into anything other than the initmodulename entry point. The executable does not trigger the static initializers; the dynamic loader does. Therefore, it doesn't matter whether the executable is linked with the C++ runtime. An appropriate C++ runtime is linked to the extension module and that is what gets invoked when the module is loaded. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >>>If there is some library with such objects that happens to get >>>wrapped and dynamically linked into a Python interpreter >> >> >> Whoa there. How would that ever happen under ordinary circumstances? >> Doesn't Python's makefile control what gets linked to Python? > > Not entirely. By extending Modules/Setup You mean http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/Setup.dist?view=markup ? > e.g. in the way freeze works), it is well possible to have > additional extension modules linked into the Python interpreter, > extension modules which are not part of the standard Python > distribution. > > In fact, before Python supported dynamic loading of extension > modules, this was the *only* way to use non-standard extension > modules. You always had to build your own version of the Python > interpreter. I believe ccpython.cc dates back to these times. That explains a lot. I contend that either: a. Anyone making that sort of extension with a C++ module should explicitly request --with-cxx, or b. The python build system should automatically detect that --with-cxx is needed based on the presence of C++ extension modules. Frankly I think b. would require an impractical amount of work and, speaking as an author of C++ extension modules, I don't think a. is much of a burden to impose. >> If there's someone around here who is responsible for this change and >> knows its real rationale, can (s)he please tell me what it is? If >> not, can we please change things back so Python doesn't get linked to >> the C++ runtime by default? > > ccpython.cc and --with-cxx was first published in Python 1.6, and > hasn't changed much since. So for some of you, it has "always" been > there. It was contributed by Geoff Furnish. > > What *has* changed now is that the configure test suddenly > determines that you need to link with g++ on Linux if main was > compiled with g++. This was not the case before, but now is (since > g++ 3.2 or something). I see. Well, everything has become clear, thank you. My proposed remedy hasn't changed, though. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
Ulrich Berning <[EMAIL PROTECTED]> writes: > If you build C++ extensions on HP-UX with aCC, Python must be compiled > and linked as a C++ program. This is documented. You mean dynamically loaded C++ extensions, or the kind that are linked into the Python executable? I'm willing to believe almost anything about HP-UX. Until recently, aCC was so broken as a C++ compiler that there was little point in trying to get Boost.Python to work on it, and I don't have much data for that system. > It will not work if Python is compiled and linked as a normal C > program (I have tried it). Even if you take out the use of C++ constructs in ccpython.cc? I just need to check all the obvious angles. > I haven't tried gcc on this platform, but I guess it is the same > (compile and link with g++). Okay, but -- at the very least -- we don't need this behavior on ELF/Linux. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
Ulrich Berning <[EMAIL PROTECTED]> writes: > David Abrahams schrieb: > >>Ulrich Berning <[EMAIL PROTECTED]> writes: >> >> >> >>> If you build C++ extensions on HP-UX with aCC, Python must be >>> compiled and linked as a C++ program. This is documented. >>> >>> >> >>You mean dynamically loaded C++ extensions, or the kind that are >>linked into the Python executable? >> >> >> > Dynamically loaded extensions, especially SIP/PyQt > (http://www.riverbankcomputing.co.uk). I see. >>I'm willing to believe almost anything about HP-UX. Until recently, >>aCC was so broken as a C++ compiler that there was little point in >>trying to get Boost.Python to work on it, and I don't have much data >>for that system. >> >> >> > I'm using the HP aC++ Compiler C.03.50 together with the patches > PHSS_29483 and PHSS_30967 on HP-UX B.11.00 and had no problems to build > Python (2.3.5), Qt, SIP and PyQt and all other extensions with it. Yeah, but Boost tends to stress a C++ compiler's correctness much more than Qt. >>>It will not work if Python is compiled and linked as a normal C >>>program (I have tried it). >> >>Even if you take out the use of C++ constructs in ccpython.cc? I just >>need to check all the obvious angles. > > What do you mean? The only C++ construct in ccpython.cc is the extern > "C" declaration of Py_Main() That's what I mean. > and this is necessary if a C++ program > references symbols from a C library. HP says, that a C++ shared library > or a C++ shared object can only be loaded by a C++ main program. Well, that's dumb, but I believe it. > I can't remember the error message/symptoms, but I tried to build > Python using python.c and couldn't load any C++ extensions. Because > I'm going on vacation for the next three weeks, I can't try anything > on HP-UX at the moment. Okay, I'm convinced that on HP/UX, Python needs to be a C++ program. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: >> I don't believe any systems require it. I realize you have said >> otherwise, but after years of working with Boost.Python I'm very >> familiar with the issues of dynamic linking and C/C++ interoperability >> on a wide variety of platforms, and I'm not convinced by your >> assertion. If such a system exists, it should be easy for someone to >> point me at it, and show that something breaks. > > I well remember that gcc 2.5.8 on Linux a.out required this sort of > setup. Sorry, a.out? Isn't that the default name a C compiler gives to the executable it builds on Unix? Is it also (part of) the name of an OS? > Dynamic linking was not supported at all on that system (atleast > not in a way where users could easily write shared libraries > themselves). Rebuilding the Python interpreter was the only option > to integrate additional modules. Understood, and I retract my former incredulity. I believe HP-UX requires it, and I believe that some systems where you have to link in extension modules explicitly require it. But the "--with-cxx if configure says you can get away with it" behavior is hurting on a major platform: ELF Linux. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >>>Not entirely. By extending Modules/Setup >> >> >> You mean >> http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/Setup.dist?view=markup >> ? > > I mean Modules/Setup. It is generated from Modules/Setup.dist > (plus some additional information) in the build process. > >> I contend that either: >> >>a. Anyone making that sort of extension with a C++ module should >>explicitly request --with-cxx, or >> >>b. The python build system should automatically detect that >>--with-cxx is needed based on the presence of C++ extension >>modules. >> >> Frankly I think b. would require an impractical amount of work and, >> speaking as an author of C++ extension modules, I don't think a. is >> much of a burden to impose. > > It is the burden of change. Well, as you say: What *has* changed now is that the configure test suddenly determines that you need to link with g++ on Linux if main was compiled with g++. This was not the case before, but now is (since g++ 3.2 or something). > Contributions are welcome. Let me first try to discover what contribution would be acceptable. What if: - we add a configure test that runs after the existing test determines that --with-cxx is needed (but not when --with-cxx is explicitly specified on the command line) - This test runs a 'C' executable that tries to load a C++ dynamic library with dlopen. - The test returns an error code if the the dynamic library's static and dynamic initializers have not been run properly - If the test fails we disable --with-cxx ?? I'm trying to intrude on the existing behavior as little as possible, yet get the semantics we want for ELF/Linux in a way that stands a good chance of generalizing to other platforms. > However, you will find that with a), people will still pass --with-cxx, > because they tend to "enable" all features they can find. That's acceptable to me. We could probably circumvent some of those cases by improving the configure --help text. > I personally could accept --with-cxx and ccpython.cc to be removed > again, but I'm uncertain whether that may break distutils in some > way. Well, that would certainly be an easy "solution," but it would break HP/UX, and it would break anyone that needs to statically link C++ modules on some platforms. It's a much more drastic change than it would be to only have the system use --with-cxx when the person running configure asks for it explicitly. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GCC version compatibility
Christoph Ludwig <[EMAIL PROTECTED]> writes: > --with-cxx=: If you plan to use C++ extension modules, then on some > platform you need to compile python's main() function with the C++ > compiler. With this option, make will use to compile main() > *and* to link the python executable. It is likely that the resulting > executable depends on the C++ runtime library of . > > Note there are platforms that do not require you to build Python with > a C++ compiler in order to use C++ extension modules. E.g., x86 Linux > with ELF shared binaries and GCC 3.x, 4.x is such a platform. We > recommend that you configure Python --without-cxx on those platforms > to avoid unnecessary dependencies. I don't think that's strong enough. What happens is that dynamically loaded Python extension modules built with other, ABI-compatible versions of G++ may *crash*. > If you need to compile main() with , but your platform does > not require that you also link the python executable with > (e.g., ), then set LINKCC='$(PURIFY) $(CC)' prior to > calling make. Then the python executable will not depend on the C++ > runtime library of . Are we sure we have an actual use case for the above? Doesn't --without-cxx cover all the actual cases we know about? > BTW, I'd also change the short explanation output by `configure --help'. > Something like: > > AC_HELP_STRING(--with-cxx=, > use to compile and link main()) > > In Python 2.4.1, the help message says "enable C++ support". That made me use > this option even though it turned out it is not necessary on my platform. Your suggestion is simple and powerful; I like it! -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GCC version compatibility
Christoph Ludwig <[EMAIL PROTECTED]> writes: > I do not claim the 2 TUs test will cover all possible scenarios. I am not even > sure this decision should be left to an automated test. Because if the test > breaks for some reason then the user is left with a linker error that is > time-consuming to track down. However, at least by the usual hierarchy of values, the sort of runtime error that results from the current needless linking with C++ on ELF/Linux is even worse. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> - we add a configure test that runs after the existing test >> determines that --with-cxx is needed (but not when --with-cxx is >> explicitly specified on the command line) >> >> - This test runs a 'C' executable that tries to load a C++ dynamic >> library with dlopen. >> >> - The test returns an error code if the the dynamic library's static >> and dynamic initializers have not been run properly >> >> - If the test fails we disable --with-cxx >> >> ?? > > What would be the purpose of such a test? The test may fail for many > reasons, for example, dlopen might not be available. I was assuming we only disable --with-cxx if the test builds successfully. I presume dlopen will fail linking if it's unavailable? > OTOH, on current Linux systems the test would succeed, so configure > would conclude to link with g++. Uhh, whoops. Change the sense of my last bullet - If the test passes we disable --with-cxx >> I'm trying to intrude on the existing behavior as little as possible, >> yet get the semantics we want for ELF/Linux in a way that stands a >> good chance of generalizing to other platforms. > > I now think that the code should be made more simple, not more complex. > Aspects of a solution I could accept: > > - ccpython.cc and linking with g++ is removed entirely. or, That would be bad for C++ users on HP/UX. Is that acceptable? > - the logic is fixed so that linking with g++ is only done if > main is in ccpython.o I don't see how that works. Somehow we need to decide whether to put main in ccpython.o in the first place, don't we? > - the configure test is extended to better match current g++ > behaviour. What do you have in mind? >> Well, that would certainly be an easy "solution," but it would break >> HP/UX, and it would break anyone that needs to statically link C++ >> modules on some platforms. It's a much more drastic change than it >> would be to only have the system use --with-cxx when the person >> running configure asks for it explicitly. > > I just checked, and it seems that the logic in use is still somewhat > different. If the configure test determines that a C++ main() > must be linked with CXX, it unconditionally changes the linker to CXX. > The test, in turn, is run always if a C++ compiler was found, > i.e. independently of whether --with-cxx was provided. That doesn't match up with reports from my testers who say they can run with C++ extension modules from many different GCC versions if they just configure their Python --without-cxx. If what you were saying were true, wouldn't --without-cxx be ignored on ELF/Linux? -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: >>>- the logic is fixed so that linking with g++ is only done if >>> main is in ccpython.o >> >> >> I don't see how that works. Somehow we need to decide whether to put >> main in ccpython.o in the first place, don't we? > > Yes, that is done through --with-cxx (alone). However, the decision > to use CXX for linking is independent on whether --with-cxx was > given. Is the above a description of existing behavior or a behavior you're proposing? > >>>- the configure test is extended to better match current g++ >>> behaviour. >> >> >> What do you have in mind? > > Somebody reported that the test works better for g++ if the > function is marked extern "C". Which function? What does "works better" mean? > This should be done for 2.4 regardless of any other change. > >>>I just checked, and it seems that the logic in use is still somewhat >>>different. If the configure test determines that a C++ main() >>>must be linked with CXX, it unconditionally changes the linker to CXX. >>>The test, in turn, is run always if a C++ compiler was found, >>>i.e. independently of whether --with-cxx was provided. >> >> >> That doesn't match up with reports from my testers who say they can >> run with C++ extension modules from many different GCC versions if >> they just configure their Python --without-cxx. If what you were >> saying were true, wouldn't --without-cxx be ignored on ELF/Linux? > > Ok, it's still different. I see three cases now: > 1. --without-cxx or --with-cxx=no specified. configure does not look >for a C++ compiler, and does not check whether linking needs >to be done with a C++ compiler, and decides to use Modules/python.c. > 2. --with-cxx not given. configure does look for a C++ compiler, >and does check whether linking with the C++ compiler is necessary, >and still uses Modules/python.c > 3. --with-cxx is given. configure requires it to point to a C++ >compiler, performs the linking check, and uses Modules/ccpython.cc. Is the above a description of existing behavior or is it a behavior you're proposing? > It would help discussion if you would use the actual code, too, > instead of just using reports from your testers. I don't know what you mean by "use the actual code." Do you mean, refer to the actual code in the discussion, or do you mean actually building and running Python --without-cxx myself? If the latter, I don't see a reason to repeat what people I trust have already done. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >>>>I don't see how that works. Somehow we need to decide whether to put >>>>main in ccpython.o in the first place, don't we? >>> > > You wouldn't have to ask these questions if you had investigated the > answers yourself. The questions should have been more precisely phrased as, "Do you mean to say ?" Since your statements about the code have not always been accurate (not blaming you; everyone makes mistakes) I'd still need to know how you intend your statements to be interpreted, not only how they correlate with the code. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [C++-sig] GCC version compatibility
Christoph Ludwig <[EMAIL PROTECTED]> writes: > I submitted patch #1239112 that implements the test involving two TUs for > Python 2.4. I plan to work on a more comprehensive patch for Python 2.5 but > that will take some time. Thanks very much for your efforts, Christoph! -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] MinGW and libpython24.a
Is the instruction at http://www.python.org/dev/doc/devel/inst/tweak-flags.html#SECTION000622000 still relevant? I am not 100% certain I didn't make one myself, but it looks to me as though my Windows Python 2.4.1 distro came with a libpython24.a. I am asking here because it seems only the person who prepares the installer would know. If this is true, in which version was it introduced? Thanks, -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] MinGW and libpython24.a
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> Is the instruction at >> http://www.python.org/dev/doc/devel/inst/tweak-flags.html#SECTION000622000 >> still relevant? I am not 100% certain I didn't make one myself, but >> it looks to me as though my Windows Python 2.4.1 distro came with a >> libpython24.a. I am asking here because it seems only the person who >> prepares the installer would know. > > That impression might be incorrect: I can tell you when I started > including libpython24.a, but I have no clue whether the instructions > you refer to are correct - I don't use the file myself at all. > >> If this is true, in which version was it introduced? > > It was introduced in 1.20/1.16.2.4 of Tools/msi/msi.py in response to > patch #1088716; this in turn was first used to release r241c1. Thanks! -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [Docs] MinGW and libpython24.a
David Abrahams <[EMAIL PROTECTED]> writes: > "Martin v. Löwis" <[EMAIL PROTECTED]> writes: > >> David Abrahams wrote: >>> Is the instruction at >>> http://www.python.org/dev/doc/devel/inst/tweak-flags.html#SECTION000622000 >>> still relevant? I am not 100% certain I didn't make one myself, but >>> it looks to me as though my Windows Python 2.4.1 distro came with a >>> libpython24.a. I am asking here because it seems only the person who >>> prepares the installer would know. >> >> That impression might be incorrect: I can tell you when I started >> including libpython24.a, but I have no clue whether the instructions >> you refer to are correct - I don't use the file myself at all. >> >>> If this is true, in which version was it introduced? >> >> It was introduced in 1.20/1.16.2.4 of Tools/msi/msi.py in response to >> patch #1088716; this in turn was first used to release r241c1. > > Thanks! As it turns out, MinGW also implemented, in version 3.0.0 (with binutils-2.13.90-20030111-1), features which make the creation of libpython24.a unnecessary. So whoever maintains this doc might want to note that you only need that step if you are using a version of Python prior to 2.4.1 with a MinGW prior to 3.0.0 (with binutils-2.13.90-20030111-1). Regards -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Docs] MinGW and libpython24.a
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> As it turns out, MinGW also implemented, in version 3.0.0 (with >> binutils-2.13.90-20030111-1), features which make the creation of >> libpython24.a unnecessary. So whoever maintains this doc might want >> to note that you only need that step if you are using a version of >> Python prior to 2.4.1 with a MinGW prior to 3.0.0 (with >> binutils-2.13.90-20030111-1). > > Can you please provide a patch to the documentation? None of the > regular documentation maintainers would know what exactly to write; > this is all user-contributed. This isn't rocket science. Or maybe it is; if adding These instructions only apply if you're using a version of Python prior to 2.4.1 with a MinGW prior to 3.0.0 (with binutils-2.13.90-20030111-1) is not acceptable then no patch I could submit would be acceptable, because I don't know how to do better either. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [C++-sig] GCC version compatibility
Christoph Ludwig <[EMAIL PROTECTED]> writes: > Hi, > > this is to continue a discussion started back in July by a posting by > Dave Abrahams http://thread.gmane.org/gmane.comp.python.devel/69651> > regarding the compiler (C vs. C++) used to compile python's main() and to link > the executable. > > > On Sat, Jul 16, 2005 at 12:13:58PM +0200, Christoph Ludwig wrote: >> On Sun, Jul 10, 2005 at 07:41:06PM +0200, "Martin v. Löwis" wrote: >> > Maybe. For Python 2.4, feel free to contribute a more complex test. For >> > Python 2.5, I would prefer if the entire code around ccpython.cc was >> > removed. >> >> I submitted patch #1239112 that implements the test involving two TUs for >> Python 2.4. I plan to work on a more comprehensive patch for Python 2.5 but >> that will take some time. > > > I finally had the spare time to look into this problem again and submitted > patch #1324762. The proposed patch implements the following: I just wanted to write to encourage some Python developers to look at (and accept!) Christoph's patch. This is really crucial for smooth interoperability between C++ and Python. Thank you, Dave -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Plea to distribute debugging lib
For years, Boost.Python has been doing some hacks to work around the fact that a Windows Python distro doesn't include the debug build of the library. http://www.boost.org/libs/python/doc/building.html#variants explains. We wanted to make it reasonably convenient for Windows developers (and our distributed testers) to work with a debug build of the Boost.Python library and of their own code. Having to download the Python source and build the debug DLL was deemed unacceptable. Well, those hacks have run out of road. VC++8 now detects that some of its headers have been #included with _DEBUG and some without, and it will refuse to build anything when it does. We have several new hacks to work around that detection, and I think we _might_ be able to get away with them for one more release. But it's really time to do it right. MS is recommending that we (Boost) start distributing a debug build of the Python DLL with Boost, but Boost really seems like the wrong place to host such a thing. Is there any way Python.org can make a debug build more accessible? Thanks, Dave -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Plea to distribute debugging lib
Tim Peters <[EMAIL PROTECTED]> writes: > [David Abrahams] >> For years, Boost.Python has been doing some hacks to work around the >> fact that a Windows Python distro doesn't include the debug build of >> the library. >> ... >> MS is recommending that we (Boost) start distributing a debug build of the >> Python DLL with Boost, but Boost really seems like the wrong place to host >> such a thing. Is there any way Python.org can make a debug build more >> accessible? > > Possibly. I don't do this anymore (this == build the Python Windows > installers), but I used to. For some time I also made available a zip > file containing various debug-build bits, captured at the time the > official installer was built. We didn't (and I'm sure we still don't) > want to include them in the main installer, because they bloat its > size for something most users truly do not want. > > I got sick of building the debug zip file, and stopped doing that too. > No two users wanted the same set of stuff in it, so it grew to > contain the union of everything everyone wanted, and then people > complained that it was "too big". This is one of the few times in > your Uncle Timmy's life that he said "so screw it -- do it yourself, > you whiny baby whiners with your incessant baby whining you " ;-) > > Based on that sure-to-be universal reaction from anyone who signs up > for this, I'd say the best thing you could do to help it along is to > define precisely (a) what an acceptable distribution format is; and, > (b) what exactly it should contain. Who knows what the whiny babies will accept? That said, I think people would be happy with a .zip file containing whatever is built by selecting the debug build in the VS project and asking it to build everything. (**) > That, and being nice to Martin, I'm always as nice as Davidly possible to Martin! > would go a long way. My fingers and toes are crossed. Thanks! (**) If you could build the ability to download the debugging binaries into the regular installer, that would be the shiznit, but I don't dare ask for it. ;-) -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Plea to distribute debugging lib
Bronek Kozicki <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> Who knows what the whiny babies will accept? That said, I think >> people would be happy with a .zip file containing whatever is built by >> selecting the debug build in the VS project and asking it to build >> everything. (**) > > Just to clarify - what we are asking for is library built with _DEBUG > and no BOOST_DEBUG_PYTHON, that is the one compatible with default > Python distribution. Bronek, I know you're trying to help, but I'm sure that's not making anything clearer for these people. They don't know anything about BOOST_DEBUG_PYTHON and would never have cause to define it. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Plea to distribute debugging lib
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> Who knows what the whiny babies will accept? That said, I think >> people would be happy with a .zip file containing whatever is built by >> selecting the debug build in the VS project and asking it to build >> everything. (**) > > I would go a step further than Tim: Send me (*) a patch to msi.py (which > is used to build the distribution) that picks up the files and packages > them in the desired way, and I will include the files it outputs > in the official distribution. This is how the libpython24.a got in > (and this is also the way in which it will get out again). Not to look a gift horse in the mouth, but won't that cause the problem that Tim was worried about, i.e. a bloated Python installer? > In the patch, preferably state whom to contact for the specific feature, > as I won't be able to answer questions about it. > > I don't have a personal need for the feature (I do have debug builds > myself, and it takes only 10 minutes or so to create them), I know, me too. It's easy enough once you get started building Python. I just think it's too big a hump for many people. > so I won't even have a way to test whether the feature works > correctly. > > Regards, > Martin > > (*) that is, sf.net/projects/python I s'pose that means, "put it in the patches tracker." grateful-ly y'rs, -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Plea to distribute debugging lib
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >>> Just to clarify - what we are asking for is library built with >>> _DEBUG and no BOOST_DEBUG_PYTHON, that is the one compatible with >>> default Python distribution. >> I know you're trying to help, but I'm sure that's not making >> anything >> clearer for these people. They don't know anything about >> BOOST_DEBUG_PYTHON and would never have cause to define it. >> > > Actually, I'm truly puzzled. I was afraid this would happen. Really, you're better off ignoring Bronek's message. > Why would a library that has _DEBUG defined > be compatible with the standard distribution? Doesn't _DEBUG cause > linkage with msvcr71d.dll? Unless you do the hacks that I mentioned in my opening message. Read http://www.boost.org/libs/python/doc/building.html#variants for details. > In addition (more correctly: for that reason), the debug build causes > python2x_d.dll to be build, instead of python2x.dll, which definitely > is incompatible with the standard DLL. It not only uses a different > C library; it also causes Py_DEBUG to be defined, which in turn creates > a different memory layout for PyObject. Exactly. > So in the end, I would assume you are requesting what you call a > debug-python, i.e. one that (in your system) *has* > BOOST_DEBUG_PYTHON defined. What I am requesting is the good old python2x_d.dll and any associated extension modules that get built as part of the Python distro, so I can stop doing the hack, drop BOOST_DEBUG_PYTHON, and tell people use _DEBUG in the usual way. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Plea to distribute debugging lib
Trent Mick <[EMAIL PROTECTED]> writes: > [Thomas Heller wrote] >> Anyway, AFAIK, the activestate distribution contains Python debug dlls. > > [Er, a month late, but I was in flitting around Australia at the time. :)] > > Yes, as a separate download. > > ftp://ftp.activestate.com/ActivePython/etc/ > ActivePython--win32-ix86-debug.zip > > And those should be binary compatible with the equivalent python.org > installs as well. Note that the simple "install.py" script in those > packages bails if the Python installation isn't ActivePython, but I > could easily remove that if you think that would be useful for your > users. Yes, please! Would Python.org be willing to post links to the Activestate package? That would help, too. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] How to get the Python-2.4.2 sources from SVN?
It isn't completely clear which branch or tag to get, and Google turned up no obvious documentation. Thanks, -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How to get the Python-2.4.2 sources from SVN?
Thomas Wouters <[EMAIL PROTECTED]> writes: > On Sat, Feb 11, 2006 at 09:10:41AM -0600, [EMAIL PROTECTED] wrote: > >> Dave> It isn't completely clear which branch or tag to get, and Google >> Dave> turned up no obvious documentation. > >> On subversion, you want releaseXY-maint for the various X.Y releases. For >> 2.4.2, release24-maint is what you want, though it may have a few bug fixes >> since 2.4.2 was released. With CVS I used to use "cvs log README" to see >> what all the tags and branches were. I don't know what the equivalent svn >> command is. > > The 'cvs log' trick only works if the file you log is actually part of the > branch. Not an issue with Python or any other project that always branches > sanely, fortunately, but there's always wackos out there ;) > You get the list of branches in SVN with: > > svn ls http://svn.python.org/projects/python/branches/ > > And similarly, tags with: > > svn ls http://svn.python.org/projects/python/tags Yes, that's easy enough, but being sure of the meaning of any given tag or branch name is less easy. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How to get the Python-2.4.2 sources from SVN?
"Martin v. Löwis" <[EMAIL PROTECTED]> writes: > David Abrahams wrote: >> It isn't completely clear which branch or tag to get, and Google >> turned up no obvious documentation. > > http://svn.python.org/projects/python/tags/r242/ Thanks. -- Dave Abrahams Boost Consulting www.boost-consulting.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
