Re: Problem with msvcrt60 vs. msvcr71 vs. strdup/free

[abkhd]

From: "" <[EMAIL PROTECTED]>
Newsgroups: comp.lang.python
Subject: Re: Problem with msvcrt60 vs. msvcr71 vs. strdup/free
Date: Wed, 22 Dec 2004 11:07:02 -0800

Gerhard Haering wrote:
> Hello,
>
> I used to build Python extension modules with mingw. Now, Python has
> switched to the MSVCR71 runtime with version 2.4, and I thought mingw
> has support for this. But I get problems with symbols being
referenced
> from the wrong DLLs.
>
> You can see the problem by compiling this:
>
> ##
> #include 
>
> int main()
> {
> char* s;
> int i;
>
>
> for (i = 0; i < 10; i++) {
> s = strdup("foo");
> free(s);
> }
>
> return 0;
> }
> ##
>
> with gcc x.c -lmsvcr71
>
> Then if you run a.exe it crashes.
>
> If you use depends.exe on it, you see that it resolves strdup() via
> msvcrt, but the rest with msvcr71.dll. That's why strdup() is using
> the one malloc, but free() a different free() from the other DLL,
> which is undoubtedly the reason for the crash.
>
> Is there any way I can force mingw to not link in msvcr for things
> like strdup?


Hello


Believe it or not I do not get a crash here.

Using objdump on a.exe I get:

[CUT]

50145084   52b8 5100

DLL Name: msvcrt.dll
vma:  Hint/Ord Member-Name Bound-To
51b4   81  _strdup

50285090   52f8 510c

DLL Name: msvcr71.dll
vma:  Hint/Ord Member-Name Bound-To
51c0   59  __getmainargs
51d0   82  __p__environ
51e0  105  __set_app_type
51f4  154  _cexit
5200  211  _fileno
520c  222  _fmode
5218  225  _fpreset
5224  273  _iob
522c  390  _onexit
5238  435  _setmode
5244  599  atexit
5250  634  free
5258  715  signal

503c50cc   5314 5148

DLL Name: msvcrt.dll
vma:  Hint/Ord Member-Name Bound-To
5264  510  abort
526c  537  fflush
5278  546  fprintf
5284  603  malloc

5050    


[CUT]


Having said that, let me add that I know where you are coming from. And
even though your a.exe did not crash with me now, I know that other
dlls and programs, in my case, did.

And let it be known that unfortunately the measures suggested by the
good people here do not always work. Changing the entry in the specs
file works most  of the time, but not on mmap.pyd (of the official
Python source distribution) for example, where a call to some _lseek of
some sort (I forgot the exact call) is still linked from msvcrt.dll not
from msvcr71. And yes the test_mmap crashes if I compile it that way.

Also changing the specs file seems to work only when the linking is
simple. But if the linking gets complex (as in linking many objects
files to build, say, pythonXY.dll) then even the specs change will not
help. To illustrate, and in trying to get the pyMinGW built Python to
link with msvcr71 that way, one ends up with a pythonXY.dll that
references both runtime libraries: msvcrt, and msvcr71 with strange
consequences. This particular version of Python crashes on  site import
hallucinating about (and I paraphrase!) "Null result from PY...XXX
without Error."

On MSDN, it is stated: "If your DLLs pass CRT resources across the
msvcrt.dll and msvcr71.dll boundary, you will encounter issues with
mismatched CRTs and need to recompile your project with Visual C++
.NET." [1]

I have a gut feeling that "DLLs" here does not mean inter-dll passing
of resources, but intra-dll passing of resources. In other words that
if the self-same DLL uses both run-times then you are in trouble; but I
don't think that this relates to the situation where one DLL is using
msvcrt and another DLL is using msvcr71. Please bear with me.

In this light, I truly don't know why would we try, or recommend
others, to link with libmsvcr71.a, if some mix-up is still possible. I
have yet to see a crash occur when Python24.dll or an extension of the
same uses the old way of  linking with libmsvcrt.a. That way we are
assured that no references are made to msvcr71 when msvcrt is the one
used when linking.

I could be mistaken, but I think people are confusing the behavior of
the MS compilers with MinGW's. The old MS compiler (V. 6) uses
msvcrt.lib by default, and even when the newer MSVS uses the same name
for the its default runtime, that msvcrt link implicitly means the
newer msvcr71 [2]. This behind the scenes behavior is what seems to
have people mixed up. You see if one has older projects compiled using
V. 6, then linking these object files to other object files compiled
with the newer MSVS could effectively mean that the resulting DLL will
have references to both run-times, something by the way is also
mentioned in the MSDN resources [3].

But as far as I know this is not the case with regards to MinGW. MinGW
explicitly uses msvcrt, and so old and newer projects should in theory
have no problems if they stick to msvcrt, as opposed to going th

Re: Problem with msvcrt60 vs. msvcr71 vs. strdup/free

[abkhd]

Martin v. Löwis wrote:
> Indeed, it *will* work most of the time, as you rarely pass resources
> across CRTs in a typical Python extension. However, it is very hard
> to tell from the source code of the extension if such resource
passing
> could ever happen (e.g. what about setlocale(), atexit(), malloc(),
> ...), so if you need something better than "seems to work", you
really
> need to make sure that you understand all the issues, and have a
build
> process that resolves them all.
>
> Regards,
> Martin




You know if I had not tried the example you gave, I'd probably still be
arguing: "but it works." Oh well. Not anymore.

For those who are interested, here is a patch to demo.c of the
\python\dist\src\Demo\embed\demo.c demonstrating what Martin has argued
about above. With such evidence at hand I guess it is safe to say that
the MinGW die-hards will have to improvise. :)

But seriously, and since I don't have the official Python 2.4
distribution yet, and since the pyMinGW compiled Python I use relies on
msvcrt, as opposed to msvcr71, I needed to reverse the roles here to
see the problem first hand: making the "extension" demo use msvcr71 to
see how it reacts with the home-made python24.dll which uses msvcrt.
And sure enough: crash.

Having said that, this seems to me not to be an issue for those who
have or can build Python and/or the extensions they need from sources.
Indeed recompiling the same "demo" to use msvcrt (to use the same
run-time library that my python24.dll uses) instead of msvcr71 yields
the expected correct results without crashes.

And so I suspect this approach (compiling sources yourself) can shield
people from this ugly mess, if they can build Python 2.4 and the
extensions and have them (core and extension) use one and the same
run-time library. But indeed this might be an issue for those who [1]
use MinGW as their first choice and [2] cannot or won't bother to build
their Python and/or extensions from sources, and hence have to rely on
binary distributions.


Regards
Khalid



--- demo.c  Sat Nov 17 08:30:20 2001
+++ demo2.c Thu Dec 23 02:02:40 2004
@@ -34,6 +34,20 @@
/* Note that you can call any public function of the Python
interpreter here, e.g. call_object(). */

+FILE *in;
+int sts;
+
+if ((in = fopen(argv[1], "rt")) == NULL) {
+  fprintf(stderr, "Cannot open program file.\n");
+  return 1;
+}
+
+printf("Working on %s\n", argv[1]);
+sts = PyRun_AnyFile(in, argv[1]) != 0;
+printf("sts: %d\n", sts);
+fclose(in);
+
+
/* Some more application specific code */
printf("\nGoodbye, cruel world\n");

--
http://mail.python.org/mailman/listinfo/python-list