Re: [Rd] calloc() vs. R_Calloc()
On 4/7/22 08:59, Adrian Dușa wrote: Dear R devs, I ran into a C level problem that hopefully is a quick fix to a trained eye. Not sure if I am able to produce a minimal reproducible example, but suppose a (test) package passes all the local tests, passes the R CMD check and also passes the tests on both rhub and https://win-builder.r-project.org The test GitHub repo is here: https://github.com/dusadrian/QCAtest As per the latest CRAN recommendation, I need to replace calloc() and free() with R's Calloc() and Free(). In the above repo, the latest commit does not pass the tests any longer, and the absolute single difference is using the recommended commands in the file: https://github.com/dusadrian/QCAtest/blob/main/src/CCubes.c The previous initial commit of this file (which has no problems) is here: https://github.com/dusadrian/QCAtest/blob/5fb13f44457a2071b322ad42ab579f3547d1c551/src/CCubes.c There must be something obvious I'm doing wrong, perhaps a header missing or not in the right place, but it just escapes me. I've tried both Calloc() and R_Calloc(), both versions result in the same errors. And what are the errors you run into? On which platforms, under what circumstances, etc? It would be much easier to give advice knowing that. In principle, one issue you may run into when switching allocators is that you accidentally end up freeing by a different allocator from the one used to allocate it. It is common on Windows but can in principle happen elsewhere as well. Also by a slightly different heap layout or different allocator implementation you may wake up bugs in the program not seen previously (use after free, invalid memory accesses, etc) Tomas Many thanks in advance for any hint, Adrian [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] calloc() vs. R_Calloc()
On 4/7/22 10:32, Adrian Dușa wrote: > On Thu, 7 Apr 2022 at 10:32, Tomas Kalibera > wrote: > > > [...] > > And what are the errors you run into? On which platforms, under what > circumstances, etc? It would be much easier to give advice knowing > that. > > In principle, one issue you may run into when switching allocators is > that you accidentally end up freeing by a different allocator from > the > one used to allocate it. It is common on Windows but can in principle > happen elsewhere as well. > > Also by a slightly different heap layout or different allocator > implementation you may wake up bugs in the program not seen > previously > (use after free, invalid memory accesses, etc) > > > That is something I do not know yet, since the only information the > server gives is this: > https://builder.r-hub.io/status/original/QCA_3.16.tar.gz-a03b4462b41df37c6284be1d5519e8b3 > > I'll probably end up debugging the C code, but since the only > difference is using Free() vs free() on exactly the same objects, I > suspected a mis-usage of the R commands. > > The same setup passes with no problems on my local MacOS, but the > errors still seem to occur on the Windows setup from r-hub. This is very likely because you are freeing memory allocated by calloc() (or something else but not R_Calloc() in your program) using R_Free() or memory allocated using R_Calloc() by using free() in your program. I would recommend checking the source code manually for that. It is not surprising that the problem doesn't appear on other platforms where the allocators happen to be the same. Best Tomas > > Best wishes, > Adrian [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] calloc() vs. R_Calloc()
Hello, Tomas is right, there are many memory operations allocating with malloc and freeing with R_Free: line: variables - operation 55: p_pichart - malloc 336: p_pichart - R_Free 236: copy_implicants - malloc 260: copy_implicants - R_Free 240: p_tempic - malloc 262: p_tempic - R_Free And there are more cases like these. For what I've seen, the frequent (unique?) case is malloc/R_Free. Hope this helps, Rui Barradas Às 09:59 de 07/04/2022, Tomas Kalibera escreveu: On 4/7/22 10:32, Adrian Dușa wrote: On Thu, 7 Apr 2022 at 10:32, Tomas Kalibera wrote: [...] And what are the errors you run into? On which platforms, under what circumstances, etc? It would be much easier to give advice knowing that. In principle, one issue you may run into when switching allocators is that you accidentally end up freeing by a different allocator from the one used to allocate it. It is common on Windows but can in principle happen elsewhere as well. Also by a slightly different heap layout or different allocator implementation you may wake up bugs in the program not seen previously (use after free, invalid memory accesses, etc) That is something I do not know yet, since the only information the server gives is this: https://builder.r-hub.io/status/original/QCA_3.16.tar.gz-a03b4462b41df37c6284be1d5519e8b3 I'll probably end up debugging the C code, but since the only difference is using Free() vs free() on exactly the same objects, I suspected a mis-usage of the R commands. The same setup passes with no problems on my local MacOS, but the errors still seem to occur on the Windows setup from r-hub. This is very likely because you are freeing memory allocated by calloc() (or something else but not R_Calloc() in your program) using R_Free() or memory allocated using R_Calloc() by using free() in your program. I would recommend checking the source code manually for that. It is not surprising that the problem doesn't appear on other platforms where the allocators happen to be the same. Best Tomas Best wishes, Adrian [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] calloc() vs. R_Calloc()
Adrian, there are many more allocations in your code - for example, you're using resize() from utils.c which uses malloc/free so it will break since it's using the wrong allocator. You may need to replace *all* allocations in your project, not just some. Cheers, Simon > On 8/04/2022, at 6:45 AM, Adrian Dușa wrote: > > Dear Rui, > > Thank you so much for the response, that's a very good observation but I've > just tested and results in the same errors, unfortunately. > > I have now changed all malloc()s to R_Calloc(), in the latest commit: > https://github.com/dusadrian/QCAtest/blob/main/src/CCubes.c > > And yet: > https://builder.r-hub.io/status/QCA_3.16.tar.gz-e652c850de7e51c4cc1311fafe44f986 > > There must be something obvious I am missing, but my C knowledge is limited > and already stretched out... > > Best wishes, > Adrian > > On Thu, 7 Apr 2022 at 18:49, Rui Barradas wrote: > >> Hello, >> >> Tomas is right, there are many memory operations allocating with malloc >> and freeing with R_Free: >> >> line: variables - operation >> >> 55: p_pichart - malloc >> 336: p_pichart - R_Free >> >> 236: copy_implicants - malloc >> 260: copy_implicants - R_Free >> >> 240: p_tempic - malloc >> 262: p_tempic - R_Free >> >> >> And there are more cases like these. For what I've seen, the frequent >> (unique?) case is malloc/R_Free. >> >> Hope this helps, >> >> Rui Barradas >> >> Às 09:59 de 07/04/2022, Tomas Kalibera escreveu: >>> >>> On 4/7/22 10:32, Adrian Dușa wrote: On Thu, 7 Apr 2022 at 10:32, Tomas Kalibera wrote: [...] And what are the errors you run into? On which platforms, under >> what circumstances, etc? It would be much easier to give advice knowing that. In principle, one issue you may run into when switching allocators >> is that you accidentally end up freeing by a different allocator from the one used to allocate it. It is common on Windows but can in >> principle happen elsewhere as well. Also by a slightly different heap layout or different allocator implementation you may wake up bugs in the program not seen previously (use after free, invalid memory accesses, etc) That is something I do not know yet, since the only information the server gives is this: >> https://builder.r-hub.io/status/original/QCA_3.16.tar.gz-a03b4462b41df37c6284be1d5519e8b3 I'll probably end up debugging the C code, but since the only difference is using Free() vs free() on exactly the same objects, I suspected a mis-usage of the R commands. The same setup passes with no problems on my local MacOS, but the errors still seem to occur on the Windows setup from r-hub. >>> >>> This is very likely because you are freeing memory allocated by calloc() >>> (or something else but not R_Calloc() in your program) using R_Free() or >>> memory allocated using R_Calloc() by using free() in your program. >>> I would recommend checking the source code manually for that. >>> >>> It is not surprising that the problem doesn't appear on other platforms >>> where the allocators happen to be the same. >>> >>> Best >>> Tomas >>> >>> Best wishes, Adrian >>> [[alternative HTML version deleted]] >>> >>> __ >>> R-devel@r-project.org mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-devel >> > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
