[R-pkg-devel] Use of long double in configuration
Is it correct to say that R's conditional use of long double is around ensuring things work on platforms which have 'long double' identical to 'double' types, as opposed to there being an odd compiler targeted that does not even have any concept of 'long double' type? As background this was motivated by a query raised in the matrixStats package: https://github.com/HenrikBengtsson/matrixStats/issues/278 __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Problem building package depending on third party c++ source library.
Le 29/04/2025 à 14:43, Nils Lüschow a écrit : Hello everybody, as a new feature for the anticlust package (https://cran.r-project.org/ web/packages/anticlust/index.html) we are trying to include a constraint programming model, written in C++ utilizing the Gecode solver. While we generally got this feature to work when Gecode is pre-installed on the system, we would like to spare the users this step and include it with the package (or inside a supplementary package) to install through the regular R tooling. However, we run into a problem when it comes to compiling Gecode during the packages build-process, which I describe in this SO post: https://stackoverflow.com/questions/79596214/configure-script-does-not- generate-makefile-properly-for-3rd-party-library-used I strongly suspect the Makevars file to be incorrect, if you want to take a look or try to compile it yourself, I have uploaded the code of the new feature as well as the source files of the Gecode library to GitHub: https://github.com/nilslueschow/Anticlust-Dispersion-Constraint-Model/ tree/main/RcppTestPackage/src I don't know if it is the source of your problem but it's safer to use CC and CXX variables provided by R building process (they are already predefined when you are in Makevars) which minimize the risks of incompatibility between R and your library. So instead of cd $(GECODE_DIR) && $(MAKE) I'd use something like $(MAKE) -C $(GECODE_DIR) CC="$(CC)" CXX="$(CXX)" CXXFLAGS="$(CXXFLAGS)" Best, Serguei. Thanks for your help! Best, Nils Lüschow __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Use of long double in configuration
On 4/30/25 10:43, Tim Taylor wrote: Cheers for the quick response. To clarify my question: Is it correct to say that as long as packages do not assume the greater precision provided by 'double' there is no reason they cannot use 'long double' to get *possible* advantages (e.g. in summations). AFAICT 'long double' is (and has always been) part of the C standard so it's use as a type should be unproblematic (this is the query relevant to matrixStats). Probably already clear from previous answers, but yes, packages can use long double type. Whenever using a long double type, one needs to be careful about making sure the algorithms work, and the tests pass (so have reasonable tolerances), even when the long double type happens to be just the same as double. This is the case on aarch64, and macOS/aarch64 is one of the platforms where packages have to work, anyway, so this shouldn't be too limiting anymore - but really one needs to test on such platform. R itself has an option to disable use of long double to make such testing in R itself possible also on other platforms. In principle one could do something similar in a package, have some ifdefs to disable long doubles, but this is not required. And I probably wouldn't do that, I'd just test on aarch64 regularly. See Writing R Extensions for more details. Best Tomas Apologies if this does not make much sense. Tim On Wed, 30 Apr 2025, at 9:33 AM, Uwe Ligges wrote: On 30.04.2025 10:25, Tim Taylor wrote: Is it correct to say that R's conditional use of long double is around ensuring things work on platforms which have 'long double' identical to 'double' types, as opposed to there being an odd compiler targeted that does not even have any concept of 'long double' type? a double is 64 bit and stored that way on all platforms, the concept of long doubles is CPU specific. x86 chips have 80bit in the floating point units for calculations before rounding (and normalizing) to a regular double. Some chips, e.g. those ARM chips used in current M[123]Macs (hence very relevant topic), do not support long doubles. And compilers offer to compile without support for long doubles which e.g. CRAN uses to check in an additional (issues) check. Best, Uwe Ligges As background this was motivated by a query raised in the matrixStats package: https://github.com/HenrikBengtsson/matrixStats/issues/278 __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel Attachments: * smime.p7s __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Problem building package depending on third party c++ source library.
В Tue, 29 Apr 2025 14:43:40 +0200 Nils Lüschow пишет: > While we generally got this feature to work when Gecode is > pre-installed on the system, we would like to spare the users this > step and include it with the package It's probably a good idea to try to achieve both. If a system copy of the package is available, link with it; otherwise build the bundled copy. Unfortunately, this kind of user convenience involves a lot of testing to make sure that both configurations work as intended. Bundling dependencies also creates headaches for distro packagers. > However, we run into a problem when it comes to compiling Gecode > during the packages build-process, which I describe in this SO post One problem is that when using $(MAKE) to compile gecode, various variables used by gecode's Makefile are overridden through the MAKEFLAGS variable. For example, $(CXX) ends up being defined as $(CXX20) $(CXX20STD) without the variables CXX20 and CXX20STD being available to the downstream Make process. What's worse is CXXFLAGS are also overridden, losing the include paths in the process. In order to build gecode from Makevars, you'll have to disable these overrides by emptying MAKEFLAGS: build_gecode: cd $(GECODE_DIR) && \ ./configure CC="$(CC)" CXX="$(CXX20) $(CXX20STD)" \ --disable-examples --enable-static --disable-shared $(MAKE) -C $(GECODE_DIR) MAKEFLAGS= (Alternatively, you can ./configure and build the library from the ./configure script of your package.) Other changes are also needed to make the package build in a portable manner: - Recipe lines in Makefiles must start with tabs, not groups of four spaces. - Don't use GNU Make append syntax (+=) without declaring SystemRequirements: GNU Make. - Don't set compiler-specific flags such as -mtune=native without at least testing that the compiler understands them. - With the current directory structure, GECODE_DIR must be just gecode, not $(CURDIR)/gecode-release-6.3.0. - Since the C++ source code already uses #include , the include path must be -I$(GECODE_DIR), not -I$(GECODE_DIR)/gecode. - Don't hard-code compiler names. Instead of ./configure CC=gcc ..., use ./configure CC="$(CC)" CXX="$(CXX20) $(CXX20STD)". - Build gecode as a static library and link it into the package shared library. Otherwise you'll need to arrange for the gecode shared libraries to be installed together with the R package and properly loaded before your package shared library. - Provide a 'clean' target so that R CMD build doesn't include the binaries from the gecode subdirectory into the source package. With that, we get the following Makevars: all: $(SHLIB) GECODE_DIR = gecode PKG_CPPFLAGS = -I$(GECODE_DIR) PKG_LIBS = -L$(GECODE_DIR) -lgecodesearch -lgecodeminimodel \ -lgecodeint -lgecodekernel -lgecodesupport CXX_STD = CXX20 $(SHLIB): gecode_built # package code relies on header files only available after gecode is # configured and probably built $(OBJECTS): gecode_built gecode_built: cd $(GECODE_DIR) && \ ./configure CC="$(CC)" CXX="$(CXX20) $(CXX20STD)" \ --disable-examples --enable-static --disable-shared \ --disable-flatzinc --disable-qt $(MAKE) -C $(GECODE_DIR) MAKEFLAGS= touch $@ clean: -cd $(GECODE_DIR) && make clean -rm -f gecode_built Hopefully, the Makevars.win with same contents but --with-host-os=windows added to the configure flags will work on Windows. (It needs to be .win, not .ucrt, because your package declares support for R > 3.6, which predates use of UCRT in R. To think of it, R-3.6.0 also predates C++20...) You can probably give some more --disable-... arguments to ./configure to remove some more components that the R package doesn't use. However, this is only the beginning of the troubles. R packages are not allowed file path lengths above 100 bytes, and your package has seven that are longer, e.g., >> RcppTestPackage/src/gecode/gecode/third-party/boost/numeric/interval/detail/x86gcc_rounding_control.hpp Maybe you can unbundle the bits of Boost and rely on the 'BH' package instead. Maybe there's a way to rename the files and patch the includes in the source code. The gecode library produces a lot of warnings, and R CMD check considers some of them serious: >> Found the following significant warnings: >> ./gecode/kernel/core.hpp:4102:24: warning: array subscript -1 is >> below array bounds of ‘unsigned int [1]’ [-Warray-bounds] >> ./gecode/kernel/core.hpp:4109:17: warning: array subscript -1 is >> below array bounds of ‘unsigned int [1]’ [-Warray-bounds] >> ./gecode/support/allocator.hpp:88:11: warning: ‘void free(void*)’ >> called on pointer returned from a mismatched allocation function >> [-Wmismatched-new-delete] The -Warray-bounds warnings might be false positives (could pc_max ever be 0?), but -Wmismatched-new-delete is probably real; the compiler sho
Re: [R-pkg-devel] Use of long double in configuration
On 30.04.2025 10:25, Tim Taylor wrote: Is it correct to say that R's conditional use of long double is around ensuring things work on platforms which have 'long double' identical to 'double' types, as opposed to there being an odd compiler targeted that does not even have any concept of 'long double' type? a double is 64 bit and stored that way on all platforms, the concept of long doubles is CPU specific. x86 chips have 80bit in the floating point units for calculations before rounding (and normalizing) to a regular double. Some chips, e.g. those ARM chips used in current M[123]Macs (hence very relevant topic), do not support long doubles. And compilers offer to compile without support for long doubles which e.g. CRAN uses to check in an additional (issues) check. Best, Uwe Ligges As background this was motivated by a query raised in the matrixStats package: https://github.com/HenrikBengtsson/matrixStats/issues/278 __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Use of long double in configuration
Tim, no, and as far as I can tell some of the claims in that thread are incorrect. Whether long double is identical to double is irrelevant and has nothing to do with it. R has a configuration option --enable-long-double which governs whether R should use algorithms which can benefit from extended precision where applicable. However, that is entirely internal to R and has nothing to do with packages, it just changes algorithms used in R itself. It is definitely a bug in matrixStats to assume that HAVE_LONG_DOUBLE of LDOUBLE are somehow defined as they are not unless matrixStats does so. Cheers, Simon > On Apr 30, 2025, at 8:25 PM, Tim Taylor > wrote: > > Is it correct to say that R's conditional use of long double is around > ensuring things work on platforms which have 'long double' identical to > 'double' types, as opposed to there being an odd compiler targeted that does > not even have any concept of 'long double' type? > > As background this was motivated by a query raised in the matrixStats package: > https://github.com/HenrikBengtsson/matrixStats/issues/278 > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel > __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Use of long double in configuration
Cheers for the quick response. To clarify my question: Is it correct to say that as long as packages do not assume the greater precision provided by 'double' there is no reason they cannot use 'long double' to get *possible* advantages (e.g. in summations). AFAICT 'long double' is (and has always been) part of the C standard so it's use as a type should be unproblematic (this is the query relevant to matrixStats). Apologies if this does not make much sense. Tim On Wed, 30 Apr 2025, at 9:33 AM, Uwe Ligges wrote: > On 30.04.2025 10:25, Tim Taylor wrote: >> Is it correct to say that R's conditional use of long double is around >> ensuring things work on platforms which have 'long double' identical to >> 'double' types, as opposed to there being an odd compiler targeted that does >> not even have any concept of 'long double' type? > > a double is 64 bit and stored that way on all platforms, the concept of > long doubles is CPU specific. x86 chips have 80bit in the floating point > units for calculations before rounding (and normalizing) to a regular > double. > > Some chips, e.g. those ARM chips used in current M[123]Macs (hence very > relevant topic), do not support long doubles. And compilers offer to > compile without support for long doubles which e.g. CRAN uses to check > in an additional (issues) check. > > Best, > Uwe Ligges > >> >> As background this was motivated by a query raised in the matrixStats >> package: >> https://github.com/HenrikBengtsson/matrixStats/issues/278 >> >> __ >> R-package-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-package-devel > > > Attachments: > * smime.p7s __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Use of long double in configuration
Thank you all! Everything is clear. Tim On Wed, 30 Apr 2025, at 10:07 AM, Tomas Kalibera wrote: > On 4/30/25 10:43, Tim Taylor wrote: >> Cheers for the quick response. >> >> To clarify my question: Is it correct to say that as long as packages do not >> assume the greater precision provided by 'double' there is no reason they >> cannot use 'long double' to get *possible* advantages (e.g. in summations). >> AFAICT 'long double' is (and has always been) part of the C standard so it's >> use as a type should be unproblematic (this is the query relevant to >> matrixStats). > > Probably already clear from previous answers, but yes, packages can use > long double type. > > Whenever using a long double type, one needs to be careful about making > sure the algorithms work, and the tests pass (so have reasonable > tolerances), even when the long double type happens to be just the same > as double. This is the case on aarch64, and macOS/aarch64 is one of the > platforms where packages have to work, anyway, so this shouldn't be too > limiting anymore - but really one needs to test on such platform. > > R itself has an option to disable use of long double to make such > testing in R itself possible also on other platforms. In principle one > could do something similar in a package, have some ifdefs to disable > long doubles, but this is not required. And I probably wouldn't do that, > I'd just test on aarch64 regularly. > > See Writing R Extensions for more details. > > Best > Tomas > >> Apologies if this does not make much sense. >> >> Tim >> >> >> >> On Wed, 30 Apr 2025, at 9:33 AM, Uwe Ligges wrote: >>> On 30.04.2025 10:25, Tim Taylor wrote: Is it correct to say that R's conditional use of long double is around ensuring things work on platforms which have 'long double' identical to 'double' types, as opposed to there being an odd compiler targeted that does not even have any concept of 'long double' type? >>> a double is 64 bit and stored that way on all platforms, the concept of >>> long doubles is CPU specific. x86 chips have 80bit in the floating point >>> units for calculations before rounding (and normalizing) to a regular >>> double. >>> >>> Some chips, e.g. those ARM chips used in current M[123]Macs (hence very >>> relevant topic), do not support long doubles. And compilers offer to >>> compile without support for long doubles which e.g. CRAN uses to check >>> in an additional (issues) check. >>> >>> Best, >>> Uwe Ligges >>> As background this was motivated by a query raised in the matrixStats package: https://github.com/HenrikBengtsson/matrixStats/issues/278 __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel >>> >>> Attachments: >>> * smime.p7s >> __ >> R-package-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Problem building package depending on third party c++ source library.
Hello everybody, as a new feature for the anticlust package (https://cran.r-project.org/web/packages/anticlust/index.html) we are trying to include a constraint programming model, written in C++ utilizing the Gecode solver. While we generally got this feature to work when Gecode is pre-installed on the system, we would like to spare the users this step and include it with the package (or inside a supplementary package) to install through the regular R tooling. However, we run into a problem when it comes to compiling Gecode during the packages build-process, which I describe in this SO post: https://stackoverflow.com/questions/79596214/configure-script-does-not-generate-makefile-properly-for-3rd-party-library-used I strongly suspect the Makevars file to be incorrect, if you want to take a look or try to compile it yourself, I have uploaded the code of the new feature as well as the source files of the Gecode library to GitHub: https://github.com/nilslueschow/Anticlust-Dispersion-Constraint-Model/tree/main/RcppTestPackage/src Thanks for your help! Best, Nils Lüschow __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Use of long double in configuration
As one of original 30-some members of 1985 IEEE 754, I find it discouraging that we are treating long-double as the exception, when it is the introduction of "short double" in M1 etc chips that have forced the issue. There are strong commercial reasons, but they aren't computational ones. JN On 2025-04-30 05:08, Tim Taylor wrote: Thank you all! Everything is clear. Tim On Wed, 30 Apr 2025, at 10:07 AM, Tomas Kalibera wrote: On 4/30/25 10:43, Tim Taylor wrote: Cheers for the quick response. To clarify my question: Is it correct to say that as long as packages do not assume the greater precision provided by 'double' there is no reason they cannot use 'long double' to get *possible* advantages (e.g. in summations). AFAICT 'long double' is (and has always been) part of the C standard so it's use as a type should be unproblematic (this is the query relevant to matrixStats). Probably already clear from previous answers, but yes, packages can use long double type. Whenever using a long double type, one needs to be careful about making sure the algorithms work, and the tests pass (so have reasonable tolerances), even when the long double type happens to be just the same as double. This is the case on aarch64, and macOS/aarch64 is one of the platforms where packages have to work, anyway, so this shouldn't be too limiting anymore - but really one needs to test on such platform. R itself has an option to disable use of long double to make such testing in R itself possible also on other platforms. In principle one could do something similar in a package, have some ifdefs to disable long doubles, but this is not required. And I probably wouldn't do that, I'd just test on aarch64 regularly. See Writing R Extensions for more details. Best Tomas Apologies if this does not make much sense. Tim On Wed, 30 Apr 2025, at 9:33 AM, Uwe Ligges wrote: On 30.04.2025 10:25, Tim Taylor wrote: Is it correct to say that R's conditional use of long double is around ensuring things work on platforms which have 'long double' identical to 'double' types, as opposed to there being an odd compiler targeted that does not even have any concept of 'long double' type? a double is 64 bit and stored that way on all platforms, the concept of long doubles is CPU specific. x86 chips have 80bit in the floating point units for calculations before rounding (and normalizing) to a regular double. Some chips, e.g. those ARM chips used in current M[123]Macs (hence very relevant topic), do not support long doubles. And compilers offer to compile without support for long doubles which e.g. CRAN uses to check in an additional (issues) check. Best, Uwe Ligges As background this was motivated by a query raised in the matrixStats package: https://github.com/HenrikBengtsson/matrixStats/issues/278 __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel Attachments: * smime.p7s __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
