Hi Sounkou,
I will briefly share the journey of adbcsnowfake, which I submitted to
CRAN at one point. The CRAN team was kind to help me through the
process; however, I was unable to resolve an issue with an upstream
dependency that resulted in a warning (the Snowflake Go driver creates a
temporary file when the shared object is loaded) and I ran out of time
to pursue the submission. We distribute using the R-multiverse and have
never had any requests to distribute it on CRAN, which means either that
the R-multiverse is sufficient for our users or that nobody is using it.
We use the same idea as for Rust - build a C static library at build
time and call Go using C. The only time we have ever encountered the
"multiple go runtimes in the same process cause a crash" issue you
linked was on MacOS Intel (you can give it a try by attempting to use
more than one of adbcsnowflake, adbcflightsql, and/or adbcbigquery in
the same R session). I gather there is also potentially high memory
usage associated with multiple garbage collected pools that hold on to
memory in the same process.
I was able to achieve an almost warning-free submission with:
- Using go vendor to create a vendor archive (so that go packages
weren't downloaded during installation)
- Patch C and/or Go sources in the vendor directory that caused check
warnings
- Create a tarball of the vendor directory and publish on Zenodo (if I
were doing this again I'd use GitHub with a Zenodo backup because Zenodo
is slow). I had to do this to keep the tarball under 5 MB (with fewer
dependencies this might not be necessary).
- Download the vendor tarball at build time
On Windows we use a standard (i.e., not msys2) Go installation for CI
and development. There may be issues with doing so but we haven't run
across them...I gather this works because the Go runtime library is
completely self-contained but I am not an expert. It is, of course,
necessary to point go build to the proper C compiler/linker/flags on all
platforms. At the time I submitted adbcsnowflake, A go installation was
available on the MacOS runner (which Simon had kindly installed to
support the submission) but not the Windows one.
After all of that I am not sure that I would recommend building R
packages with Go dependencies, but it can be done and there are some
fantastic Go libraries that just don't have equivalents in any other
language (pmtiles is another great example).
Best of luck!
-dewey
On 2025-10-18 08:35, Sounkou Mahamane Toure wrote:
Dear All,
I have recently submitted a Go using Package to CRAN and it was
archived (i have yet to receive the rejection email).
I have anticipated the warnings (at least on Unix) and made comments
during the submission. A pointer from Ivan Krylov around the potential
of crashing R in the case of loading multiple go buildmode c-shared
loaded in the same process suggests that my comments may have been
misguided (even though I used the c-archive mode).
I figure for the fairly limited use case and the existing design
strategy of limiting communication between the go and C runtime to
pipes, I can execute an external go process instead. One issue remains
tough, is that in order to build this binary at installation time, a
golang toolchain is required.My package failed on win-builder on
windows because go was not found on the PATH of that machine. And as
it happens that shipping with the package pre-built cross-compiled
binaries like it is done in the pmtiles package is not advisable,
options become limited.
Krilov also referenced the case of the adbcbigquery and allied
packages that are likely published on r-multiverse instead of CRAN for
similar reasons.
It would be great if CRAN were to clarify its views on go using
packages and the toolchain.
Thank you in advance
Best regards,
Sounkou Mahamamane Toure
References :
The check on debian can be found here :
https://win-builder.r-project.org/incoming_pretest/goserveR_0.1.3_20251005_222222/Debian/00check.log
golang c-shared restrictions :
https://github.com/golang/go/issues/65050
The check on windows :
https://win-builder.r-project.org/incoming_pretest/goserveR_0.1.3_20251005_222222/Windows/00install.out
pmtiles builder of cross-compiled binaries :
https://github.com/walkerke/pmtiles/blob/5fcd3082bb21d5f442d458c42d5165c64c1cf173/tools/build_binaries.sh
arrow-adbc issue : the https://github.com/apache/arrow-adbc/pull/2262
[[alternative HTML version deleted]]
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
