Re: [R-pkg-devel] How to debug segfault when running build -> document in Rstudio that includes TMB module
Le 24/01/2024 à 04:22, Carl Schwarz a écrit :
I'm trying to update my SPAS package to respond to a CRAN check. Before
starting the changes, I tried to rebuild my package, but now get a segfault
when I try to do a devtools::document() or devtools::check(args =
c('--as-cran')). See below for output from the Rstudio "Build" window.
I've
- reinstalled ALL packages
- reinstalled R (4.3.2 on MacOS Intel Chip)
- reinstalled Rstudio
When I try a rebuild/document on a sister package it runs fine, so I
suspect that the problem is related to using a TMB module that is part of
the SPAS package written in Cpp.
How do I start to "debug" this to identify the problem?
Why not simply run devtools::document() from 'R -d gdb' ?
Best,
Serguei.
Thanks
Carl Schwarz
==> devtools::document(roclets = c('rd', 'collate', 'namespace', 'vignette'))
ℹ Updating SPAS documentationℹ Loading SPAS
*** caught segfault ***
address 0x54e40, cause 'memory not mapped'
Traceback:
1: dyn.load(dll_copy_file)
2: library.dynam2(path, lib)
3: load_dll(path)
4: pkgload::load_all(path, helpers = FALSE, attach_testthat = FALSE)
5: load_code(base_path)
6: roxygen2::roxygenise(pkg$path, roclets)
7: devtools::document(roclets = c("rd", "collate", "namespace",
"vignette"))
8: withCallingHandlers(expr, packageStartupMessage = function(c)
tryInvokeRestart("muffleMessage"))
9: suppressPackageStartupMessages({oldLC <-
Sys.getlocale(category = "LC_COLLATE")Sys.setlocale(category =
"LC_COLLATE", locale = "C")on.exit(Sys.setlocale(category =
"LC_COLLATE", locale = oldLC))devtools::document(roclets = c("rd",
"collate", "namespace", "vignette"))})
An irrecoverable exception occurred. R is aborting now ...
Exited with status 139.
[[alternative HTML version deleted]]
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] new maintainer for CRAN package XML
On Mon, Jan 22, 2024 at 3:51 PM Uwe Ligges wrote: > > Dear package developers, > > the CRAN team (and Professor Ripley in particular) has been the defacto > maintainer of CRAN package 'XML'. > Our hope was that maintainers of packages depending on XML will migrate > to other packages for reading XML structures. This has not happened and > we still see dozens of strong dependencies on XML. How is this hope communicated? Many R users assume that XML package is in great shape and the preferable choice because it is maintained by the CRAN team and r-core members. Perhaps one could follow the precedent from the rgdal retirement, and set a deadline. One way to communicate this effectively would be by introducing a formal deprecation field in the package description. This could then be displayed on the XML CRAN html page, and when loading the package interactively. Other packages that import such a deprecated package could be given a CMD check warning. __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] new maintainer for CRAN package XML
if XML is deprecated, then what would be the choice for a package maintainer? Move to xml2 probably at some point I assume I use XML in the R packages I've been developing. For some of them, I started before CRAN started being the maintainer, and before xml2 inception. The thing is that XML fulfills requirements, it works and fulfills needs of depending packages that made the choice to use it. For this, it deserves to be maintained in CRAN, without having to enter into comparison exercices with other packages that , as of today, may be better to rely on (with certainly very good reasons). Moving to xml2 (or whatever other package), which although I could agree on the principle, can be costly for packages that use extensively XML. Doing so would mean that we first get the assurance that all XML features are covered elsewhere, and can be migrated smoothly. In any case, please acknowledge that this kind of migration may take time and require resources that vary (or even are missing) depending on the package projects. I doubt having CRAN setting a common deadline for retirement is a good way to foster an efficient maintenance of R packages depending on XML. It would be good to receive guidance how to migrate, while ensuring backward compatibility on our package features. Best Le 24/01/2024 à 15:59, Jeroen Ooms a écrit : On Mon, Jan 22, 2024 at 3:51 PM Uwe Ligges wrote: Dear package developers, the CRAN team (and Professor Ripley in particular) has been the defacto maintainer of CRAN package 'XML'. Our hope was that maintainers of packages depending on XML will migrate to other packages for reading XML structures. This has not happened and we still see dozens of strong dependencies on XML. How is this hope communicated? Many R users assume that XML package is in great shape and the preferable choice because it is maintained by the CRAN team and r-core members. Perhaps one could follow the precedent from the rgdal retirement, and set a deadline. One way to communicate this effectively would be by introducing a formal deprecation field in the package description. This could then be displayed on the XML CRAN html page, and when loading the package interactively. Other packages that import such a deprecated package could be given a CMD check warning. __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] new maintainer for CRAN package XML
On 24.01.2024 16:38, Emmanuel Blondel wrote: if XML is deprecated, then what would be the choice for a package maintainer? Move to xml2 probably at some point I assume I use XML in the R packages I've been developing. For some of them, I started before CRAN started being the maintainer, and before xml2 inception. The thing is that XML fulfills requirements, it works and fulfills needs of depending packages that made the choice to use it. For this, it deserves to be maintained in CRAN, without having to enter into comparison exercices with other packages that , as of today, may be better to rely on (with certainly very good reasons). Moving to xml2 (or whatever other package), which although I could agree on the principle, can be costly for packages that use extensively XML. Doing so would mean that we first get the assurance that all XML features are covered elsewhere, and can be migrated smoothly. In any case, please acknowledge that this kind of migration may take time and require resources that vary (or even are missing) depending on the package projects. I doubt having CRAN setting a common deadline for retirement is a good way to foster an efficient maintenance of R packages depending on XML. It would be good to receive guidance how to migrate, while ensuring backward compatibility on our package features. We learned that it is hard to fade out XML support, and CRAN took over maintainance as XML had too many reverse dependencies. CRAN certainly does not have the resources to be able to deprecate XML in the same way as Roger Bivand did for rgdal. So now we are looking for a new maintainer on a public list, feel free to raise your hand and take over. Then you could even consider to deprecate it and help other maintainers to migrate. I won't spend more time on any discussions. We are just looking for a volunteer. Best, Uwe Ligges Best Le 24/01/2024 à 15:59, Jeroen Ooms a écrit : On Mon, Jan 22, 2024 at 3:51 PM Uwe Ligges wrote: Dear package developers, the CRAN team (and Professor Ripley in particular) has been the defacto maintainer of CRAN package 'XML'. Our hope was that maintainers of packages depending on XML will migrate to other packages for reading XML structures. This has not happened and we still see dozens of strong dependencies on XML. How is this hope communicated? Many R users assume that XML package is in great shape and the preferable choice because it is maintained by the CRAN team and r-core members. Perhaps one could follow the precedent from the rgdal retirement, and set a deadline. One way to communicate this effectively would be by introducing a formal deprecation field in the package description. This could then be displayed on the XML CRAN html page, and when loading the package interactively. Other packages that import such a deprecated package could be given a CMD check warning. __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] new maintainer for CRAN package XML
On 24.01.2024 15:59, Jeroen Ooms wrote: On Mon, Jan 22, 2024 at 3:51 PM Uwe Ligges wrote: Dear package developers, the CRAN team (and Professor Ripley in particular) has been the defacto maintainer of CRAN package 'XML'. Our hope was that maintainers of packages depending on XML will migrate to other packages for reading XML structures. This has not happened and we still see dozens of strong dependencies on XML. How is this hope communicated? Many R users assume that XML package is in great shape and the preferable choice because it is maintained by the CRAN team and r-core members. Perhaps one could follow the precedent from the rgdal retirement, and set a deadline. One way to communicate this effectively would be by introducing a formal deprecation field in the package description. This could then be displayed on the XML CRAN html page, and when loading the package interactively. Other packages that import such a deprecated package could be given a CMD check warning. Sure, the new maintainer may do all these things. Best, Uwe Ligges __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] R-package-devel Digest, Vol 105, Issue 19
Solved...
The src/ directory also included a .o and .so objects from the last build
of the package that must be "out of date" because once I removed those
older objects, the Build -> Document and build -> Check package now work
fine without crashing...A newer version of the .o and .so objects are now
built and it now works fine.
Thanks
Carl Schwarz
On Wed, Jan 24, 2024 at 10:57 AM Carl Schwarz
wrote:
> Thanks for your suggestions. I followed the suggestion in today's message
> (see results below) which all run without issue.
> I tried to isolate the problem more
>
> The issue appears to be with load_dll()
>
> When I try
> > getwd()
> [1] "/Users/cschwarz/Library/CloudStorage/Dropbox/SPAS-R/SPAS"
> > load_dll()
>
> It crashes.
>
>
> I moved the package outside of CloudStorage to see if that is the issue.
> > getwd()
> [1] "/Users/cschwarz/Desktop/SPAS"
> > load_dll()
>
> It crashes.
>
>
> I tried doing a dll_load() where there is NO c++ object, i.e. a random
> directory and it terminates with a sensible error message
> > setwd("/Users/cschwarz/Desktop/BikePics")
> > library(pkgload)
> > load_dll()
> Error in `value[[3L]]()`:
> ! Could not find a root 'DESCRIPTION' file that starts with '^Package' in
> /Users/cschwarz/Desktop/BikePics.
> ℹ Are you in your project directory and does your project have a
> 'DESCRIPTION' file?
> Run `rlang::last_trace()` to see where the error occurred.
>
> I'm following the suggestions on including TMB code in a package at
>
> https://stackoverflow.com/questions/48627069/guidelines-for-including-tmb-c-code-in-an-r-package
> and appear to have all the necessary files
>
> I created my own load_dll() function by copying over the code and adding a
> browser().
> It appears to run fine until the statement library.dynam2(path.lib) where
> it cannot find the function library.dynam2
>
>
> > my_load_dll()
> Called from: my_load_dll()
> Browse[1]> n
> debug at #4: package <- pkg_name(path)
> Browse[2]> n
> debug at #5: env <- ns_env(package)
> Browse[2]> n
> debug at #6: nsInfo <- parse_ns_file(path)
> Browse[2]> n
> debug at #7: dlls <- list()
> Browse[2]> n
> debug at #8: dynLibs <- nsInfo$dynlibs
> Browse[2]> n
> debug at #9: nativeRoutines <- list()
> Browse[2]> n
> debug at #10: for (i in seq_along(dynLibs)) {
> lib <- dynLibs[i]
> dlls[[lib]] <- library.dynam2(path, lib)
> routines <- assignNativeRoutines(dlls[[lib]], lib, env,
> nsInfo$nativeRoutines[[lib]])
> nativeRoutines[[lib]] <- routines
> if (!is.null(names(nsInfo$dynlibs)) &&
> nzchar(names(nsInfo$dynlibs)[i]))
> env[[names(nsInfo$dynlibs)[i]]] <- dlls[[lib]]
> setNamespaceInfo(env, "DLLs", dlls)
> }
> Browse[2]> n
> debug at #11: lib <- dynLibs[i]
> Browse[2]> n
> debug at #12: dlls[[lib]] <- library.dynam2(path, lib)
> Browse[2]> n
> Error in library.dynam2(path, lib) :
> could not find function "library.dynam2"
>
> I'm unable to find where the library.dynam2() function lies... A google
> search for library.dynam2 doesn't show anything except for a cryptic
> comment in
> https://rdrr.io/cran/pkgload/src/R/load-dll.R
> which says
>
> ## The code below taken directly from base::loadNamespace
> ##
> https://github.com/wch/r-source/blob/tags/R-3-3-0/src/library/base/R/namespace.R#L466-L485
> ## except for the call to library.dynam2, which is a special version of
> ## library.dynam
>
> This is now beyond my pay grade..
>
> Suggestions?
>
>
> --
>
> From James Lamb
>
> Using the shell:
>
> R CMD build .
> - success with
>
> * checking for file ‘./DESCRIPTION’ ... OK
>
> * preparing ‘SPAS’:
>
> * checking DESCRIPTION meta-information ... OK
>
> * cleaning src
>
> * installing the package to build vignettes
>
> * creating vignettes ... OK
>
> * cleaning src
>
> * checking for LF line-endings in source and make files and shell scripts
>
> * checking for empty or unneeded directories
>
> * building ‘SPAS_2024.1.31.tar.gz’
>
>
> R CMD INSTALL --with-keep.source ./SPAS_*.tar.gz
> - success. Lots of warning from the C compiler but appears to terminate
> successfully with
>
>
> installing to /Users/cschwarz/Rlibs/00LOCK-SPAS/00new/SPAS/libs
>
> ** R
>
> ** inst
>
> ** byte-compile and prepare package for lazy loading
>
> ** help
>
> *** installing help indices
>
> ** building package indices
>
> ** installing vignettes
>
> ** testing if installed package can be loaded from temporary location
>
> ** checking absolute paths in shared objects and dynamic libraries
>
> ** testing if installed package can be loaded from final location
>
> ** testing if installed package keeps a record of temporary installation
> path
>
> * DONE (SPAS)
>
>
> # (assuming you use roxygen2)
> Rscript -e "roxygen2::roxygenize(load = 'installed')"
>
>
> - success?, it just returns with no messages
>
> cschwarz@schwarz-iMac-2020 SPAS % Rscript -e "roxygen2::roxygenize(load =
> 'installed')"
>
> cschwarz@schwarz-iMac-2020 SPAS %
>
>
> R CMD check --as-cran ./SPAS_*.tar.gz
>
>
Re: [R-pkg-devel] R-package-devel Digest, Vol 105, Issue 19
This is a reminder why one should never build packages directly in their source
directory since it can only be done once (for packages with native source code)
- always use
R CMD build --no-build-vignettes foo && R CMD INSTALL foo_*.tar.gz
if you plan to edit files in the source directory and re-use it.
Cheers,
Simon
> On 25/01/2024, at 8:34 AM, Carl Schwarz wrote:
>
> Solved...
>
> The src/ directory also included a .o and .so objects from the last build
> of the package that must be "out of date" because once I removed those
> older objects, the Build -> Document and build -> Check package now work
> fine without crashing...A newer version of the .o and .so objects are now
> built and it now works fine.
>
> Thanks
> Carl Schwarz
>
> On Wed, Jan 24, 2024 at 10:57 AM Carl Schwarz
> wrote:
>
>> Thanks for your suggestions. I followed the suggestion in today's message
>> (see results below) which all run without issue.
>> I tried to isolate the problem more
>>
>> The issue appears to be with load_dll()
>>
>> When I try
>>> getwd()
>> [1] "/Users/cschwarz/Library/CloudStorage/Dropbox/SPAS-R/SPAS"
>>> load_dll()
>>
>> It crashes.
>>
>>
>> I moved the package outside of CloudStorage to see if that is the issue.
>>> getwd()
>> [1] "/Users/cschwarz/Desktop/SPAS"
>>> load_dll()
>>
>> It crashes.
>>
>>
>> I tried doing a dll_load() where there is NO c++ object, i.e. a random
>> directory and it terminates with a sensible error message
>>> setwd("/Users/cschwarz/Desktop/BikePics")
>>> library(pkgload)
>>> load_dll()
>> Error in `value[[3L]]()`:
>> ! Could not find a root 'DESCRIPTION' file that starts with '^Package' in
>> /Users/cschwarz/Desktop/BikePics.
>> ℹ Are you in your project directory and does your project have a
>> 'DESCRIPTION' file?
>> Run `rlang::last_trace()` to see where the error occurred.
>>
>> I'm following the suggestions on including TMB code in a package at
>>
>> https://stackoverflow.com/questions/48627069/guidelines-for-including-tmb-c-code-in-an-r-package
>> and appear to have all the necessary files
>>
>> I created my own load_dll() function by copying over the code and adding a
>> browser().
>> It appears to run fine until the statement library.dynam2(path.lib) where
>> it cannot find the function library.dynam2
>>
>>
>>> my_load_dll()
>> Called from: my_load_dll()
>> Browse[1]> n
>> debug at #4: package <- pkg_name(path)
>> Browse[2]> n
>> debug at #5: env <- ns_env(package)
>> Browse[2]> n
>> debug at #6: nsInfo <- parse_ns_file(path)
>> Browse[2]> n
>> debug at #7: dlls <- list()
>> Browse[2]> n
>> debug at #8: dynLibs <- nsInfo$dynlibs
>> Browse[2]> n
>> debug at #9: nativeRoutines <- list()
>> Browse[2]> n
>> debug at #10: for (i in seq_along(dynLibs)) {
>>lib <- dynLibs[i]
>>dlls[[lib]] <- library.dynam2(path, lib)
>>routines <- assignNativeRoutines(dlls[[lib]], lib, env,
>> nsInfo$nativeRoutines[[lib]])
>>nativeRoutines[[lib]] <- routines
>>if (!is.null(names(nsInfo$dynlibs)) &&
>> nzchar(names(nsInfo$dynlibs)[i]))
>>env[[names(nsInfo$dynlibs)[i]]] <- dlls[[lib]]
>>setNamespaceInfo(env, "DLLs", dlls)
>> }
>> Browse[2]> n
>> debug at #11: lib <- dynLibs[i]
>> Browse[2]> n
>> debug at #12: dlls[[lib]] <- library.dynam2(path, lib)
>> Browse[2]> n
>> Error in library.dynam2(path, lib) :
>> could not find function "library.dynam2"
>>
>> I'm unable to find where the library.dynam2() function lies... A google
>> search for library.dynam2 doesn't show anything except for a cryptic
>> comment in
>> https://rdrr.io/cran/pkgload/src/R/load-dll.R
>> which says
>>
>> ## The code below taken directly from base::loadNamespace
>> ##
>> https://github.com/wch/r-source/blob/tags/R-3-3-0/src/library/base/R/namespace.R#L466-L485
>> ## except for the call to library.dynam2, which is a special version of
>> ## library.dynam
>>
>> This is now beyond my pay grade..
>>
>> Suggestions?
>>
>>
>> --
>>
>> From James Lamb
>>
>> Using the shell:
>>
>> R CMD build .
>> - success with
>>
>> * checking for file ‘./DESCRIPTION’ ... OK
>>
>> * preparing ‘SPAS’:
>>
>> * checking DESCRIPTION meta-information ... OK
>>
>> * cleaning src
>>
>> * installing the package to build vignettes
>>
>> * creating vignettes ... OK
>>
>> * cleaning src
>>
>> * checking for LF line-endings in source and make files and shell scripts
>>
>> * checking for empty or unneeded directories
>>
>> * building ‘SPAS_2024.1.31.tar.gz’
>>
>>
>> R CMD INSTALL --with-keep.source ./SPAS_*.tar.gz
>> - success. Lots of warning from the C compiler but appears to terminate
>> successfully with
>>
>>
>> installing to /Users/cschwarz/Rlibs/00LOCK-SPAS/00new/SPAS/libs
>>
>> ** R
>>
>> ** inst
>>
>> ** byte-compile and prepare package for lazy loading
>>
>> ** help
>>
>> *** installing help indices
>>
>> ** building package indices
>>
>> ** installing vignettes
>>
>> ** testing if installed package can be loaded from tempor
[R-pkg-devel] How to deal with issues when using devtools::check_rhub(), rhub::check(), and web form
I fixed all of the new issues identified by CRAN, and these seem to be from the r-devel-linux-x86_64-fedora-gcc and similar machines where it flags documentation in a function. However, when I use devtools::check_rhub(), it just "hangs", i.e. generates messages saying to check a web file that remains blank for several hours. Ditto for rhub::check(). I tried using the web interface at https://builder.r-hub.io/ to select the denebian machines, and it returns a message saying We're sorry, but something went wrong. If you are the application owner check the logs for more information. We're sorry, but something went wrong. If you are the application owner check the logs for more information. Same thing happens when I select the windows release machine. Revised packages passes devtools::check_win_release() or devtools::check_win_devel() and my local (on a Mac) package checks. So how do I tell if this a "Rhub issue" or an issue with my package? Or do I just give up on using Rhub to check the denebian machines? THanks Carl Schwarz [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] New Package Removal because Shared Library Too Large from Debugging Symbols
On Mon, 22 Jan 2024 17:14:04 +0100 Tomas Kalibera wrote: > Yes, inside a bigger email, reports can get overlooked, particularly > when in a thread with a rather different subject. It wasn't > overlooked this time thanks to Martin. Then additional thanks goes to Martin, and I'll make sure to report in the right place if a similar situation happens again. -- Best regards, Ivan __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] How to deal with issues when using devtools::check_rhub(), rhub::check(), and web form
В Wed, 24 Jan 2024 16:14:05 -0800 Carl Schwarz пишет: > I tried using the web interface at https://builder.r-hub.io/ to > select the denebian machines, and it returns a message saying > > We're sorry, but something went wrong. > If you are the application owner check the logs for more information. > So how do I tell if this a "Rhub issue" or an issue with my package? A problem with your package would look more like the check at least starting and then producing errors. Here, it doesn't look like the check is even starting. > Or do I just give up on using Rhub to check the denebian machines? For a while, Rhub used to offer the only on-demand checking service specifically on Linux machines (there was Win-builder by Uwe Ligges and macOS builder by Simon Urbanek, but no "Linux builder"), including Debian [*]. Now that the funding ran out [**], you can try using various continuous integration services to run your checks in a Linux virtual machine. Many of them offer free compute minutes. I think that you've already fulfilled the requirements of the CRAN policy by fixing all known problems and having R CMD check --as-cran on R-devel run for you by Win-Builder (which is what devtools::check_win_devel() does). -- Best regards, Ivan [*] Named after Debra Lynn and Ian Murdock [**] https://github.com/RConsortium/r-repositories-wg/blob/main/minutes/2023-09-07_Minutes.md __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
