Re: [Rd] Sys.timezone() fails on Linux under Microsoft WSL
> Brenton Wiernik
> on Tue, 13 Apr 2021 09:15:50 -0400 writes:
> In Microsoft’s Windows Subsystem for Linux (WSL or WSL2),
> there is not system framework, so utilities that depend on
> it fail. This includes timedatectl which R uses in
> Sys.timezone(). The timedatectl utility is present on
> Linux systems installed under WSL/WSL2, but is
> non-functional. So, when Sys.timezone() checks for
> Sys.which("timedatectl"), it receives a false
> positive. The subsequent methods after this if () do work,
> however.
> This can be fixed if line 42 of Sys.timezone() were changed from:
> if (nzchar(Sys.which("timedatectl"))) {
> to:
> if (nzchar(Sys.which("timedatectl")) && !grepl("microsoft", system("uname
-r", intern = TRUE), ignore.case = TRUE)) {
> "uname -r" returns for example: "5.4.72-microsoft-standard-WSL2"
> So checking for "microsoft" or "WSL" would probably work.
> Brenton Wiernik
Thank you. This all makes sense.
However, using system("uname -r") creates another platform
dependency (it fails, i.e., signals an error, e.g., on our Windows Server).
Could Sys.info() be used instead?
What does it give on your platform?
> [[alternative HTML version deleted]]
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] custom allocators, Valgrind and uninitialized memory
Andres, correct me if I'm wrong, but the issue here is not initialisation but rather valgrind flagging. You simply have to call VALGRIND_MAKE_MEM_DEFINED() in your code after allocVector3() to declare that you have initialised the memory - or am I missing something? Cheers, Simon > On 30/03/2021, at 9:18 AM, Andreas Kersting wrote: > > Hi Tomas, > > Thanks for sharing your view on this! I understand your point, but still I > think that the current situation is somewhat unfortunate: > > I would argue that mmap() is a natural candidate to be used together with > allocVector3(); it is even mentioned explicitly here: > https://github.com/wch/r-source/blob/trunk/src/main/memory.c#L2575-L2576 > > However, when using a non-anonymous mapping, i.e. we want mmap() to > initialize the memory e.g. from a file or a POSIX shared memory object, this > means that we need to use MAP_FIXED in case we are obliged to initialize the > memory AFTER allocVector3() returned it; at least I cannot think of a > different way to achieve this. > > The use of MAP_FIXED > - is discouraged (e.g. > https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/mmap.2.html) > - requires two calls to mmap(): (1) to obtain the (anonymous) memory to be > handed out by the custom allocater and (2) to actually map the file "over" > the just allocated vector (using MAP_FIXED), which will overwrite the vector > header; hence, we need to first back it up to later restore it > > I have implemented my function using MAP_FIXED here: > https://github.com/gfkse/bettermc/commit/f34c4f4c45c9ab11abe9b9e9b8b48064f128d731#diff-7098a5dde34efab163bbef27fe32f95c29e76236649479985d09c70100e4c737R278-R323 > > This solution, to me, is much more complicated and hacky than my previous > one, which assumed it is OK to hand out already initialized memory directly > from allocVector3(). > > Regards, > Andreas > > > 2021-03-29 10:41 GMT+02:00 "Tomas Kalibera" : >> Hi Andreas, >> On 3/26/21 8:48 PM, Andreas Kersting wrote: >>> Hi Dirk, > > Sure, let me try to explain: > > CRAN ran the tests of my >>> package using R which was configured > --with-valgrind-instrumentation > 0. >>> Valgrind reported many errors > related to the use of supposedly >>> uninitialized memory and the CRAN > team asked me to tackle these. > > >>> These errors are false positives, because I pass a custom allocator > to >>> allocVector3() which hands out memory which is already > initialized. >>> However, this memory is explicitly marked for Valgrind > as uninitialized >>> by allocVector3(), and I do not initialize it > subsequently, so Valgrind >>> complains. > > Now I am asking if it is correct that allocVector3() marks >>> memory as > uninitialized/undefined, even if it comes from a custom >>> allocator. > This is because allocVector3() cannot know if the memory might >>> > already by initialized. >> I think the semantics of allocVector/allocVector3 should be the same >> regardless of whether custom allocators are used. The semantics of >> allocVector is to provide uninitialized memory (non-pointer types, Writing R >> Extensions 5.9.2). Therefore, it is the caller who needs to take care of >> initialization. This is also the semantics of "malloc" and Rallocators.h >> says "custom_alloc_t mem_alloc; /* malloc equivalent */". >> >> So I think that your code using your custom allocator needs to initialize >> allocated memory to be correct. If your allocator initializes the memory, >> that is fine, but unnecessary. >> >> So technically speaking, the valgrind reports are not false alarms. I think >> your call sites should initialize. >> >> Best >> Tomas >> >> >> > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] custom allocators, Valgrind and uninitialized memory
Andreas, What does any of this to do with CRAN? This not a the CRAN list - we're discussing the proper approach of using valgrind and R can only assume that the memory is uninitialised (since it cannot safely assume anything else) so it is up to you to declare the memory as initialised if you can guarantee that being true. I don't quite get your earlier response about allocating *after* the call since that makes no sense to me - the whole point of a custom allocator is to allow you to allocate the memory, so whether it is initialised or not is under your control - but that means also it is your responsibility to flag the state accordingly. Note, however, that this is not merely just true by the virtue of using mmap - the memory content is only valid (initialised) if you used mmap with previously initialised content. Again, entirely up to you to decide what the semantics are since you are the author of the custom allocator. Does that make sense? Cheers, Simon > On Mar 30, 2021, at 18:27, Andreas Kersting wrote: > > Hi Simon, > > Yes, if this was acceptable on CRAN, I would agree that calling > VALGRIND_MAKE_MEM_DEFINED() in my code would be sufficient. > > But since Tomas said, "So I think that your code using your custom allocator > needs to initialize allocated memory to be correct. If your allocator > initializes the memory, that is fine, but unnecessary.", I am not sure if it > is acceptable. > > Regards, > Andreas > > 2021-03-30 00:39 GMT+02:00 "Simon Urbanek" : >> Andres, >> >> correct me if I'm wrong, but the issue here is not initialisation but rather >> valgrind flagging. You simply have to call VALGRIND_MAKE_MEM_DEFINED() in >> your code after allocVector3() to declare that you have initialised the >> memory - or am I missing something? >> >> Cheers, >> Simon >> >> >> >>> On 30/03/2021, at 9:18 AM, Andreas Kersting wrote: >>> >>> Hi Tomas, >>> >>> Thanks for sharing your view on this! I understand your point, but still I >>> think that the current situation is somewhat unfortunate: >>> >>> I would argue that mmap() is a natural candidate to be used together with >>> allocVector3(); it is even mentioned explicitly here: >>> https://github.com/wch/r-source/blob/trunk/src/main/memory.c#L2575-L2576 >>> >>> However, when using a non-anonymous mapping, i.e. we want mmap() to >>> initialize the memory e.g. from a file or a POSIX shared memory object, >>> this means that we need to use MAP_FIXED in case we are obliged to >>> initialize the memory AFTER allocVector3() returned it; at least I cannot >>> think of a different way to achieve this. >>> >>> The use of MAP_FIXED >>> - is discouraged (e.g. >>> https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/mmap.2.html) >>> - requires two calls to mmap(): (1) to obtain the (anonymous) memory to be >>> handed out by the custom allocater and (2) to actually map the file "over" >>> the just allocated vector (using MAP_FIXED), which will overwrite the >>> vector header; hence, we need to first back it up to later restore it >>> >>> I have implemented my function using MAP_FIXED here: >>> https://github.com/gfkse/bettermc/commit/f34c4f4c45c9ab11abe9b9e9b8b48064f128d731#diff-7098a5dde34efab163bbef27fe32f95c29e76236649479985d09c70100e4c737R278-R323 >>> >>> This solution, to me, is much more complicated and hacky than my previous >>> one, which assumed it is OK to hand out already initialized memory directly >>> from allocVector3(). >>> >>> Regards, >>> Andreas >>> >>> >>> 2021-03-29 10:41 GMT+02:00 "Tomas Kalibera" : Hi Andreas, On 3/26/21 8:48 PM, Andreas Kersting wrote: > Hi Dirk, > > Sure, let me try to explain: > > CRAN ran the tests of my > package using R which was configured > --with-valgrind-instrumentation > > 0. Valgrind reported many errors > related to the use of supposedly > uninitialized memory and the CRAN > team asked me to tackle these. > > > These errors are false positives, because I pass a custom allocator > to > allocVector3() which hands out memory which is already > initialized. > However, this memory is explicitly marked for Valgrind > as uninitialized > by allocVector3(), and I do not initialize it > subsequently, so Valgrind > complains. > > Now I am asking if it is correct that allocVector3() marks > memory as > uninitialized/undefined, even if it comes from a custom > allocator. > This is because allocVector3() cannot know if the memory > might > already by initialized. I think the semantics of allocVector/allocVector3 should be the same regardless of whether custom allocators are used. The semantics of allocVector is to provide uninitialized memory (non-pointer types, Writing R Extensions 5.9.2). Therefore, it is the caller who needs to take care of initialization. This is also the semantics of "malloc" and Rallocators.h says "
Re: [Rd] [Solved] Possible x11 window manager window aggregation under one icon?
On Fri, 26 Mar 2021 14:49:56 +0100 Martin Maechler wrote: > I concluded I liked the first [patch] because it would achieve > what's considered "uniformly better" in the sense that it makes > R graphics behave like "all other" desktop applications *and* it > would do so for all possible window manager scheme without any > need of some desktop setting (which a typical user would not > know about, nor know that s?he should/could change). Martin, here is some information on how X11 (and rgl) windows are grouped on different platforms, depending on the presence of the first patch and the .desktop file: - GNOME [1] needs a .desktop file to group windows by their WM_CLASS and otherwise groups windows by their WM_HINTS.window_group only, which makes different configurations possible: * no window_group, no .desktop file: no grouping * window_group patch, no .desktop file: plot windows grouped per-process, x11 and rgl separately * StartupWMClass in .desktop file: x11 and rgl windows from all R processes grouped together - the XQuartz window manager apparently uses WM_HINTS.window_group but not WM_CLASS to group windows [2] (patch is only relevant for rgl, .desktop files aren't relevant) - Xfce groups windows by WM_CLASS [3], so adding WM_HINTS.window_group or changing the .desktop file doesn't group them any more - KDE seems to follow the .desktop file (if StartupWMClass is present) and group by WM_CLASS otherwise [4], but I don't have it installed to check. On the other hand, KDE might be ignoring WM_HINTS.window_group, judging by the absence of "WM2GroupLeader" in the source code of Plasma Workspace. This implies no changes in window grouping from patching window_group or changing the .desktop file. [5] To summarise, the patch to x11() leads to visible window grouping in GNOME-based environments. My impression is that most environments group windows by WM_CLASS (which had been happening without the patch), but there are far too many window managers to check that impression. -- Best regards, Ivan [1] Including Cinammon: https://github.com/linuxmint/cinnamon/blob/72732da43e971b83d926af56998c83ee8d000394/src/cinnamon-window-tracker.c#L482 Likely Unity forks too. [2] Judging by https://stat.ethz.ch/pipermail/r-devel/2021-March/080571.html [3] With minor inconsistencies regarding x11 vs rgl windows between Win+Tab "switch between application windows" and the window buttons on the task bar, but let's not focus on that. [4] https://invent.kde.org/plasma/plasma-workspace/-/blob/7c49a0ae/libtaskmanager/xwindowtasksmodel.cpp#L439 [5] Strictly speaking, StartupWMClass= in the .desktop file lets the desktop environment associate the plot windows with the R launcher button if it's also present on the task bar. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Sys.timezone() fails on Linux under Microsoft WSL
That would work.
Sys.info()['release']
# release
# "5.4.72-microsoft-standard-WSL2"
Brenton
From: Martin Maechler
Sent: Wednesday, April 14, 2021 03:28
To: Brenton Wiernik
Cc: r-devel@r-project.org
Subject: Re: [Rd] Sys.timezone() fails on Linux under Microsoft WSL
> Brenton Wiernik
> on Tue, 13 Apr 2021 09:15:50 -0400 writes:
> In Microsoft’s Windows Subsystem for Linux (WSL or WSL2),
> there is not system framework, so utilities that depend on
> it fail. This includes timedatectl which R uses in
> Sys.timezone(). The timedatectl utility is present on
> Linux systems installed under WSL/WSL2, but is
> non-functional. So, when Sys.timezone() checks for
> Sys.which("timedatectl"), it receives a false
> positive. The subsequent methods after this if () do work,
> however.
> This can be fixed if line 42 of Sys.timezone() were changed from:
> if (nzchar(Sys.which("timedatectl"))) {
> to:
> if (nzchar(Sys.which("timedatectl")) && !grepl("microsoft", system("uname
-r", intern = TRUE), ignore.case = TRUE)) {
> "uname -r" returns for example: "5.4.72-microsoft-standard-WSL2"
> So checking for "microsoft" or "WSL" would probably work.
> Brenton Wiernik
Thank you. This all makes sense.
However, using system("uname -r") creates another platform
dependency (it fails, i.e., signals an error, e.g., on our Windows Server).
Could Sys.info() be used instead?
What does it give on your platform?
> [[alternative HTML version deleted]]
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] problem adding gdb to RTOOLS40 on Windows
I've not been able to install gdb for RTOOLS40 on Windows 10. The rtools installer (rtools40-x64_86.exe) doesn't seem to include gdb by default, and when I follow the instructions for adding gdb (which I tracked down at https://github.com/r-windows/docs/blob/master/faq.md) this is what happened: $ pacman -S mingw-w64-x86_64-gdb resolving dependencies... looking for conflicting packages... Packages (8) mingw-w64-x86_64-expat-2.2.9-9002 mingw-w64-x86_64-gettext-0.19.8.1-9002 mingw-w64-x86_64-libsystre-1.0.1-9002 mingw-w64-x86_64-libtre-git-r128.6fb7206-9002 mingw-w64-x86_64-ncurses-6.1.20180526-9002 mingw-w64-x86_64-readline-8.0.001-2 mingw-w64-x86_64-termcap-1.3.1-9002 mingw-w64-x86_64-gdb-8.3.1-9500 Total Download Size:3.46 MiB Total Installed Size: 83.77 MiB :: Proceed with installation? [Y/n] y :: Retrieving packages... error: failed retrieving file 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from cloud.r-project.org : The requested URL returned error: 404 error: failed retrieving file 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from cran.r-project.org : The requested URL returned error: 404 error: failed retrieving file 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from dl.bintray.com : The requested URL returned error: 404 warning: failed to retrieve some files error: failed to commit transaction (failed to retrieve some files) Errors occurred, no packages were upgraded. Something very similar happens if I try the 32bit version. Do you have any suggestions? - I used to have this stuff working OK with R3.6/RTOOLS35 but have not needed to go back to C for a while. The version of gdb in RTOOLS is 7.9.1; I tried copying that gdb.exe into RTOOLS40 but it just exited instantly when I tried to run it from there. - NB I have absolutely no idea what is meant by msys2 or pacman or any of that, I'm just following instructions... Thanks Mark Mark Bravington CSIRO Marine Lab Hobart Australia __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] problem adding gdb to RTOOLS40 on Windows
Hi Mark, Try pacman -Suy first to update pacman's package database. It's quite possible that the versions pacman is trying to install are no longer available due to being out of date. HTH, Cesko -Original Message- From: R-devel On Behalf Of Bravington, Mark (Data61, Hobart) Sent: Thursday, April 15, 2021 6:53 AM To: R-Devel-2 Subject: [Rd] problem adding gdb to RTOOLS40 on Windows I've not been able to install gdb for RTOOLS40 on Windows 10. The rtools installer (rtools40-x64_86.exe) doesn't seem to include gdb by default, and when I follow the instructions for adding gdb (which I tracked down at https://github.com/r-windows/docs/blob/master/faq.md) this is what happened: $ pacman -S mingw-w64-x86_64-gdb resolving dependencies... looking for conflicting packages... Packages (8) mingw-w64-x86_64-expat-2.2.9-9002 mingw-w64-x86_64-gettext-0.19.8.1-9002 mingw-w64-x86_64-libsystre-1.0.1-9002 mingw-w64-x86_64-libtre-git-r128.6fb7206-9002 mingw-w64-x86_64-ncurses-6.1.20180526-9002 mingw-w64-x86_64-readline-8.0.001-2 mingw-w64-x86_64-termcap-1.3.1-9002 mingw-w64-x86_64-gdb-8.3.1-9500 Total Download Size:3.46 MiB Total Installed Size: 83.77 MiB :: Proceed with installation? [Y/n] y :: Retrieving packages... error: failed retrieving file 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from cloud.r-project.org : The requested URL returned error: 404 error: failed retrieving file 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from cran.r-project.org : The requested URL returned error: 404 error: failed retrieving file 'mingw-w64-x86_64-gdb-8.3.1-9500-any.pkg.tar.xz' from dl.bintray.com : The requested URL returned error: 404 warning: failed to retrieve some files error: failed to commit transaction (failed to retrieve some files) Errors occurred, no packages were upgraded. Something very similar happens if I try the 32bit version. Do you have any suggestions? - I used to have this stuff working OK with R3.6/RTOOLS35 but have not needed to go back to C for a while. The version of gdb in RTOOLS is 7.9.1; I tried copying that gdb.exe into RTOOLS40 but it just exited instantly when I tried to run it from there. - NB I have absolutely no idea what is meant by msys2 or pacman or any of that, I'm just following instructions... Thanks Mark Mark Bravington CSIRO Marine Lab Hobart Australia __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
