Re: [Rd] Offer zip builds
> Iñaki Ucar > on Tue, 4 Jun 2019 18:06:34 +0200 writes: > FWIW, innoextract extracts the contents of the installer just fine. > Iñaki Thank you, Iñaki For me too. On the Windows server (I almost only use for testing R and ESS), I've always been happy I could install *several* versions of R (and use them easily simultaneously with ESS in the same emacs instance running). Maybe the R project is rather the example the other projects should take: It costs much less maintainer time (including documenting, helping users to solve problems, ... da da da) to provide it in this one form which *does* provide the flexibility of installing anywhere on your computer and *is* based on FOSS. > On Tue, 4 Jun 2019 at 17:40, Steven Penny wrote: >> >> On Mon, Jun 3, 2019 at 6:54 PM Marc Schwartz wrote: >> > I am on macOS primarily, albeit, I have run both Windows and Linux routinely >> > in years past. >> >> With all due respect, then you have no business in this thread. >> >> > That being said, these days, I do run Windows 10 under a Parallels VM on >> > macOS, as I have a single commercial application that I need to run for >> > clients now and then, and it sadly only runs on a real Windows install (e.g. >> > not with Wine). >> >> Further demonstrating my point. You run Windows in a virtual machine, meaning >> even if you encountered some bad installer, you could just revert to a snapshot >> or similar. >> >> > To your points: >> > >> > [bunch of links] >> >> I am sorry if I miscommunicated, I didnt and dont wish to be convinced about how >> well behaved R installer is. >> I wish for R to offer zip builds. Are you volunteering to do this for every release? .. on an ongoing basis .. unpaid ? To mention examples of BIG BIG company products such as Microsoft or Google (below) is really ridiculous. They have lots of money to spend and pay many many work hours to pay. We don't want to: Given such (and potentially many more similar) e-mail threads plus the issues mentioned above (plus Virus scanners, plus broken file transfers less easily detected than with an *.exe, plus ...), this is mainly a big time sink with an epsilon benefit. As Duncan Murdoch explained nicel, you may do it. Martin Maechler ETH Zurich and R Core Team >> Many other programming languages do: >> >> - http://strawberryperl.com/releases.html >> - https://dotnet.microsoft.com/download/dotnet-core/2.2 >> - https://golang.org/dl >> - https://nim-lang.org/install_windows.html >> - https://python.org/downloads/release/python-373 >> - https://windows.php.net/download >> >> As I see it, the question isnt "should R offer zip builds", its "why isnt R >> offering zip builds". >> >> > Unless you can make the case to them to expend the finite resources that they >> > have to support this as part of each version release process, in light of the >> > prior discussions, it is not clear that this appears to be a priority. >> >> Thats the point of my original post. If they choose to continue with only EXE, >> I will just keep using other programming languages. So you could see how it >> might be in R interest to offer this, as no zip builds might be one of the >> reasons people avoid the language. >> >> __ >> R-devel@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > -- > Iñaki Úcar > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] MacOS parallel::makeCluster fails
Hi Dominik,
from the output, the master process could not "listen" on the port where
it expects a connection from the worker. We need to find out why. I'd
recommend first to create a minimal reproducible example (and one that
does not use future, only parallel, and a minimal number of threads,
ideally just 2). Then I'd recommend to check if the problem still exists
with R-devel. Then I'd check if the problem happens in all invocations,
even after reboots, on a clean system, without many running applications
- if it does, this is good news. Then you could post such example and we
could help more - if we can reproduce on our system indeed we could
debug, if not there could at least be more directed advice on how to
debug on your side. What I'd do myself if I could reproduce on my system
would be instrument R around Sock_listen in internet module to see
exactly what has failed with which error. Maybe dtruss would help too,
but instrumenting may be easier. The earlier problem you mention has
never been diagnosed (it was only intermittent on the reporter's
machine, we could not reproduce on our systems, and despite a lot of
effort on our side and on the reporter's, we could not reliably
diagnose). In principle, it could be some race condition in R (one has
been fixed since the previous report), but especially if it is
deterministic it would more likely be some OS limit on your system. You
could of course try playing with OS limits, on the number of open files,
etc, with changing the port number (port= option), etc, but I would
recommend the systematic approach of debugging the cause.
Best
Tomas
On 6/4/19 10:45 AM, Dominik Leutnant wrote:
Hi all,
The call parallel::makeCluster(1L) hangs infinitely on my MacOS machine which
seems to be already reported by some people (e.g.,
https://stat.ethz.ch/pipermail/r-devel/2018-February/075565.html).
However, the solutions posted on SO, GH or R-devel do not work in my case.
So far, I unsuccessfully tested …
1. Couple of reboots
2. Adding 192.0.0.1 to /etc/hosts
3. Using R.app instead of RStudio.app
4. Turn off the firewall
Following Hendriks advice, “cl <- future::makeClusterPSOCK(1L, verbose = TRUE,
timeout = 60)” gives (note: without adding the timeout parameter, R just hangs):
Sys.setenv(LANGUAGE='en')
cl <- future::makeClusterPSOCK(1L, verbose = TRUE, timeout = 60)
[local output] Workers: [n = 1] ‘localhost’
[local output] Base port: 11867
[local output] Creating node 1 of 1 ...
[local output] - setting up node
Testing if worker's PID can be inferred: ‘'/Library/Frameworks/R.framework/Resources/bin/Rscript'
-e
'try(cat(Sys.getpid(),file="/var/folders/5s/kgm05t2s0_52gz1s445mnlgwgn/T//RtmpZp1RX6/future.parent=835.3434fe0c5c6.pid"),
silent = TRUE)' -e
"file.exists('/var/folders/5s/kgm05t2s0_52gz1s445mnlgwgn/T//RtmpZp1RX6/future.parent=835.3434fe0c5c6.pid')"’
- Possible to infer worker's PID: TRUE
[local output] Starting worker #1 on ‘localhost’:
'/Library/Frameworks/R.framework/Resources/bin/Rscript'
--default-packages=datasets,utils,grDevices,graphics,stats,methods -e
'try(cat(Sys.getpid(),file="/var/folders/5s/kgm05t2s0_52gz1s445mnlgwgn/T//RtmpZp1RX6/future.parent=835.3434fe0c5c6.pid"),
silent = TRUE)' -e 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11867 OUT=/dev/null
TIMEOUT=60 XDR=TRUE
[local output] - Exit code of system() call: 0
[local output] Waiting for worker #1 on ‘localhost’ to connect back
[local output] Detected a warning from socketConnection(): ‘problem in
listening on this socket’
Killing worker process (PID 903) if still alive
Worker (PID 903) was successfully killed: TRUE
Error in socketConnection("localhost", port = port, server = TRUE, blocking =
TRUE, :
Failed to launch and connect to R worker on local machine ‘localhost’ from
local machine ‘Dominiks-MBP.local’.
* The error produced by socketConnection() was: ‘cannot open the connection’
* In addition, socketConnection() produced 1 warning(s):
- Warning #1: ‘problem in listening on this socket’
* The localhost socket connection that failed to connect to the R worker used
port 11867 using a communication timeout of 60 seconds and a connection timeout
of 120 seconds.
* Worker launch call: '/Library/Frameworks/R.framework/Resources/bin/Rscript'
--default-packages=datasets,utils,grDevices,graphics,stats,methods -e
'try(cat(Sys.getpid(),file="/var/folders/5s/kgm05t2s0_52gz1s445mnlgwgn/T//RtmpZp1RX6/future.parent=835.3434fe0c5c6.pid"),
silent = TRUE)' -e 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11867 OUT=/dev/null
TIMEOUT=60 XDR=TRUE.
* Worker (PID 903) was successfully killed: TRUE
* Troubleshooting suggestions:
- Suggestion #1: Set 'outfile=NULL' to see output from worker.
In addition: Warning message:
In socketConnection("localhost", port = port, server = TRUE, blocking = TRUE, :
problem in listening on this socket
My session looks like:
sessionInfo()
R version 3.6.0 (2019-04-26)
Platfor
Re: [Rd] [External] undefined symbol errors when compiling package using ALTREP API
For now you can use
R_altrep_inherits(x, R_compact_intseq_class)
The variable R_compact_intseq_class should currently be visible to
packages on all platforms, though that may change if we eventually
provide a string-based lookup mechanism, e.g. somehting like
R_find_altrep_class("compact_intseq", "base")
Best,
luke
On Tue, 4 Jun 2019, Mark Klik wrote:
> Hi Gabriel,
>
> thanks for your detailed explanation, that definitely clarifies the design
> choices that were made in setting up the ALTREP framework and I can see how
> those choices make sure existing code won't break.
>
> My specific use-case for wanting to check whether a vector is an ALTREP is
> the following: the fst package wraps an external C++ library (fstlib,
> independent from R) that was made for high speed serialization of
> dataframe's. Sequences are fairly common in dataframe's and I'm planning to
> add the concept of a sequence to the (R-agnostic) fst format. When I can
> detect, e.g. a 'compact_intseq' ALTREP vector and just retrieve it's 3
> integer internal representation, serialization could be very fast.
> Alternatively, as you describe, the vector needs to be expanded first
> before serialization, which will actually be slower than using an already
> expanded vector and can take a lot of RAM for large datasets.
>
> So being able to make use of the internal representation of (a few of the)
> base ALTREP vectors can be very interesting for (non-R) serialization
> schemes.
>
> thanks for your time!
> Mark
>
>
> On Tue, Jun 4, 2019 at 11:50 PM Gabriel Becker
> wrote:
>
>> Hi Mark,
>>
>> So depending pretty strongly on what you mean by "ALTREP aware", packages
>> aren't necessarily supposed to be ALTREP aware. What I mean by this is that
>> as of right now, ALTREP objects are designed to be interacted with by
>> non-ALTREP-implementing package code, *more-or-less *exactly as standard
>> (non-AR) SEXPs are: via the published C API. The more or less comes from
>> the fact that in some cases, doing things that are good ideas on standard
>> SEXPS will work, but may not be a good idea for ALTREPs.
>>
>> The most "low-hanging-fruit" example of something that was best practice
>> for standard vectors but is not a good idea for ALTREP vectors is grabbing
>> a DATAPTR and iterating over the values without modification in a tight
>> loop. This will work (absent allocation failure or, I suppose, the ALTREP
>> being specifically designed to refuse to give you a full DATAPTR), but with
>> ALTREP in place its no longer what you want to do.
>>
>> That said, you don't want to check whether something is an ALTREP yourself
>> and branch your code, what you want to do is use the ITERATE_BY_REGION
>> macro in R_ext/Itermacros.h for ALL SEXPs, which will be nearly as for
>> standard vectors and work safely for ALTREP vectors.
>>
>> Basically any time you find yourself wanting to check if something is an
>> ALTREP and if so, call a specific ALT*_BLAH method, the intention is that
>> there should be a universal API point you can call which will work for both
>> types.
>>
>> This is true, e.g., of INTEGER_IS_SORTED (which will always work and just
>> returns UNKNOWN_SORTEDNESS, ie INT_MIN, ie NA_INTEGER for non-ALTREPs).,
>> for REAL_GET_REGION, (which populates a double* with the requested values
>> for both standard and ALTREP REALSXPs), etc.
>>
>> Does the above make sense?
>>
>> If you feel a universal API point is missing, you can raise that here,
>> though I can't promise that will ultimately result in the method being
>> added.
>>
>> Best,
>> ~G
>>
>> On Tue, Jun 4, 2019 at 2:22 PM Mark Klik wrote:
>>
>>> thanks for clearing that up, so these methods are actually not meant to be
>>> exported on Windows and OSX?
>>> Some of the ALTREP methods that now use 'attribute_hidden' would be very
>>> useful to packages that aim to be ALTREP aware, should the currently
>>> (exported) API be considered final?
>>>
>>> thanks for your time & best,
>>> Mark
>>>
>>> On Tue, Jun 4, 2019 at 6:52 PM Tierney, Luke
>>> wrote:
>>>
On Tue, 4 Jun 2019, Mark Klik wrote:
> Hello,
>
> I'm developing a package (lazyvec) that makes full use of the ALTREP
> framework (R >= 3.6.0).
> One application of the package is to wrap existing ALTREP vectors in a
new
> ALTREP vector and pass all calls from R to the contained object. The
> purpose of this is to provide a diagnostic framework for working with
> ALTREP vectors and show information about internal calls.
>
> The package builds on Windows and OSX but fails to build on Linux as
>>> can
be
> seen from the link to the Travis build:
> https://travis-ci.org/fstpackage/lazyvec/jobs/539442806
>
> The reason of build failure is that many ALTREP methods generate
'undefined
> symbol' errors upon building the package (on Linux). I've checked the
>>> R
> source code and the undefined symbols seems to be related to the
> 'attribute_hidden' bef
Re: [Rd] Offer zip builds
> If they choose to continue with only EXE, > I will just keep using other programming languages. I did agree with your original suggestion. However, I don't think that a lack of zip formats, is a disincentive from using R. If you have an issue with the Windows installer, the obvious option is to install the source version, and compile from it. This is, after all, how open source is designed to work. Also, I agree with what Duncan said. Abs [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [External] undefined symbol errors when compiling package using ALTREP API
thanks Luke, I can work with that and will watch out for changes and new
developments in the ALTREP code with great interest.
all the best,
Mark
On Wed, Jun 5, 2019 at 6:02 PM Tierney, Luke wrote:
> For now you can use
>
> R_altrep_inherits(x, R_compact_intseq_class)
>
> The variable R_compact_intseq_class should currently be visible to
> packages on all platforms, though that may change if we eventually
> provide a string-based lookup mechanism, e.g. somehting like
>
> R_find_altrep_class("compact_intseq", "base")
>
> Best,
>
> luke
>
>
> On Tue, 4 Jun 2019, Mark Klik wrote:
>
> > Hi Gabriel,
> >
> > thanks for your detailed explanation, that definitely clarifies the
> design
> > choices that were made in setting up the ALTREP framework and I can see
> how
> > those choices make sure existing code won't break.
> >
> > My specific use-case for wanting to check whether a vector is an ALTREP
> is
> > the following: the fst package wraps an external C++ library (fstlib,
> > independent from R) that was made for high speed serialization of
> > dataframe's. Sequences are fairly common in dataframe's and I'm planning
> to
> > add the concept of a sequence to the (R-agnostic) fst format. When I can
> > detect, e.g. a 'compact_intseq' ALTREP vector and just retrieve it's 3
> > integer internal representation, serialization could be very fast.
> > Alternatively, as you describe, the vector needs to be expanded first
> > before serialization, which will actually be slower than using an already
> > expanded vector and can take a lot of RAM for large datasets.
> >
> > So being able to make use of the internal representation of (a few of
> the)
> > base ALTREP vectors can be very interesting for (non-R) serialization
> > schemes.
> >
> > thanks for your time!
> > Mark
> >
> >
> > On Tue, Jun 4, 2019 at 11:50 PM Gabriel Becker
> > wrote:
> >
> >> Hi Mark,
> >>
> >> So depending pretty strongly on what you mean by "ALTREP aware",
> packages
> >> aren't necessarily supposed to be ALTREP aware. What I mean by this is
> that
> >> as of right now, ALTREP objects are designed to be interacted with by
> >> non-ALTREP-implementing package code, *more-or-less *exactly as standard
> >> (non-AR) SEXPs are: via the published C API. The more or less comes from
> >> the fact that in some cases, doing things that are good ideas on
> standard
> >> SEXPS will work, but may not be a good idea for ALTREPs.
> >>
> >> The most "low-hanging-fruit" example of something that was best practice
> >> for standard vectors but is not a good idea for ALTREP vectors is
> grabbing
> >> a DATAPTR and iterating over the values without modification in a tight
> >> loop. This will work (absent allocation failure or, I suppose, the
> ALTREP
> >> being specifically designed to refuse to give you a full DATAPTR), but
> with
> >> ALTREP in place its no longer what you want to do.
> >>
> >> That said, you don't want to check whether something is an ALTREP
> yourself
> >> and branch your code, what you want to do is use the ITERATE_BY_REGION
> >> macro in R_ext/Itermacros.h for ALL SEXPs, which will be nearly as for
> >> standard vectors and work safely for ALTREP vectors.
> >>
> >> Basically any time you find yourself wanting to check if something is an
> >> ALTREP and if so, call a specific ALT*_BLAH method, the intention is
> that
> >> there should be a universal API point you can call which will work for
> both
> >> types.
> >>
> >> This is true, e.g., of INTEGER_IS_SORTED (which will always work and
> just
> >> returns UNKNOWN_SORTEDNESS, ie INT_MIN, ie NA_INTEGER for non-ALTREPs).,
> >> for REAL_GET_REGION, (which populates a double* with the requested
> values
> >> for both standard and ALTREP REALSXPs), etc.
> >>
> >> Does the above make sense?
> >>
> >> If you feel a universal API point is missing, you can raise that here,
> >> though I can't promise that will ultimately result in the method being
> >> added.
> >>
> >> Best,
> >> ~G
> >>
> >> On Tue, Jun 4, 2019 at 2:22 PM Mark Klik wrote:
> >>
> >>> thanks for clearing that up, so these methods are actually not meant
> to be
> >>> exported on Windows and OSX?
> >>> Some of the ALTREP methods that now use 'attribute_hidden' would be
> very
> >>> useful to packages that aim to be ALTREP aware, should the currently
> >>> (exported) API be considered final?
> >>>
> >>> thanks for your time & best,
> >>> Mark
> >>>
> >>> On Tue, Jun 4, 2019 at 6:52 PM Tierney, Luke
> >>> wrote:
> >>>
> On Tue, 4 Jun 2019, Mark Klik wrote:
>
> > Hello,
> >
> > I'm developing a package (lazyvec) that makes full use of the ALTREP
> > framework (R >= 3.6.0).
> > One application of the package is to wrap existing ALTREP vectors in
> a
> new
> > ALTREP vector and pass all calls from R to the contained object. The
> > purpose of this is to provide a diagnostic framework for working with
> > ALTREP vectors and show information about internal calls.
> >
> > The pack
Re: [Rd] Offer zip builds
On 05/06/2019 4:48 p.m., Abby Spurdle wrote: > If they choose to continue with only EXE, > I will just keep using other programming languages. I did agree with your original suggestion. However, I don't think that a lack of zip formats, is a disincentive from using R. If you don't want to run the installer, see Iñaki's message for how to extract the files without running it. I don't know if any of the configuration done by the installer is essential; you can read the source in https://svn.r-project.org/R/trunk/src/gnuwin32/installer if you are curious. Duncan Murdoch If you have an issue with the Windows installer, the obvious option is to install the source version, and compile from it. This is, after all, how open source is designed to work. Also, I agree with what Duncan said. Abs __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
