Re: [Rd] callNextMethod() and NAMESPACE

[Henrik Bengtsson]

It could be that you define the below in two different source files
and you are only updating the first and it is overwritten by the
second which you never edit? /Henrik

On Thu, Apr 15, 2010 at 4:21 PM, Adrian Waddell  wrote:
> Hello there,
>
> I define a accessor method for one of my classes, i.e.
>
> setMethod(f = "[",
>  signature = "NG_data",
>    definition = function(x,i,j,drop){
>       if(all(is.na(match(j,x...@shortnames)) == FALSE)){
>         return(x[,match(j,x...@shortnames)])                           }else{
>         callNextMethod()
>       }
>    }
> )
>
> where the class "NG_data" inherits from the "data.frame" class. Hence I
> added the line
>
> exportMethods("[")
>
> to my NAMESPACE file. After package building, installing and loading, I try
> to use this accessor method
>
> myObject[,1]
>
> but I get the error message:
>
> Error in callNextMethod() : bad object found as method (class "function")
>
> Interestingly, if I then execute the setMethod(f = "["...  in the command
> prompt
>
> myObject[,1]
>
> works. Does anybody has a clue what could go wrong?
>
>
> Adrian Waddell
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Issue with aggregate.ts and/or %\% on Windows

[Prof Brian Ripley]

However, in this case it is the use of %/% that is wrong: the fuzz 
ts.eps is supposed to be used.  Will alter (in R-devel for now).


On Tue, 20 Apr 2010, Peter Dalgaard wrote:


Patrick Aboyoun wrote:

I've stumbled across an issue with aggregate.ts that either is due to a
misuse of %/% or something deeper relating to numerical precision on
Windows. The test code is

x <- rep(6:10, 1:5)
as.vector(aggregate(as.ts(x), FUN = mean, ndeltat = 5))

On Linux and Mac I get the correct answer

> x <- rep(6:10, 1:5)
> as.vector(aggregate(as.ts(x), FUN = mean, ndeltat = 5)
[1]  7.2  8.8 10.0

> sessionInfo()
R version 2.11.0 RC (2010-04-18 r51771)
i386-apple-darwin9.8.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base


and on Windows I get an incorrect answer

> x <- rep(6:10, 1:5)
> as.vector(aggregate(as.ts(x), FUN = mean, ndeltat = 5))
[1] 7.0 8.5 9.5

> sessionInfo()
R version 2.11.0 beta (2010-04-11 r51685)
i386-pc-mingw32

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base


Walking through the aggregate.ts code I found this difference is due to
what 1 %/% 0.2 produces on the different platforms.

On Mac and Linux I get

> 1 %/% 0.2
[1] 5

and on Windows I get

> 1 %/% 0.2
[1] 4


I don't know if %/% supports floating point operands so I'm not sure how
to report this issue, but here it is anyway.


It's not as straightforward as that. I get 4 on one (32 bit) Linux, and
5 on another (64 bit). Based on samples of size one, I wouldn't want to
conjecture that "bitness" is the root cause, but it is probably not the
OS per se, rather the CPU or compiler version.

It is even more insidious: I see on the SAME system


1%/%0.2

[1] 4

1/0.2==5

[1] TRUE

so it isn't just the usual precision issue.

Of course, exact calculations with floating-point numbers is "unsafe at
any speed",  but this is quite peculiar. Presumably, it comes about
because of intermediate storage in an extended precision register.


--
Peter Dalgaard
Center for Statistics, Copenhagen Business School
Phone: (+45)38153501
Email: pd@cbs.dk  Priv: pda...@gmail.com

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



--
Brian D. Ripley,  rip...@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel:  +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax:  +44 1865 272595

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] transient memory allocation and external pointers

[Melissa Jane Hubisz]

Thanks for the responses.  Seth's example is indeed what I was trying
(hoping) to do, it seems to work on my system fine (ubuntu x86_64, R
2.10.1).  But if it doesn't work for him, then that definitely answers
my question.  I guess I'll have to go the Calloc/Free route.
Thanks,
Melissa

On Mon, Apr 19, 2010 at 1:22 PM, Seth Falcon  wrote:
> On 4/19/10 8:59 AM, Simon Urbanek wrote:
>>
>> On Apr 19, 2010, at 10:39 AM, Melissa Jane Hubisz wrote:
>>
>>> Hello,
>>> The Writing R extensions manual section 6.1.1 describes the transient
>>> memory allocation function R_alloc, and states that memory allocated
>>> by R_alloc is automatically freed after the .C or .Call function is
>>> completed.  However, based on my understanding of R's memory handling,
>>> as well as some test functions I have written, I suspect that this is
>>> not quite accurate.  If the .Call function returns an external pointer
>>> to something created with R_alloc, then this object seems to stick
>>> around after the .Call function is completed, and is subject to
>>> garbage collection once the external pointer object is removed.
>>>
>>
>
>> Yes, because the regular rules for the lifetime of an R object apply
>> since it is in fact an R object. It is subject to garbage collection
>> so if you assign it anywhere its lifetime will be tied to that object
>> (in your example EXTPTRSXP).
>
> I may be misunderstanding the question, but I think the answer is actually
> that it is *not* safe to put memory allocated via R_alloc into the external
> pointer address of an EXTPTRSXP.
>
> Here's what I think Melissa is doing:
>
> SEXP make_test_xp(SEXP s)
> {
>    SEXP ans;
>    const char *s0 = CHAR(STRING_ELT(s, 0));
>    char *buf = (char *)R_alloc(strlen(s0) + 1, sizeof(char));
>    memcpy(buf, s0, strlen(s0) + 1);
>    ans = R_MakeExternalPtr(buf, R_NilValue, R_NilValue);
>    return ans;
> }
>
> The memory allocated by R_alloc is "released" at the end of the .Call via
> vmaxset(vmax).  Using R_alloc in this way will lead to memory corruption (it
> does for me when I made a simple test case).
>
> For memory that really is external (not SEXP), then you should instead use
> Calloc and register a finalizer for the external pointer that will do any
> required cleanup and then call Free.
>
> If instead you want to have an externally managed SEXP, you could put it in
> the protected slot of the external pointer, but then you should allocate it
> using standard R allocation functions.
>
>
>
> + seth
>
> --
> Seth Falcon | @sfalcon | http://userprimary.net/
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] CRAN: MacOS X binary: not available, see check log?

[Simon Urbanek]

On Apr 20, 2010, at 2:57 AM, Henrik Bengtsson wrote:

> Hi,
> 
> On Thu, Apr 15, 2010 at 3:45 PM, Simon Urbanek
>  wrote:
>> 
>> On Apr 15, 2010, at 2:26 AM, Henrik Bengtsson wrote:
>> 
>>> For a couple of days, MacOS X binaries are not build on CRAN (for my
>>> recently uploaded packages only?):
>>> 
>> 
>> AFAICS your package was posted on Apr 13 so at the earliest it can be built 
>> in the Apr 14 run = yesterday. There was no Apr 14 run because the R build 
>> failed on Apr 13. I was chasing the subsequent issues yesterday and I think 
>> the latest R build is now working so today's package update should be on 
>> CRAN soon. Don't forget that packages have to swim across the Atlantic to 
>> get built and then back as binaries, so the migration can take a day or two 
>> ;).
> 
> The R.oo doesn't like water and prefers the faster option of flying.
> But it must have got stranded due to the ash cloud, because it hasn't
> made the jump back and forth over pond yet. Any updates on next
> available flight?
> 

As you know the ashes over Iceland grounded all flight traffic from Europe, so 
we're in a pinch ;).

But seriously - R.oo is on CRAN for a while now:
http://cran.at.r-project.org/bin/macosx/leopard/contrib/2.11/R.oo_1.7.2.tgz

R.oo_1.7.2.tgz  13-Apr-2010 11:20   717K

Cheers,
Simon


> /Henrik
> 
> PS. The R.oo is a frequent flyer with lots of miles.
> 
> 
>> 
>> Cheers,
>> Simon
>> 
>> 
>>> The R.oo package is listed as "MacOS X binary: not available, see
>>> check log?", but the 'check log' show no errors
>>> URL: http://cran.r-project.org/web/packages/R.oo/
>>> 
>>> Is this a known issue?
>>> 
>> 
>> 
>> 
>>> /Henrik
>>> 
>>> __
>>> R-devel@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>> 
>>> 
>> 
>> 
> 
> 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] transient memory allocation and external pointers

[Simon Urbanek]

On Apr 19, 2010, at 1:22 PM, Seth Falcon wrote:

> On 4/19/10 8:59 AM, Simon Urbanek wrote:
>> 
>> On Apr 19, 2010, at 10:39 AM, Melissa Jane Hubisz wrote:
>> 
>>> Hello,
>>> The Writing R extensions manual section 6.1.1 describes the transient
>>> memory allocation function R_alloc, and states that memory allocated
>>> by R_alloc is automatically freed after the .C or .Call function is
>>> completed.  However, based on my understanding of R's memory handling,
>>> as well as some test functions I have written, I suspect that this is
>>> not quite accurate.  If the .Call function returns an external pointer
>>> to something created with R_alloc, then this object seems to stick
>>> around after the .Call function is completed, and is subject to
>>> garbage collection once the external pointer object is removed.
>>> 
>> 
> 
>> Yes, because the regular rules for the lifetime of an R object apply
>> since it is in fact an R object. It is subject to garbage collection
>> so if you assign it anywhere its lifetime will be tied to that object
>> (in your example EXTPTRSXP).
> 
> I may be misunderstanding the question, but I think the answer is actually 
> that it is *not* safe to put memory allocated via R_alloc into the external 
> pointer address of an EXTPTRSXP.
> 
> Here's what I think Melissa is doing:
> 
> SEXP make_test_xp(SEXP s)
> {
>SEXP ans;
>const char *s0 = CHAR(STRING_ELT(s, 0));
>char *buf = (char *)R_alloc(strlen(s0) + 1, sizeof(char));
>memcpy(buf, s0, strlen(s0) + 1);
>ans = R_MakeExternalPtr(buf, R_NilValue, R_NilValue);
>return ans;
> }
> 
> The memory allocated by R_alloc is "released" at the end of the .Call via 
> vmaxset(vmax).  Using R_alloc in this way will lead to memory corruption (it 
> does for me when I made a simple test case).
> 

Can you elaborate on that? (It's really tricky to test this since you cannot 
attach a finalizer to the allocated memory).

AFAICT the R_alloc allocates a regular R vector (raw or real depending on size) 
so the usual R object rules apply. Then it is attached to the VStack. If you 
also assign it to any other object accessible from the GC roots (before the 
VStack goes away) then even removing the VStack entry won't cause de-allocation 
because it will be flagged from the other root at mark time so it won't be 
garbage collected. VStack is not released blindly it is simply pruned and left 
to garbage collection to decide whether to release the objects or not.

That said, the lesson to Melissa is that you can simply allocate a raw vector 
with the same effect - there is no need to use R_alloc() in her case (is user 
code PROTECTing is sort of equivalent to the VStack used internally).

Cheers,
Simon



> For memory that really is external (not SEXP), then you should instead use 
> Calloc and register a finalizer for the external pointer that will do any 
> required cleanup and then call Free.
> 
> If instead you want to have an externally managed SEXP, you could put it in 
> the protected slot of the external pointer, but then you should allocate it 
> using standard R allocation functions.
> 
> 
> 
> + seth
> 
> -- 
> Seth Falcon | @sfalcon | http://userprimary.net/
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
> 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] transient memory allocation and external pointers

[Simon Urbanek]

On Apr 20, 2010, at 10:12 AM, Simon Urbanek wrote:

> 
> On Apr 19, 2010, at 1:22 PM, Seth Falcon wrote:
> 
>> On 4/19/10 8:59 AM, Simon Urbanek wrote:
>>> 
>>> On Apr 19, 2010, at 10:39 AM, Melissa Jane Hubisz wrote:
>>> 
 Hello,
 The Writing R extensions manual section 6.1.1 describes the transient
 memory allocation function R_alloc, and states that memory allocated
 by R_alloc is automatically freed after the .C or .Call function is
 completed.  However, based on my understanding of R's memory handling,
 as well as some test functions I have written, I suspect that this is
 not quite accurate.  If the .Call function returns an external pointer
 to something created with R_alloc, then this object seems to stick
 around after the .Call function is completed, and is subject to
 garbage collection once the external pointer object is removed.
 
>>> 
>> 
>>> Yes, because the regular rules for the lifetime of an R object apply
>>> since it is in fact an R object. It is subject to garbage collection
>>> so if you assign it anywhere its lifetime will be tied to that object
>>> (in your example EXTPTRSXP).
>> 
>> I may be misunderstanding the question, but I think the answer is actually 
>> that it is *not* safe to put memory allocated via R_alloc into the external 
>> pointer address of an EXTPTRSXP.
>> 
>> Here's what I think Melissa is doing:
>> 
>> SEXP make_test_xp(SEXP s)
>> {
>>   SEXP ans;
>>   const char *s0 = CHAR(STRING_ELT(s, 0));
>>   char *buf = (char *)R_alloc(strlen(s0) + 1, sizeof(char));
>>   memcpy(buf, s0, strlen(s0) + 1);
>>   ans = R_MakeExternalPtr(buf, R_NilValue, R_NilValue);
>>   return ans;
>> }
>> 
>> The memory allocated by R_alloc is "released" at the end of the .Call via 
>> vmaxset(vmax).  Using R_alloc in this way will lead to memory corruption (it 
>> does for me when I made a simple test case).
>> 
> 
> Can you elaborate on that? (It's really tricky to test this since you cannot 
> attach a finalizer to the allocated memory).
> 
> AFAICT the R_alloc allocates a regular R vector (raw or real depending on 
> size) so the usual R object rules apply. Then it is attached to the VStack. 
> If you also assign it to any other object accessible from the GC roots 
> (before the VStack goes away) then even removing the VStack entry won't cause 
> de-allocation because it will be flagged from the other root at mark time so 
> it won't be garbage collected. VStack is not released blindly it is simply 
> pruned and left to garbage collection to decide whether to release the 
> objects or not.
> 

Ah, I now see the issue - I missed the part that you're NOT using it as SEXP 
(tag/prot) in the EXTPTR but as void pointer in which case it is not traversed 
at GC time - point taken. If you assign it as SEXP anywhere (list, vector, 
etc.) then my point remains ;). But, again, use PROTECT(allocVector(RAWSXP, 
..)) for the same yet safe effect.

Cheers,
Simon


> That said, the lesson to Melissa is that you can simply allocate a raw vector 
> with the same effect - there is no need to use R_alloc() in her case (is user 
> code PROTECTing is sort of equivalent to the VStack used internally).
> 
> Cheers,
> Simon
> 
> 
> 
>> For memory that really is external (not SEXP), then you should instead use 
>> Calloc and register a finalizer for the external pointer that will do any 
>> required cleanup and then call Free.
>> 
>> If instead you want to have an externally managed SEXP, you could put it in 
>> the protected slot of the external pointer, but then you should allocate it 
>> using standard R allocation functions.
>> 
>> 
>> 
>> + seth
>> 
>> -- 
>> Seth Falcon | @sfalcon | http://userprimary.net/
>> 
>> __
>> R-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>> 
>> 
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
> 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] transient memory allocation and external pointers

[luke]

On Tue, 20 Apr 2010, Simon Urbanek wrote:



On Apr 19, 2010, at 1:22 PM, Seth Falcon wrote:


On 4/19/10 8:59 AM, Simon Urbanek wrote:


On Apr 19, 2010, at 10:39 AM, Melissa Jane Hubisz wrote:


Hello,
The Writing R extensions manual section 6.1.1 describes the transient
memory allocation function R_alloc, and states that memory allocated
by R_alloc is automatically freed after the .C or .Call function is
completed.  However, based on my understanding of R's memory handling,
as well as some test functions I have written, I suspect that this is
not quite accurate.  If the .Call function returns an external pointer
to something created with R_alloc, then this object seems to stick
around after the .Call function is completed, and is subject to
garbage collection once the external pointer object is removed.






Yes, because the regular rules for the lifetime of an R object apply
since it is in fact an R object. It is subject to garbage collection
so if you assign it anywhere its lifetime will be tied to that object
(in your example EXTPTRSXP).


I may be misunderstanding the question, but I think the answer is actually that 
it is *not* safe to put memory allocated via R_alloc into the external pointer 
address of an EXTPTRSXP.

Here's what I think Melissa is doing:

SEXP make_test_xp(SEXP s)
{
   SEXP ans;
   const char *s0 = CHAR(STRING_ELT(s, 0));
   char *buf = (char *)R_alloc(strlen(s0) + 1, sizeof(char));
   memcpy(buf, s0, strlen(s0) + 1);
   ans = R_MakeExternalPtr(buf, R_NilValue, R_NilValue);
   return ans;
}

The memory allocated by R_alloc is "released" at the end of the .Call via 
vmaxset(vmax).  Using R_alloc in this way will lead to memory corruption (it does for me 
when I made a simple test case).



Can you elaborate on that? (It's really tricky to test this since you cannot 
attach a finalizer to the allocated memory).

AFAICT the R_alloc allocates a regular R vector (raw or real depending on size) 
so the usual R object rules apply. Then it is attached to the VStack. If you 
also assign it to any other object accessible from the GC roots (before the 
VStack goes away) then even removing the VStack entry won't cause de-allocation 
because it will be flagged from the other root at mark time so it won't be 
garbage collected. VStack is not released blindly it is simply pruned and left 
to garbage collection to decide whether to release the objects or not.


But R_alloc returns the pointer to the data associated with the SEXPR
that goes in the vstack, and there is no official way to get from that
data pointer to the SEXPR.  So the allocation can't be GC protected by
anything done in the code that calls R_alloc.

In any case the implementation of R_alloc is not intended to be public
and could change.

luke



That said, the lesson to Melissa is that you can simply allocate a raw vector 
with the same effect - there is no need to use R_alloc() in her case (is user 
code PROTECTing is sort of equivalent to the VStack used internally).

Cheers,
Simon




For memory that really is external (not SEXP), then you should instead use 
Calloc and register a finalizer for the external pointer that will do any 
required cleanup and then call Free.

If instead you want to have an externally managed SEXP, you could put it in the 
protected slot of the external pointer, but then you should allocate it using 
standard R allocation functions.



+ seth

--
Seth Falcon | @sfalcon | http://userprimary.net/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel




__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel



--
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa  Phone: 319-335-3386
Department of Statistics andFax:   319-335-3017
   Actuarial Science
241 Schaeffer Hall  email:  l...@stat.uiowa.edu
Iowa City, IA 52242 WWW:  http://www.stat.uiowa.edu

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Rd] Serial connections?

[Blair Christian]

Does anybody know if there is any support to read from serial ports?
I just got an arduino, and wanted to write some scripts for working
with real time streaming sensor data...

In base::connections documentation, it's not clear if there's an easy
way to do this?  Any ideas on hacking it?  I'm open to win/linux/mac
solutions.  I'm not sure how sockets work, but possibly there is a way
to pipe things to a buffer and read from a buffer in bash (in my linux
mind I have the thought of trying to redirect /dev/something to a
file, or symlinking a file to point to the hardware, but know that
there has to be some secret sauce to go from streaming in to a
readable file, but don't know what the missing components are).

Thoughts?

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] CRAN: MacOS X binary: not available, see check log?

[Henrik Bengtsson]

On Tue, Apr 20, 2010 at 3:58 PM, Simon Urbanek
 wrote:
>
> On Apr 20, 2010, at 2:57 AM, Henrik Bengtsson wrote:
>
>> Hi,
>>
>> On Thu, Apr 15, 2010 at 3:45 PM, Simon Urbanek
>>  wrote:
>>>
>>> On Apr 15, 2010, at 2:26 AM, Henrik Bengtsson wrote:
>>>
 For a couple of days, MacOS X binaries are not build on CRAN (for my
 recently uploaded packages only?):

>>>
>>> AFAICS your package was posted on Apr 13 so at the earliest it can be built 
>>> in the Apr 14 run = yesterday. There was no Apr 14 run because the R build 
>>> failed on Apr 13. I was chasing the subsequent issues yesterday and I think 
>>> the latest R build is now working so today's package update should be on 
>>> CRAN soon. Don't forget that packages have to swim across the Atlantic to 
>>> get built and then back as binaries, so the migration can take a day or two 
>>> ;).
>>
>> The R.oo doesn't like water and prefers the faster option of flying.
>> But it must have got stranded due to the ash cloud, because it hasn't
>> made the jump back and forth over pond yet. Any updates on next
>> available flight?
>>
>
> As you know the ashes over Iceland grounded all flight traffic from Europe, 
> so we're in a pinch ;).
>
> But seriously - R.oo is on CRAN for a while now:
> http://cran.at.r-project.org/bin/macosx/leopard/contrib/2.11/R.oo_1.7.2.tgz
>
> R.oo_1.7.2.tgz  13-Apr-2010 11:20       717K

I see. So, install.packages() will work, and it is only the generated
package page that has a hiccup, cf.

http://cran.r-project.org/web/packages/R.oo/index.html

Thanks (with a deja vu)

/Henrik

>
> Cheers,
> Simon
>
>
>> /Henrik
>>
>> PS. The R.oo is a frequent flyer with lots of miles.
>>
>>
>>>
>>> Cheers,
>>> Simon
>>>
>>>
 The R.oo package is listed as "MacOS X binary: not available, see
 check log?", but the 'check log' show no errors
 URL: http://cran.r-project.org/web/packages/R.oo/

 Is this a known issue?

>>>
>>>
>>>
 /Henrik

 __
 R-devel@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-devel


>>>
>>>
>>
>>
>
>

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Serial connections?

[Simon Urbanek]

On Apr 20, 2010, at 10:33 AM, Blair Christian wrote:

> Does anybody know if there is any support to read from serial ports? I just 
> got an arduino, and wanted to write some scripts for working with real time 
> streaming sensor data...
> 

Yes (I have Arduinos reporting measurements from all sensors in the house to R 
on my iMac which produces plots that are synchronized with my webserver). In 
principle you can simply use /dev/tty.usb... and read from it. In most cases 
the default setting is already fine (9600,n,8,1 on Mac) or you can use tools 
the set it up in advance (setserial on Linux etc.) so you don't have to worry 
about setting up the serial from R.

Depending on your OS you may be able to read from the serial device directly 
with a regular file connection or you can use a pipe connection to a tool which 
pipes out from the tty to stdout (written for a Mac but may work on other 
unices):

https://svn.rforge.net/C/trunk/tools/ttys.c

and then use something like

f=pipe("ttys /dev/tty.usbserial-X1234")

A rather handy option -d prepends current time to each line so you can track 
output over time. I have some more tools for this (even allowing you to share 
form Arduino output with several computers or even send remote commands to your 
Arduino including encryption etc ...).

Cheers,
Simon

PS: From experience I can say that Arduinos are highly addictive so beware ;).


> In base::connections documentation, it's not clear if there's an easy
> way to do this?  Any ideas on hacking it?  I'm open to win/linux/mac
> solutions.  I'm not sure how sockets work, but possibly there is a way
> to pipe things to a buffer and read from a buffer in bash (in my linux
> mind I have the thought of trying to redirect /dev/something to a
> file, or symlinking a file to point to the hardware, but know that
> there has to be some secret sauce to go from streaming in to a
> readable file, but don't know what the missing components are).
> 
> Thoughts?
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
> 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Serial connections?

[shotwelm]

I've done some microcontroller work over serial also. Unfortunately,
interfacing with a serial port is system dependent, and the mechanisms
can be quite different, as you probably know. It appears that Simon has
a solution below that will work if you are willing to accept the default
baud rate (9600 is way too slow for good sensor data), parity, etc.. or
use external tools. On POSIX systems, you would need access to the
termios.h header and the system ioctl function in order to change these
settings. Although I'm not 100% sure, I don't think R has this
capability ... yet. 

I'm new to the list, but I'd be surprised if the R developers that have
been around awhile haven't already considered adding support for ioctls
and the POSIX terminal interface. This makes me wonder why it's not
there. If there is no good reason, I'm starting to see a series of R
packages (or core extensions) developing. With a package for ioctls, we
could use all sorts of cool stuff, like Video4Linux2 (webcams, HAM
radio, tuners)...

When I collect sensor data over serial, I do it in python or write a
small C program to dump a single-column csv. Of course, R is excellent
for digital signal processing after that. Check out the DSP
( http://biostatmatt.com/archives/78 ) I did in R with some ECG data I
collected with an Atmel uC. 

-Matt


On Tue, 2010-04-20 at 11:05 -0400, Simon Urbanek wrote:
> On Apr 20, 2010, at 10:33 AM, Blair Christian wrote:
> 
> > Does anybody know if there is any support to read from serial ports? I just 
> > got an arduino, and wanted to write some scripts for working with real time 
> > streaming sensor data...
> > 
> 
> Yes (I have Arduinos reporting measurements from all sensors in the house to 
> R on my iMac which produces plots that are synchronized with my webserver). 
> In principle you can simply use /dev/tty.usb... and read from it. In most 
> cases the default setting is already fine (9600,n,8,1 on Mac) or you can use 
> tools the set it up in advance (setserial on Linux etc.) so you don't have to 
> worry about setting up the serial from R.
> 
> Depending on your OS you may be able to read from the serial device directly 
> with a regular file connection or you can use a pipe connection to a tool 
> which pipes out from the tty to stdout (written for a Mac but may work on 
> other unices):
> 
> https://svn.rforge.net/C/trunk/tools/ttys.c
> 
> and then use something like
> 
> f=pipe("ttys /dev/tty.usbserial-X1234")
> 
> A rather handy option -d prepends current time to each line so you can track 
> output over time. I have some more tools for this (even allowing you to share 
> form Arduino output with several computers or even send remote commands to 
> your Arduino including encryption etc ...).
> 
> Cheers,
> Simon
> 
> PS: From experience I can say that Arduinos are highly addictive so beware ;).
> 
> 
> > In base::connections documentation, it's not clear if there's an easy
> > way to do this?  Any ideas on hacking it?  I'm open to win/linux/mac
> > solutions.  I'm not sure how sockets work, but possibly there is a way
> > to pipe things to a buffer and read from a buffer in bash (in my linux
> > mind I have the thought of trying to redirect /dev/something to a
> > file, or symlinking a file to point to the hardware, but know that
> > there has to be some secret sauce to go from streaming in to a
> > readable file, but don't know what the missing components are).
> > 
> > Thoughts?
> > 
> > __
> > R-devel@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> > 
> > 
> 
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] transient memory allocation and external pointers

[Seth Falcon]

On 4/20/10 6:24 AM, Melissa Jane Hubisz wrote:

Thanks for the responses.  Seth's example is indeed what I was trying
(hoping) to do, it seems to work on my system fine (ubuntu x86_64, R
2.10.1).  But if it doesn't work for him, then that definitely answers
my question.  I guess I'll have to go the Calloc/Free route.


I expect that you could get your approach to not work on your system as 
well, you just have to try harder ;-)


Memory related bugs can be quite tricky, because incorrect code may run 
fine most of the time.  To trigger a problem, you need to have the right 
pattern of allocation such that data will be written over the memory 
that your invalid external pointer points to.


+ seth

--
Seth Falcon | @sfalcon | http://userprimary.net/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] Serial connections?

[Simon Urbanek]

On Apr 20, 2010, at 11:51 AM, shotwelm wrote:

> I've done some microcontroller work over serial also. Unfortunately, 
> interfacing with a serial port is system dependent, and the mechanisms can be 
> quite different, as you probably know. It appears that Simon has a solution 
> below that will work if you are willing to accept the default baud rate (9600 
> is way too slow for good sensor data

[OT: define "good" ;) - good doesn't mean fast - besides it won't be any good 
if it is too fast to be meaningfully processed -- that's a different story, 
though :P - and it is trivial to change so the solution works in general]


> ), parity, etc.. or use external tools. On POSIX systems, you would need 
> access to the termios.h header and the system ioctl function in order to 
> change these settings. Although I'm not 100% sure, I don't think R has this 
> capability ... yet. 
> 
> I'm new to the list, but I'd be surprised if the R developers that have been 
> around awhile haven't already considered adding support for ioctls and the 
> POSIX terminal interface. This makes me wonder why it's not there. If there 
> is no good reason, I'm starting to see a series of R packages (or core 
> extensions) developing.

Good luck ;). The issue is that connections are inherently backend-independent 
which implies that packages have no access to connection internals as they can 
change at any time. This means that you can't enhance them without putting the 
enhancements into R itself. This implies that you have to make a strong case 
since you need a volunteer in R-core to maintain that code etc. 


> With a package for ioctls, we could use all sorts of cool stuff, like 
> Video4Linux2 (webcams, HAM radio, tuners)...
> 

Ioctls are highly system-specific which is orthogonal to the design of 
connections. You could probably hack together a FD-based access system but it 
would not be compatible with connections (unless you exploit undocumented 
things if possible at all ...). Also ioctls can change the stream semantics 
entirely thus breaking anything that deals with the FD assuming some defined 
state ...


> When I collect sensor data over serial, I do it in python or write a small C 
> program to dump a single-column csv. Of course, R is excellent for digital 
> signal processing after that. Check out the DSP ( 
> http://biostatmatt.com/archives/78 ) I did in R with some ECG data I 
> collected with an Atmel uC. 
> 

Well, we're back to calling tools to do the interfacing like the ttys (I do 
prefer pipe to intermediate files)... It's not that complicated and has several 
benefits (implicit parallelization, process separation in case things go wrong 
etc.) so it is not obvious that it's a bad thing ...

I suspect that we're simply suck until the connection API is either exposed or 
re-written so packages can provide new connections types or extend existing 
one. Again, this is not trivial especially when you start messing with ioctl 
since it's easy to depart from defined behavior in that case ... That said, I 
agree that expanding connections is useful so some progress there would be 
desirable - but the "how" and "who" is not clear to me ...

That's just my $0.02, though ...

Cheers,
Simon


> 
> On Tue, 2010-04-20 at 11:05 -0400, Simon Urbanek wrote:
>> On Apr 20, 2010, at 10:33 AM, Blair Christian wrote:
>> 
>>> Does anybody know if there is any support to read from serial ports? I just 
>>> got an arduino, and wanted to write some scripts for working with real time 
>>> streaming sensor data...
>>> 
>> 
>> Yes (I have Arduinos reporting measurements from all sensors in the house to 
>> R on my iMac which produces plots that are synchronized with my webserver). 
>> In principle you can simply use /dev/tty.usb... and read from it. In most 
>> cases the default setting is already fine (9600,n,8,1 on Mac) or you can use 
>> tools the set it up in advance (setserial on Linux etc.) so you don't have 
>> to worry about setting up the serial from R.
>> 
>> Depending on your OS you may be able to read from the serial device directly 
>> with a regular file connection or you can use a pipe connection to a tool 
>> which pipes out from the tty to stdout (written for a Mac but may work on 
>> other unices):
>> 
>> https://svn.rforge.net/C/trunk/tools/ttys.c
>> 
>> and then use something like
>> 
>> f=pipe("ttys /dev/tty.usbserial-X1234")
>> 
>> A rather handy option -d prepends current time to each line so you can track 
>> output over time. I have some more tools for this (even allowing you to 
>> share form Arduino output with several computers or even send remote 
>> commands to your Arduino including encryption etc ...).
>> 
>> Cheers,
>> Simon
>> 
>> PS: From experience I can say that Arduinos are highly addictive so beware 
>> ;).
>> 
>> 
>>> In base::connections documentation, it's not clear if there's an easy
>>> way to do this?  Any ideas on hacking it?  I'm open to win/linux/mac
>>> solutions.  I'm not sure how sock

Re: [Rd] Serial connections?

[shotwelm]

Simon is right of course, there are plenty of sensors that would work
just fine at 9600 baud (like a thermistor rigged to an ADC). There's a
theorem along these lines (Nyquist sampling theorem?). I think piping
the output to R is a clever solution. I added a few lines to the ttys.c
program so that the baud rate is a command line option (i.e. -B9600)
 and confirmed it will compile in
Linux (2.6.30). Maybe it will save a step. Microcontrollers really are
addictive!

For an ioctl package, I was originally thinking of using file
descriptors directly. However, I agree this feels like subverting what
could be an extension of the connections API. Given that "everything is
a file" in POSIX systems, there may be an argument for an ioctl package
that is independent of the connections implementation, say to do things
that connections were not designed to do. For example, interfacing with
V4L2 devices usually involves many ioctl calls, an mmap call, but rarely
read or write calls. But maybe it would just be better to pipe this type
of output to R also...

-Matt

On Tue, 2010-04-20 at 16:42 -0400, Simon Urbanek wrote:
> On Apr 20, 2010, at 11:51 AM, shotwelm wrote:
> 
> > I've done some microcontroller work over serial also. Unfortunately, 
> > interfacing with a serial port is system dependent, and the mechanisms can 
> > be quite different, as you probably know. It appears that Simon has a 
> > solution below that will work if you are willing to accept the default baud 
> > rate (9600 is way too slow for good sensor data
> 
> [OT: define "good" ;) - good doesn't mean fast - besides it won't be any good 
> if it is too fast to be meaningfully processed -- that's a different story, 
> though :P - and it is trivial to change so the solution works in general]
> 
> 
> > ), parity, etc.. or use external tools. On POSIX systems, you would need 
> > access to the termios.h header and the system ioctl function in order to 
> > change these settings. Although I'm not 100% sure, I don't think R has this 
> > capability ... yet. 
> > 
> > I'm new to the list, but I'd be surprised if the R developers that have 
> > been around awhile haven't already considered adding support for ioctls and 
> > the POSIX terminal interface. This makes me wonder why it's not there. If 
> > there is no good reason, I'm starting to see a series of R packages (or 
> > core extensions) developing.
> 
> Good luck ;). The issue is that connections are inherently 
> backend-independent which implies that packages have no access to connection 
> internals as they can change at any time. This means that you can't enhance 
> them without putting the enhancements into R itself. This implies that you 
> have to make a strong case since you need a volunteer in R-core to maintain 
> that code etc. 
> 
> 
> > With a package for ioctls, we could use all sorts of cool stuff, like 
> > Video4Linux2 (webcams, HAM radio, tuners)...
> > 
> 
> Ioctls are highly system-specific which is orthogonal to the design of 
> connections. You could probably hack together a FD-based access system but it 
> would not be compatible with connections (unless you exploit undocumented 
> things if possible at all ...). Also ioctls can change the stream semantics 
> entirely thus breaking anything that deals with the FD assuming some defined 
> state ...
> 
> 
> > When I collect sensor data over serial, I do it in python or write a small 
> > C program to dump a single-column csv. Of course, R is excellent for 
> > digital signal processing after that. Check out the DSP ( 
> > http://biostatmatt.com/archives/78 ) I did in R with some ECG data I 
> > collected with an Atmel uC. 
> > 
> 
> Well, we're back to calling tools to do the interfacing like the ttys (I do 
> prefer pipe to intermediate files)... It's not that complicated and has 
> several benefits (implicit parallelization, process separation in case things 
> go wrong etc.) so it is not obvious that it's a bad thing ...
> 
> I suspect that we're simply suck until the connection API is either exposed 
> or re-written so packages can provide new connections types or extend 
> existing one. Again, this is not trivial especially when you start messing 
> with ioctl since it's easy to depart from defined behavior in that case ... 
> That said, I agree that expanding connections is useful so some progress 
> there would be desirable - but the "how" and "who" is not clear to me ...
> 
> That's just my $0.02, though ...
> 
> Cheers,
> Simon
> 
> 
> > 
> > On Tue, 2010-04-20 at 11:05 -0400, Simon Urbanek wrote:
> >> On Apr 20, 2010, at 10:33 AM, Blair Christian wrote:
> >> 
> >>> Does anybody know if there is any support to read from serial ports? I 
> >>> just got an arduino, and wanted to write some scripts for working with 
> >>> real time streaming sensor data...
> >>> 
> >> 
> >> Yes (I have Arduinos reporting measurements from all sensors in the house 
> >> to R on my iMac which produces plots t