Re: [Rd] write.csv

[Duncan Murdoch]

On 04/07/2017 10:01 AM, Jim Hester wrote:
On linux at least you can use `/dev/full` [1] to test writing to a full device.
    > echo 'foo' > /dev/full
    bash: echo: write error: No space left on device

Unfortunately, I get a permission denied error if I try to write there 
from MacOS.  I don't know if Windows has an equivalent.

I've taken a look at the code.  Essentially it comes down to a call to 
the C function vfprintf, which is supposed to return the number of bytes 
written, or a negative value for an error. This return value is often 
not checked; in particular, write.table and friends don't check it.

I'll add code to signal an error if there's a negative value.

I don't think it's feasible to check the number of bytes (formatted text 
with possible translation to a different encoding could have any number 
of bytes) if it's positive.  So hopefully all of our file systems will 
correctly signal an error, and not just report how many bytes were 
successfully written.


Although that won't be a perfect test for this case where part of the
file is written successfully.

An alternative suggestion for testing this is to create and mount a
loop device [2] with a small file.

[1]: https://en.wikipedia.org/wiki//dev/full
[2]: https://stackoverflow.com/a/16044420/2055486

Loop devices sound ideal, but seem to be Linux-only (at least with that 
recipe).

Duncan



On Tue, Jul 4, 2017 at 3:38 PM, Duncan Murdoch <murdoch.dun...@gmail.com> wrote:
On 04/07/2017 8:40 AM, Lipatz Jean-Luc wrote:

I would really like the bug fixed. At least this one, because I know
people in my institute using this function.
I understand your arguments about open source, but I also saw in this mail
list a proposal for a fix for this bug for which there were no answer from
the people who are able to include it in the distribution. It looks like if
there were interesting bugs and the other ones.


Please post a link to that, and I'll look.  Bug reports should be posted to
the bug list.  It's unfortunate that it is currently so difficult to do so,
but if they are only posted here, they are often overlooked.

I don't understand the other arguments : the example was reproduced with a
simple USB key and you cannot state that a disk will eternally be empty
enough, specially when it has several users.


I am not denying that it's a bug, I'm just saying that it is a difficult one
to test automatically (so we probably won't add a regression test once it's
fixed), and it's not one that has been reported often.  I didn't know there
were any reports before yours.

Duncan Murdoch


JLL


-----Message d'origine-----
De : Duncan Murdoch [mailto:murdoch.dun...@gmail.com]
Envoyé : mardi 4 juillet 2017 14:24
À : Lipatz Jean-Luc; r-devel@r-project.org
Objet : Re: [Rd] write.csv

On 04/07/2017 5:40 AM, Lipatz Jean-Luc wrote:

Hi all,

I am currently studying how to generalize the usage of R in my
statistical institute and I encountered a problem that I cannot declare on
bugzilla (cannot understand why).


Bugzilla was badly abused by spammers last year, so you need to have your
account created manually by one of the admins to post there.  Write to me
privately if you'd like me to create an account for you.  (If you want it
attached to a different email address, that's fine.)

Sorry for trying this mailing list but I am really worried about the
problem itself and the possible implications in using R in a professionnal
data production context.

The issue about 'write.csv' is that it just doesn't check if there is
enough space on disk and doesn't report failure to write data.

Example (R 3.4.0 windows 32 bits, but I reproduced the problem with older
versions and under Mac OS/X)

fwrite(as.list(1:1000000),"G:/Test")

Error in fwrite(as.list(1:1e+06), "G:/Test") :
  No space left on device: 'G:/Test'

write.csv(1:1000000,"G:/Test")


I have a big concern here, because it means that you could save some
important data at one point of time and discover a long time after that you
actually lost them.

 > I suppose that the fix is relatively straightforward, but how can we
be sure that there is no another function with the same bad properties?

R is open source.  You could work out the patch for this bug, and in the
process see the pattern of coding that leads to it.  Then you'll know if
other functions use the same buggy pattern.

Is the lesson that you should not use a R function, even from the core,
without having personnally tested it against extreme conditions?


I think the answer to that is yes.  Most people never write such big
files that they fill their disk:  if they did, all sorts of things would
go wrong on their systems.  So this kind of extreme condition isn't
often tested.  It's not easy to test in a platform independent way:  R
would need to be able to create a volume with a small capacity.  That's
a very system-dependent thing to do.

And wouldn't it be the work of the developpers to do such elementary
tests?


Again, R is open source.  You can and should contribute code (and
therefore become one of the developers) if you are working in unusual
conditions.

R states quite clearly in the welcome message every time it starts: "R
is free software and comes with ABSOLUTELY NO WARRANTY."  This is
essentially the same lack of warranty that you get with commercial
software, though it's stated a lot more clearly.

Duncan Murdoch


______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel