[R] subset ffdf does not accept bit vector anymore (package ffbase)

2014-09-25 Thread christian.kamenik
Hi everyone Since I updated package 'ffbase', subset.ffdf does not work with bit vectors anymore. Here is a short example: data(iris) library(ffbase) iris.ffdf <- as.ffdf(iris) index <- sample(c(FALSE,TRUE), nrow(iris), TRUE) index.bit <- as.bit(index) subset(iris.ffdf, subset=index.bit) resu

[R] Trouble with subset.ffdf

2014-04-28 Thread christian.kamenik
Dear all I am having trouble with subsetting an ffdf object, hopefully somebody can help... I have an index, which is a ff object of vmode "logical": > index.SAS ff (open) logical length=4977231 (4977231) [1] [2] [3] [4] [5] [6] [7] [8]

Re: [R] read.table.ffdf and fixed width files

2013-08-12 Thread christian.kamenik
Dear R Users This is a summary of the things I tried with read.table.ffdf and fixed-width files. I would like to thank Jan Wijffels and Jan van der Laan for their suggestions and the time they spent on my problem! My objective was to import a file with 6'079'455 lines and 32 variables using th

Re: [R] laf_open_fwf

2013-08-09 Thread christian.kamenik
Jan, Many thanks for your suggestion! The code runs perfectly fine on the test set. Applying it to the complete data set, however, results in the following error: > while (TRUE) { + lines <- readLines(con, encoding='LATIN1') + if (length(lines) == 0) break +

Re: [R] laf_open_fwf

2013-08-08 Thread christian.kamenik
Dear Jan Many thanks for your help. In fact, all lines are shorter than my column width... my.column.widths: 238 range(nchar(lines)):235 237 So, it seems I have an inconsistent file structure... I guess there is no way to handle this in an automated way? Best Regard Christian Kameni

[R] laf_open_fwf

2013-08-06 Thread christian.kamenik
Dear all I was trying the (fairly new) LaF package, and came across the following problem: I opened a connection to a fixed width ASCII file using laf_open_fwf(my.filename, my.column_types, my.column_widths, my.column_names) When looking at the data, it turned out that \n (newline) and \r (carr

[R] read.table.ffdf and fixed width files

2013-08-06 Thread christian.kamenik
Dear all I am working on Windows 7 32-bit, and the ff- package is my daily life-saver to overcome the inherent memory limitations. Recently, I tried using read.table.ffdf to import data from a fixed-width ASCII file (file size: 1'440'865'015 Bytes) with 6'079'455 lines and 32 variables using th