[Rd] SQLite database file gets corrupted and XP goes blue

2013-12-31 Thread Sachko Honda
Dear fellows,

I'm getting blue screen on XP while populating SQLite database, using
RStudio and RQLite package.

The blue screen says "INVALID PROCESS ATTACH ATTEMPT" before goes into
memory check and reboot.
The database (file) is corrupted and not readable after this happens.

I'm iterating over CSV files, each contans a time-series of as many as
500x5000: rows being dates, columns being dependent variables.  For each
CSV creates a 500x5000 matrix (preallocated), and it is "remove"-ed
immediately after its use, among other relatively large objects.

Without this explicit resource deallocation, the blue screen stuff
occurred at much earlier during the process.

Still, XP's Process Monitor shows steady increase in memory usage. 

My XP has 3GB RAM.  It seems that the screen goes blue when the usage
goes near 1GB.

The database file, when it is corrupted, has only 150MB.  SQLite's max
db size is 2TB, XP32's max file size is 4GB.  So, I'm well under the limits.

In the past, when an R process goes out of memory, it rather graceful
returned, like "Out of memory".

I have closed any other applications that accesses the database such as
SQLite Browser.

Any insight appreciated.
Sachko

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


Re: [Rd] SQLite database file gets corrupted and XP goes blue

2013-12-31 Thread Hadley Wickham
I'd recommend sending a reproducible example to the maintainer of the
RSQLite package (cc'd in case he doesn't follow r-devel any more)
Hadley

On Mon, Dec 30, 2013 at 7:06 PM, Sachko Honda
 wrote:
> Dear fellows,
>
> I'm getting blue screen on XP while populating SQLite database, using
> RStudio and RQLite package.
>
> The blue screen says "INVALID PROCESS ATTACH ATTEMPT" before goes into
> memory check and reboot.
> The database (file) is corrupted and not readable after this happens.
>
> I'm iterating over CSV files, each contans a time-series of as many as
> 500x5000: rows being dates, columns being dependent variables.  For each
> CSV creates a 500x5000 matrix (preallocated), and it is "remove"-ed
> immediately after its use, among other relatively large objects.
>
> Without this explicit resource deallocation, the blue screen stuff
> occurred at much earlier during the process.
>
> Still, XP's Process Monitor shows steady increase in memory usage.
>
> My XP has 3GB RAM.  It seems that the screen goes blue when the usage
> goes near 1GB.
>
> The database file, when it is corrupted, has only 150MB.  SQLite's max
> db size is 2TB, XP32's max file size is 4GB.  So, I'm well under the limits.
>
> In the past, when an R process goes out of memory, it rather graceful
> returned, like "Out of memory".
>
> I have closed any other applications that accesses the database such as
> SQLite Browser.
>
> Any insight appreciated.
> Sachko
>
> __
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel



-- 
http://had.co.nz/

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel