Re: row cache re-fill very slow

2012-11-20 Thread aaron morton
> INFO [OptionalTasks:1] 2012-11-19 13:08:58,868 ColumnFamilyStore.java (line > 451) completed loading (5175655 ms; 13259976 keys) row cache So it was reading 2,562 rows per second during startup. I'd say that's not unreasonable performance for 13 million rows. It will get faster in 1.2, but for

Re: row cache re-fill very slow

2012-11-19 Thread Rob Coli
On Mon, Nov 19, 2012 at 6:17 AM, Andras Szerdahelyi < andras.szerdahe...@ignitionone.com> wrote: > How is the "saved row cache file" processed? Are the cached row keys > simply iterated over and their respective rows read from SSTables - > possibly creating random reads with small enough sstable f

Re: row cache re-fill very slow

2012-11-19 Thread Andras Szerdahelyi
Wei, i'm using the off-heap ( serialised ) row cache and front the entire thing with memcached in the middle layer ( to prevent the most actively requested rows from pressuring the Cassandra heap ). If you ask how much the pointers to the off-heap memory will take.. time will tell ( it should b

Re: row cache re-fill very slow

2012-11-19 Thread Andras Szerdahelyi
Aaron, What version are you on ? 1.1.5 Do you know how many rows were loaded ? INFO [OptionalTasks:1] 2012-11-19 13:08:58,868 ColumnFamilyStore.java (line 451) completed loading (5175655 ms; 13259976 keys) row cache In both cases I do not believe the cache is stored in token (or key) order.

Re: row cache re-fill very slow

2012-11-19 Thread Wei Zhu
hanks. -Wei From: aaron morton To: user@cassandra.apache.org Sent: Monday, November 19, 2012 1:00 PM Subject: Re: row cache re-fill very slow i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec ) re-fill of the row cache at start u

Re: row cache re-fill very slow

2012-11-19 Thread aaron morton
> i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec > ) re-fill of the row cache at start up. It was mentioned the other day. What version are you on ? Do you know how many rows were loaded ? When complete it will log a message with the pattern "completed loading

row cache re-fill very slow

2012-11-19 Thread Andras Szerdahelyi
Hey list, i was just wondering if anyone else is experiencing very slow ( ~ 3.5 MB/sec ) re-fill of the row cache at start up. We operate with a large row cache ( 10-15GB currently ) and we already measure startup times in hours :-) How is the "saved row cache file" processed? Are the cached ro