Hi, we are trying to use SSTableLoader to bootstrap a new 7-node cassandra (v. 1.0.10) cluster with the snapshots taken from a 3-node cassandra cluster. The new cluster is in a different data centre.
After reading the articles at [1] http://www.datastax.com/dev/blog/bulk-loading [2] http://geekswithblogs.net/johnsPerfBlog/archive/2011/07/26/how-to-use-cassandrs-sstableloader.aspx we are tried to follow this procedure 1) we took a snapshot of our keyspaces in the old cluster and moved them to the data folder of 3 of the new machines 2) started cassandra in the new cluster but we noticed that some column families were missing, other had missing data. After that we tried to use sstableloader 1) we reinstalled cassandra in the new cluster 2) run sstableloader (as explained in [2]) to load the keyspaces SSTableLoader starts, but the progress is always 0 and the transfer rate is 0MB/s. Some warning and exceptions are present in the logs ./sstableloader /opt/analytics/analytics/ Starting client (and waiting 30 seconds for gossip) ... Streaming revelant part of /opt/analytics/analytics/chart-hd-104-Data.db /opt/analytics/analytics/chart-hd-105-Data.db /opt/analytics/analytics/chart-hd-106-Data.db /opt/analytics/analytics/chart-hd-107-Data.db /opt/analytics/analytics/chart-hd-108-Data.db to [/1x.xx.xx.xx5, /1x.xx.xx.xx7, /1x.xx.xx.xx0, /1x.xx.xx.xx7, /1x.xx.xx.xx3, /1x.xx.xx.xx8, /1x.xx.xx.xx7] WARN 09:02:38,534 Unable to instantiate cache provider org.apache.cassandra.cache.SerializingCacheProvider; using default org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@5d59054d instead WARN 09:02:38,549 Unable to instantiate cache provider org.apache.cassandra.cache.SerializingCacheProvider; using default org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@5d59054d instead [….] ERROR 09:02:38,614 Error in ThreadPoolExecutor java.lang.RuntimeException: java.io.EOFException: unable to seek to position 93069003 in /opt/analytics/analytics/chart-hd-104-Data.db (65737276 bytes) in read-only mode at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:689) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.EOFException: unable to seek to position 93069003 in /opt/analytics/analytics/chart-hd-104-Data.db (65737276 bytes) in read-only mode at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:253) at org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:136) at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more Exception in thread "Streaming:1" java.lang.RuntimeException: java.io.EOFException: unable to seek to position 93069003 in /opt/analytics/analytics/chart-hd-104-Data.db (65737276 bytes) in read-only mode at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:689) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.EOFException: unable to seek to position 93069003 in /opt/analytics/analytics/chart-hd-104-Data.db (65737276 bytes) in read-only mode at org.apache.cassandra.io.util.RandomAccessReader.seek(RandomAccessReader.java:253) at org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:136) at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more --------------- What's the correct approach to migrate data from one cluster to another? How can I troubleshoot the problem with sstableloader? Thanks, -- Filippo Diotalevi
