You might look at colleciton aliasing, this is sometimes used for
time-series data (which I'm guessing this is).
But I have to ask whether migrating tuf faround like that is really
necessary. 2M docs isn't very many, have you stress tested with just
indexing them all to a single collection? Is the traffic on the Hot
part really heavy enough to warrant the complexity?
Best,
Erick
On Tue, Dec 15, 2015 at 7:57 AM, philippa griggs
<philippa.gri...@hotmail.co.uk> wrote:
> Hello,
>
>
> Solr 5.2.1.
>
>
> I'm using the collection API migrate statement in our test environment with
> the view to implement a Hot, Cold arrangement- newer documents will be kept
> on the Hot collection and each night the oldest documents will be migrated
> into the Cold collection. I've got it all working with a small amount of
> documents (around 28,000).
>
>
> I'm now trying to migrate around 200,000 documents and am getting 'migrate
> the collection time out:180s' message back.
>
>
> The logs from the source collection are:
>
>
> INFO - 2015-12-15 14:43:19.183; [HotSessions ]
> org.apache.solr.cloud.OverseerCollectionProcessor; Successfully created
> replica of temp source collection on target leader node
> INFO - 2015-12-15 14:43:19.183; [HotSessions ]
> org.apache.solr.cloud.OverseerCollectionProcessor; Requesting merge of temp
> source collection replica to target leader
> INFO - 2015-12-15 14:45:36.648; [ ]
> org.apache.solr.cloud.DistributedQueue$LatchWatcher; NodeDeleted fired on
> path /overseer/collection-queue-work/qnr-0000000004 state SyncConnected
> INFO - 2015-12-15 14:45:36.651; [ ]
> org.apache.solr.cloud.DistributedQueue$LatchWatcher; NodeChildrenChanged
> fired on path /overseer/collection-queue-work state SyncConnected
> ERROR - 2015-12-15 14:45:36.651; [ ] org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: migrate the collection time out:180s
> at
> org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:237)
> etc
>
>
> The logs from the target collection are:
>
> INFO - 2015-12-15 14:43:19.128; [split_shard1_temp_shard2 shard1
> split_shard1_temp_shard2_shard1_replica2] org.apache.solr.update.UpdateLog;
> Took 22 ms to seed version buckets with highest version 1520634636692094979
> INFO - 2015-12-15 14:43:19.129; [split_shard1_temp_shard2 shard1
> split_shard1_temp_shard2_shard1_replica2]
> org.apache.solr.cloud.RecoveryStrategy; Finished recovery process.
> core=split_shard1_temp_shard2_shard1_replica2
> INFO - 2015-12-15 14:43:19.199; [ ]
> org.apache.solr.update.DirectUpdateHandler2; start mergeIndexes{}
>
> As there are no errors in the target collection, am I right in assuming the
> timeout occured because the merge took too long? If that is so, how to I
> increase the timeout period? Ideally I will need to migrate around 2 million
> documents a night.
>
>
> Any help would be much appreciated.
>
>
> Philippa
>
>
