On Tue, Oct 27, 2015, at 10:50 AM, Nikolay Shuyskiy wrote:
Den 2015-10-22 17:54:44 skrev Shawn Heisey <apa...@elyograg.org>:
On 10/22/2015 8:29 AM, Nikolay Shuyskiy wrote:
>> I imagined that I could, say, add two new nodes to SolrCloud, and
split
>> shard so that two new shards ("halves" of the one being split) will
be
>> created on those new nodes.
>>
>> Right now the only way to split shard in my situation I see is to
create
>> two directories (shard_1_0 and shard_1_1) and mount new volumes onto
>> them *before* calling SPLITSHARD. Then I would be able to split
shards,
>> and after adding two new nodes, these new shards will be replicated,
and
>> I'll be able to clean up all the data on the first node.
>
> The reason that they must be on the same node is because index
splitting
> is a *Lucene* operation, and Lucene has no knowledge of Solr nodes,
only
> the one index on the one machine.
>
> Depending on the overall cloud distribution, one option *might* be to
> add a replica of the shard you want to split to one or more new nodes
> with plenty of disk space, and after it is replicated, delete it from
> any nodes where the disk is nearly full. Then do the split operation,
> and once it's done, use ADDREPLICA/DELETEREPLICA to arrange everything
> the way you want it.
Thank you, that makes sense and is a usable alternative for us for the
time being.
Probably we have to consider using implicit routing for the future so
that we could add new nodes without dealing with splitting.
Depends upon the use-case. For things like log files, use time based
collections, then create/destroy collection aliases to point to them.
I've had a "today" alias that points to logs_20151027 and logs_20151026,
meaning all content for the last 24hrs is available via
http://localhost:8983/solr/today. I had "week" and "month" also.
Dunno if that works for you.
Thanks for sharing your experience, but in our case any kind of time-based
splitting is irrelevant. If worse comes to worst, we can impose some kind
of pre-grouping on our documents (thank you for idea!), but it'd
complicate application logic (and Solr maintenance, I'm afraid) too much
for our taste.
--
Yrs sincerely,
Nikolay Shuyskiy
