[
https://issues.apache.org/jira/browse/SOLR-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270973#comment-17270973
]
Ilan Ginzburg commented on SOLR-14928:
--------------------------------------
Very preliminary numbers running a Collection creation load from JMeter.
10 threads creating 10 collections each.
Each collection has 2 shards with 2 replicas each.
The cluster is composed of 3 nodes. All running on a MacBook Pro (created using
the "cloud" example) but each given 10GB heap.
The Overseer based cluster state change:
* Average collection creation time: 10559ms
* Min collection creation time: 8003ms
* Max collection creation time: 14403ms
The distributed cluster state change:
* Average collection creation time: 7414ms
* Min collection creation time: 5945ms
* Max collection creation time: 10503ms
I will rebase on latest master, cleanup the code a bit, run some more testing
ans will be ready for a PR...
(current non rebased branch is
[here|https://github.com/murblanc/lucene-solr/tree/SOLR-14928_ClusterStateUpdater])
> Remove Overseer ClusterStateUpdater
> -----------------------------------
>
> Key: SOLR-14928
> URL: https://issues.apache.org/jira/browse/SOLR-14928
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: Ilan Ginzburg
> Assignee: Ilan Ginzburg
> Priority: Major
> Labels: cluster, collection-api, overseer
>
> Remove the Overseer {{ClusterStateUpdater}} thread and associated Zookeeper
> queue at {{<_chroot_>/overseer/queue}}.
> Change cluster state updates so that each (Collection API) command execution
> does the update directly in Zookeeper using optimistic locking (Compare and
> Swap on the {{state.json}} Zookeeper files).
> Following this change cluster state updates would still be happening only
> from the Overseer node (that's where Collection API commands are executing),
> but the code will be ready for distribution once such commands can be
> executed by any node (other work done in the context of parent task
> SOLR-14927).
> See the [Cluster State
> Updater|https://docs.google.com/document/d/1u4QHsIHuIxlglIW6hekYlXGNOP0HjLGVX5N6inkj6Ok/edit#heading=h.ymtfm3p518c]
> section in the Removing Overseer doc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
