[
https://issues.apache.org/jira/browse/SOLR-14788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230125#comment-17230125
]
Ilan Ginzburg edited comment on SOLR-14788 at 11/11/20, 5:38 PM:
-----------------------------------------------------------------
??Most of the work done by the overseer is useless??
Well... Overseer is running all Collection API commands (can't be qualified as
useless) and handles all the cluster state updates.
Some of the cluster state updates are definitely useless (a node going down
triggering potentially thousands of replica state updates) but most other
cluster updates are useful (sharding changes, added/removed replicas etc.).
I believe getting rid of the massive updates due to node up/down is the low
hanging fruit here. Doesn't feel too complicated to do (until reality kicks in
at some later point...).
Eventually though (for performance, scale and code maintainability) I do
believe we need to get rid of Overseer, but I plan to show proof that this is
actually the case before suggesting we do that change.
was (Author: murblanc):
??Most of the work done by the overseer is useless??
Well... Overseer is running all Collection API commands (can't be qualified as
useless) and handles all the cluster state updates.
Some of the cluster state updates are definitely useless (a node going down
triggering potentially thousands of replica state updates) but most other
cluster updates are useful (sharding changes, added/removed replicas etc.).
I believe getting rid of the massive updates due to node up/down is the low
hanging fruit here. Doesn't feel too complicated to do (until reality kicks in
at some later point...).
Eventually though (for performance and code maintainability) I do believe we
need to get rid of Overseer, but I plan to show proof that this is actually the
case before suggesting we do that change.
> Solr: The Next Big Thing
> ------------------------
>
> Key: SOLR-14788
> URL: https://issues.apache.org/jira/browse/SOLR-14788
> Project: Solr
> Issue Type: Task
> Reporter: Mark Robert Miller
> Assignee: Mark Robert Miller
> Priority: Critical
>
> h3.
> [!https://www.unicode.org/consortium/aacimg/1F46E.png!|https://www.unicode.org/consortium/adopted-characters.html#b1F46E]{color:#00875a}*The
> Policeman is on duty!*{color}
> {quote}_{color:#de350b}*When The Policeman is on duty, sit back, relax, and
> have some fun. Try to make some progress. Don't stress too much about the
> impact of your changes or maintaining stability and performance and
> correctness so much. Until the end of phase 1, I've got your back. I have a
> variety of tools and contraptions I have been building over the years and I
> will continue training them on this branch. I will review your changes and
> peer out across the land and course correct where needed. As Mike D will be
> thinking, "Sounds like a bottleneck Mark." And indeed it will be to some
> extent. Which is why once stage one is completed, I will flip The Policeman
> to off duty. When off duty, I'm always* {color:#de350b}*occasionally*{color}
> *down for some vigilante justice, but I won't be walking the beat, all that
> stuff about sit back and relax goes out the window.*{color}_
> {quote}
>
> I have stolen this title from Ishan or Noble and Ishan.
> This issue is meant to capture the work of a small team that is forming to
> push Solr and SolrCloud to the next phase.
> I have kicked off the work with an effort to create a very fast and solid
> base. That work is not 100% done, but it's ready to join the fight.
> Tim Potter has started giving me a tremendous hand in finishing up. Ishan and
> Noble have already contributed support and testing and have plans for
> additional work to shore up some of our current shortcomings.
> Others have expressed an interest in helping and hopefully they will pop up
> here as well.
> Let's organize and discuss our efforts here and in various sub issues.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
