[
https://issues.apache.org/jira/browse/SOLR-14636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159090#comment-17159090
]
Mark Robert Miller commented on SOLR-14636:
-------------------------------------------
Thanks Noble!
Test comparisons are hard by the way. No on measures them right, and a huge
number of the current tests can run for 30 seconds or 3 minutes. When you run
them individually they tend to have the luck to take the fast path. When you
run them in groups, they politely take turns being devils. And then Junit
reports just the test time and not our setup and tear down which is almost
always where the longest crap is. Very, very few people can look at these tests
and make any kind of comparison because almost no one knows these tests. They
play super hard to get.
Anyway, most of the development of SolrCloud since Yonik and I tossed it
together, has been straight on top of my first distributed system after a few
years on the job - to the point it's like someone bought a one floor shack and
built a skyscraper on it. Now, we didn't turn out so bad anyway in some
regards, looking at the state of the other open source distributed java
projects. Their tests are often worse than ours - Lucene forced Solr to be
better in a lot of ways there. But the tests are still garbage. And the
production record of these other systems is way the hell better. Because this
thing is a buggy black box of alchemy that nobody fully or half understands. A
few of us know it better than others, but NOBODY understands. I know, I built
most of the foundation, I've spent more time, I've done more things. I know
those tests inside and out and they are a nightmare, like most projects.
The problem is, our production ability sucks. Tons of money and time and
resources go into working around it. Why? It's not fixable. It's a dragon. You
will die trying. But I won't go out that way - not a crappy system that got me
a lot of money and job security - it was undeserved. So I talk about tests and
faster tests and people think, woohoo, nice, who cares. No no. The tests are a
means to an end. The process is the secret. Other projects harden in production
and limp with tests. This will be hardened by tests and then buffed by
production. And it will be awesome. These tests and this system is right now
entering another league on this branch. It's not catching up, it's getting
ready to leap frog.
I don't care, call me a dragon slayer, cause I decided I could just do the
impossible and then I did, and if others can even come half as close, I'd love
to have a competition and lose. I'd love it if others took SolrCloud away from
and made it fly. But I'm practical. So many times I had so many balls in the
air and was so tired and burned out and you think, I'm going to lose it, I'm
deep in the middle, so much invested and I can't keep all these plates spinning
and find a way out...even the idea of attempting this is a recipe for a waste
of time. The results will stand how they stand.
> Provide a reference implementation for SolrCloud that is stable and fast.
> -------------------------------------------------------------------------
>
> Key: SOLR-14636
> URL: https://issues.apache.org/jira/browse/SOLR-14636
> Project: Solr
> Issue Type: Task
> Reporter: Mark Robert Miller
> Assignee: Mark Robert Miller
> Priority: Major
>
> SolrCloud powers critical infrastructure and needs the ability to run quickly
> with stability. This reference implementation will allow for this.
> *location*: [https://github.com/apache/lucene-solr/tree/reference_impl]
> *status*: alpha
> *speed*: ludicrous
> *tests***:
> * *core*: near {color:#00875a}*solid*{color} with
> *{color:#de350b}ignores{color}*
> * *solrj*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *test-framework*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/analysis-extras*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/analytics*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/clustering*: near {color:#00875a}*solid*{color} with
> *{color:#de350b}ignores{color}*
> * *contrib/dataimporthandler*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/dataimporthandler-extras*: near {color:#00875a}*solid*{color}
> with *{color:#de350b}ignores{color}*
> * *contrib/extraction*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/jaegertracer-configurator*: near {color:#00875a}*solid*{color}
> with {color:#de350b}*ignores*{color}
> * *contrib/langid*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/prometheus-exporter*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> * *contrib/velocity*: near {color:#00875a}*solid*{color} with
> {color:#de350b}*ignores*{color}
> _* Running tests quickly and efficiently with strict policing will more
> frequently find bugs and requires a period of hardening._
> _** Non Nightly currently, Nightly comes last._
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
