Re: SIP-7 New Admin UI

2024-07-17 Thread Eric Pugh
I appreciate the vote for Kotlin Jan!

I hadn’t really thought about the argument that if our base is primarily Java 
centric devs, that Kotlin versus React might be an easier route….


> On Jul 16, 2024, at 11:56 PM, Christos Malliaridis  
> wrote:
> 
> Thanks for your reply too, Jan. You mention important points.
> 
>> If frontend devs are to maintain it and we want to attract frontend
> professionals, then stick to industry standard React or similar.
> 
> This is something that troubled me a lot. Following the industry-standards
> would probably be the safest path. But getting new frontend developers may
> also be challenging and would probably limit their contribution to only the
> frontend, which is why I think focusing on the current maintainers
> would be safe
> alternative too.
> 
>> I did Swing programming, which is hopefully far from what Compose offers
> 
> I also developed a UI with Swing a couple years ago, and I am glad Compose
> does things different.
> 
>> I like the simplificy of the current UI
> 
> I was surprised when I saw it. I think it will be difficult to achieve
> similar results in simplicity with modern frameworks.
> 
>> I'd like for the UI to be served by a separate servlet/backend that acts
> as a proxy, so that the Admin UI could be installed separately in a DMZ
> network
> 
> This is something I had to give some thought first. I believe that the UI
> should be implemented against the API, regardless of the deployment method.
> It should probably not be the responsibility of the UI how or where it is
> deployed, but it should be flexible enough so that it can work no matter
> the environment. I agree that there are probably better ways
> (security-wise) to host it, but this is a matter of abstraction and should
> be addressed separately (e.g. different environments may use different
> proxies).
> 
>> If we managed to separate the new UI as an independent servlet, perhaps
> with its own /login logic, it would be so much easier to later move the
> entire UI to a separate repo, should the need arise.
> 
> I think it's already quite easy to deploy the UI outside or move the source
> code to a separate project thanks to Gradle modules and the API
> abstraction. My thoughts from above apply here too and this proxy could be
> addressed while the new UI is under development (e.g. in combination with a
> desktop client for example).
> 
>> just skip the "new" keywork and semicolons, hehe
> 
> That's a great way to describe Kotlin to Java devs. :D
> On Mon, 15 Jul 2024, 22:39 Jan Høydahl,  wrote:
> 
>>> - What technology stack would you consider and why?
>>> - Would you consider a web-based / javascript-based framework easier to
>> get
>>> started with, or a JVM-based / kotlin-based UI framework?
>> 
>> I consider these related. And it boils down to who will maintain the Admin
>> UI app.
>> If frontend devs are to maintain it and we want to attract frontend
>> professionals, then stick to industry standard React or similar.
>> However, for the lifetime of the project it's been Java devs who have
>> maintained the UI with a few huge re-writes being the exceptions, but those
>> were mostly one-offs or at least one-person.
>> 
>> So I see why you propse the Kotlin based Compose. In a previous life in
>> the 90's with Java 1, I did Swing programming, which is hopefully far from
>> what Compose offers, which I know nothing about.
>> 
>>> - What was your experience so far with Solr's UI code? What would you
>> avoid
>>> doing again, what did you like before?
>> 
>> I helped patch both the jQuery and the Angular UI. Notable contributions
>> include the OIDC auth impl, the Nodes screen and the ZK Status page.
>> On one side I like the simplificy of the current UI, just some JS/HTML/CSS
>> files, not build, compiling etc.
>> On the other hand, it makes it hard to add 3rd party modules, upgrade libs
>> etc.
>> I dislike the fact that the UI is hosted by the main Solr process and
>> talks directly to Solr backend APIs. I'd like for the UI to be served by a
>> separate servlet/backend that acts as a proxy, so that the Admin UI could
>> be installed separately in a DMZ network and poke a hole in firewalls
>> between the AdminUI's own backend and the Solr cluster (which would be on a
>> secure inner network).
>> 
>> If we managed to separate the new UI as an independent servlet, perhaps
>> with its own /login logic, it would be so much easier to later move the
>> entire UI to a separate repo, should the need arise.
>> 
>>> - Would you be interested in contributing to the UI implementation?
>> 
>> I could probably lend a helping hand here and there, do some reviews, and
>> if we manage to partition the elephant, pick a few tasks further down the
>> road.
>> 
>> I do Kotlin in day job, and it is an absolute joy to work with. Not hard
>> at all, so to committers fearing a "new" language, it is actually not that
>> different, just skip the "new" keywork and semicolons, hehe.
>> 
>> Jan Høydahl
>> 
>>> 15. 

Solr exception while deployment in K8s

2024-07-17 Thread Mugi, Krishnavamsireddy
Hi Team,

We are trying to deploy one of solr site in K8s, But we are seeing below 
exception while deployment. We are migrating solr from 8.11.2 to 9.6.1. Can you 
please help me on this?

2024-07-16 15:07:57.314 WARN  (main) [   ] o.e.j.w.WebAppContext Failed startup 
of context 
o.e.j.w.WebAppContext@4bdb04c8{solr-jetty-context.xml,/solr,file:///opt/solr-9.6.1/server/solr-webapp/webapp/,UNAVAILABLE}{/opt/solr-9.6.1/server/solr-webapp/webapp}
 => java.lang.VerifyError: Bad type on operand stack
Exception Details:
java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:

org/eclipse/jetty/servlet/listener/ELContextCleaner.contextDestroyed(Ljavax/servlet/ServletContextEvent;)V
 @157: invokeinterface
  Reason:
Type 'java/lang/Object' (current frame, stack[2]) is not assignable to 
'java/lang/Throwable'
  Current Frame:
bci: @157
flags: { }
locals: { 'org/eclipse/jetty/servlet/listener/ELContextCleaner', 
'javax/servlet/ServletContextEvent', 'com/newrelic/agent/bridge/ExitTracer', 
integer, top, 'javax/servlet/ServletContextEvent', 
'org/eclipse/jetty/servlet/listener/ELContextCleaner', 'java/lang/Object' }
stack: { 'org/slf4j/Logger', 'java/lang/String', 'java/lang/Object' }
  Bytecode:
000: 014d b200 272a 1009 0110 0eb9 003b 0500
010: 4da7 0004 5703 3eb8 003f b900 4501 00b2
020: 004b 0312 4d05 bd00 4f59 032a b600 53b6
030: 0059 5359 0412 7b53 b900 6005 0057 a700
040: 163a 04b2 0027 1904 1228 b900 2e03 00a7
050: 0003 043e 2a2b 3a05 3a06 127d b800 833a
060: 0719 0619 07b6 0087 3a08 1908 04b6 008d
070: 1906 1908 b600 91b2 0093 b900 9901 0099
080: 000d b200 9312 9bb9 009f 0200 a700 253a
090: 07a7 0020 3a07 b200 9312 a119 07b9 00a5
0a0: 0300 a700 0f3a 07b2 0093 12a7 b900 9f02
0b0: 00a7 0003 031d 9f00 122c c600 0d2c 1100
0c0: b101 b900 6603 00b1 2cc6 000d 2c11 00b1
0d0: 01b9 0066 0300 b13a 04b2 0027 1904 1228
0e0: b900 2e03 00a7 0003 2cc6 000d 2c11 00b1
0f0: 01b9 0066 0300 b12c c600 0b59 2c5f b900
100: 6902 00bf
  Exception Handler Table:
bci [2, 21] => handler: 20
bci [90, 140] => handler: 143
bci [90, 140] => handler: 148
bci [90, 140] => handler: 148
bci [90, 140] => handler: 148
bci [90, 140] => handler: 165
bci [23, 65] => handler: 65
bci [200, 215] => handler: 215
bci [2, 247] => handler: 247
  Stackmap Table:
full_frame(@20,{Object[#3],Object[#111],Object[#98]},{Object[#31]})
same_frame(@21)
full_frame(@65,{Object[#3],Object[#111],Object[#98],Integer},{Object[#31]})
append_frame(@82,Object[#31])
chop_frame(@84,1)

full_frame(@140,{Object[#3],Object[#111],Object[#98],Integer,Top,Object[#111],Object[#3],Object[#85],Object[#137]},{})

full_frame(@143,{Object[#3],Object[#111],Object[#98],Integer,Top,Object[#111],Object[#3]},{Object[#114]})
same_locals_1_stack_item_frame(@148,Object[#5])
same_locals_1_stack_item_frame(@165,Object[#122])
append_frame(@177,Object[#5])
same_frame(@180)
same_frame(@199)
same_frame(@200)
same_frame(@214)
same_locals_1_stack_item_frame(@215,Object[#31])

full_frame(@232,{Object[#3],Object[#111],Object[#98],Integer,Object[#31],Object[#111],Object[#3],Object[#5]},{})
same_frame(@246)
full_frame(@247,{Object[#3],Object[#111],Object[#98]},{Object[#31]})
same_locals_1_stack_item_frame(@259,Object[#31])

  at java.base/java.lang.Class.getDeclaredConstructors0(Native 
Method) ~[?:?]
  at 
java.base/java.lang.Class.privateGetDeclaredConstructors(Unknown Source) ~[?:?]
  at java.base/java.lang.Class.getConstructor0(Unknown Source) 
~[?:?]
  at java.base/java.lang.Class.getDeclaredConstructor(Unknown 
Source) ~[?:?]
  at 
org.eclipse.jetty.server.handler.ContextHandler$StaticContext.createInstance(ContextHandler.java:2900)
 ~[jetty-server-10.0.20.jar:10.0.20]
  at 
org.eclipse.jetty.servlet.ServletContextHandler$Context.createInstance(ServletContextHandler.java:1292)
 ~[jetty-servlet-10.0.20.jar:10.0.20]
  at 
org.eclipse.jetty.servlet.ServletContextHandler$Context.createInstance(ServletContextHandler.java:1301)
 ~[jetty-servlet-10.0.20.jar:10.0.20]
  at 
org.eclipse.jetty.servlet.BaseHolder.createInstance(BaseHolder.java:204) 
~[jetty-servlet-10.0.20.jar:10.0.20]
  at 
org.eclipse.jetty.servlet.ListenerHolder.createInstance(ListenerHolder.java:100)
 ~[jetty-servlet-10.0.20.jar:10.0.20]
  at 
org.eclipse.jetty.servlet.ListenerHolder.doStart(ListenerHolder.java:89) 
~[jetty-servlet-10.0.20.jar:10.0.20]
  at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
 ~[je

Re: [DISCUSS] Solr 9.7 release

2024-07-17 Thread Jan Høydahl
Hi,

Sounds ok. 

Will try to merge some of the latest dependency upgrades. There are still ~30 
dependency PRs, so anyone feel free to jump in and resolve some of them before 
the release.

Jan

> 16. juli 2024 kl. 22:12 skrev Anshum Gupta :
> 
> Hi everyone,
> 
> The Change log for Solr 9.7 looks pretty good already with the Lucene
> upgrade and a bunch of other fixes, improvements, and features.
> 
> I'd like to start the release process next *Tuesday, July 23 *unless there
> are objections or reasons to wait.
> 
> -Anshum


-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org



Re: [DISCUSS] Solr 9.7 release

2024-07-17 Thread David Smiley
There is at least one blocker, a notable one:
https://issues.apache.org/jira/browse/SOLR-13350 (search segments in
parallel)

On Tue, Jul 16, 2024 at 4:12 PM Anshum Gupta  wrote:
>
> Hi everyone,
>
> The Change log for Solr 9.7 looks pretty good already with the Lucene
> upgrade and a bunch of other fixes, improvements, and features.
>
> I'd like to start the release process next *Tuesday, July 23 *unless there
> are objections or reasons to wait.
>
> -Anshum

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org



Re: [DISCUSS] Solr 9.7 release

2024-07-17 Thread Eric Pugh
I would like to see the back port of solr cli changes make it…
https://github.com/apache/solr/pull/2540

Thanks to some great work from Jan I suspect it can be committed this week.



On Wed, Jul 17, 2024 at 6:55 PM David Smiley  wrote:

> There is at least one blocker, a notable one:
> https://issues.apache.org/jira/browse/SOLR-13350 (search segments in
> parallel)
>
> On Tue, Jul 16, 2024 at 4:12 PM Anshum Gupta  wrote:
> >
> > Hi everyone,
> >
> > The Change log for Solr 9.7 looks pretty good already with the Lucene
> > upgrade and a bunch of other fixes, improvements, and features.
> >
> > I'd like to start the release process next *Tuesday, July 23 *unless
> there
> > are objections or reasons to wait.
> >
> > -Anshum
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>


Re: Solr exception while deployment in K8s

2024-07-17 Thread Houston Putman
How are you deploying Solr to Kubernetes?

Are you using the official Solr docker image? Are you using the Solr
Operator?

- Houston

On Wed, Jul 17, 2024 at 9:49 AM Mugi, Krishnavamsireddy <
krishnavamsireddy.m...@paramount.com> wrote:

> Hi Team,
>
> We are trying to deploy one of solr site in K8s, But we are seeing below
> exception while deployment. We are migrating solr from 8.11.2 to 9.6.1. Can
> you please help me on this?
>
> 2024-07-16 15:07:57.314 WARN  (main) [   ] o.e.j.w.WebAppContext Failed
> startup of context o.e.j.w.WebAppContext@4bdb04c8
> {solr-jetty-context.xml,/solr,file:///opt/solr-9.6.1/server/solr-webapp/webapp/,UNAVAILABLE}{/opt/solr-9.6.1/server/solr-webapp/webapp} o.e.j.w.WebAppContext@4bdb04c8%7bsolr-jetty-context.xml,/solr,file:///opt/solr-9.6.1/server/solr-webapp/webapp/,UNAVAILABLE%7d%7b/opt/solr-9.6.1/server/solr-webapp/webapp%7d>
> => java.lang.VerifyError: Bad type on operand stack
> Exception Details:
> java.lang.VerifyError: Bad type on operand stack
> Exception Details:
>   Location:
>
> org/eclipse/jetty/servlet/listener/ELContextCleaner.contextDestroyed(Ljavax/servlet/ServletContextEvent;)V
> @157: invokeinterface
>   Reason:
> Type 'java/lang/Object' (current frame, stack[2]) is not assignable to
> 'java/lang/Throwable'
>   Current Frame:
> bci: @157
> flags: { }
> locals: { 'org/eclipse/jetty/servlet/listener/ELContextCleaner',
> 'javax/servlet/ServletContextEvent',
> 'com/newrelic/agent/bridge/ExitTracer', integer, top,
> 'javax/servlet/ServletContextEvent',
> 'org/eclipse/jetty/servlet/listener/ELContextCleaner', 'java/lang/Object' }
> stack: { 'org/slf4j/Logger', 'java/lang/String', 'java/lang/Object' }
>   Bytecode:
> 000: 014d b200 272a 1009 0110 0eb9 003b 0500
> 010: 4da7 0004 5703 3eb8 003f b900 4501 00b2
> 020: 004b 0312 4d05 bd00 4f59 032a b600 53b6
> 030: 0059 5359 0412 7b53 b900 6005 0057 a700
> 040: 163a 04b2 0027 1904 1228 b900 2e03 00a7
> 050: 0003 043e 2a2b 3a05 3a06 127d b800 833a
> 060: 0719 0619 07b6 0087 3a08 1908 04b6 008d
> 070: 1906 1908 b600 91b2 0093 b900 9901 0099
> 080: 000d b200 9312 9bb9 009f 0200 a700 253a
> 090: 07a7 0020 3a07 b200 9312 a119 07b9 00a5
> 0a0: 0300 a700 0f3a 07b2 0093 12a7 b900 9f02
> 0b0: 00a7 0003 031d 9f00 122c c600 0d2c 1100
> 0c0: b101 b900 6603 00b1 2cc6 000d 2c11 00b1
> 0d0: 01b9 0066 0300 b13a 04b2 0027 1904 1228
> 0e0: b900 2e03 00a7 0003 2cc6 000d 2c11 00b1
> 0f0: 01b9 0066 0300 b12c c600 0b59 2c5f b900
> 100: 6902 00bf
>   Exception Handler Table:
> bci [2, 21] => handler: 20
> bci [90, 140] => handler: 143
> bci [90, 140] => handler: 148
> bci [90, 140] => handler: 148
> bci [90, 140] => handler: 148
> bci [90, 140] => handler: 165
> bci [23, 65] => handler: 65
> bci [200, 215] => handler: 215
> bci [2, 247] => handler: 247
>   Stackmap Table:
> full_frame(@20,{Object[#3],Object[#111],Object[#98]},{Object[#31]})
> same_frame(@21)
>
> full_frame(@65,{Object[#3],Object[#111],Object[#98],Integer},{Object[#31]})
> append_frame(@82,Object[#31])
> chop_frame(@84,1)
>
> full_frame(@140,{Object[#3],Object[#111],Object[#98],Integer,Top,Object[#111],Object[#3],Object[#85],Object[#137]},{})
>
> full_frame(@143,{Object[#3],Object[#111],Object[#98],Integer,Top,Object[#111],Object[#3]},{Object[#114]})
> same_locals_1_stack_item_frame(@148,Object[#5])
> same_locals_1_stack_item_frame(@165,Object[#122])
> append_frame(@177,Object[#5])
> same_frame(@180)
> same_frame(@199)
> same_frame(@200)
> same_frame(@214)
> same_locals_1_stack_item_frame(@215,Object[#31])
>
> full_frame(@232,{Object[#3],Object[#111],Object[#98],Integer,Object[#31],Object[#111],Object[#3],Object[#5]},{})
> same_frame(@246)
> full_frame(@247,{Object[#3],Object[#111],Object[#98]},{Object[#31]})
> same_locals_1_stack_item_frame(@259,Object[#31])
>
>   at java.base/java.lang.Class.getDeclaredConstructors0(Native
> Method) ~[?:?]
>   at
> java.base/java.lang.Class.privateGetDeclaredConstructors(Unknown Source)
> ~[?:?]
>   at java.base/java.lang.Class.getConstructor0(Unknown Source)
> ~[?:?]
>   at java.base/java.lang.Class.getDeclaredConstructor(Unknown
> Source) ~[?:?]
>   at
> org.eclipse.jetty.server.handler.ContextHandler$StaticContext.createInstance(ContextHandler.java:2900)
> ~[jetty-server-10.0.20.jar:10.0.20]
>   at
> org.eclipse.jetty.servlet.ServletContextHandler$Context.createInstance(ServletContextHandler.java:1292)
> ~[jetty-servlet-10.0.20.jar:10.0.20]
>   at
> org.eclipse.jetty.servlet.ServletContextHandler$Context.createInstance(ServletContextHandler.java:1301)
> ~[jetty-servlet-10.0.20.jar:10.0.20]
>   at
> org.eclipse.jetty.servlet.BaseHolder.createInstance(BaseHolder.java:204)
> ~[jetty-servlet-10.0.

Re: SIP-7 New Admin UI

2024-07-17 Thread David Smiley
+1 for Kotlin; I've used it in the past and was fond of the
experience.  I'm sad we don't use it in our Gradle files.
But take this with a grain of salt:  I don't *intend* to work on the
UI to be honest so my opinion of the tech there doesn't matter much.
I suppose I might end up touching it if I work on something that the
UI exposes.

On Wed, Jul 17, 2024 at 3:37 AM Eric Pugh
 wrote:
>
> I appreciate the vote for Kotlin Jan!
>
> I hadn’t really thought about the argument that if our base is primarily Java 
> centric devs, that Kotlin versus React might be an easier route….
>
>
> > On Jul 16, 2024, at 11:56 PM, Christos Malliaridis 
> >  wrote:
> >
> > Thanks for your reply too, Jan. You mention important points.
> >
> >> If frontend devs are to maintain it and we want to attract frontend
> > professionals, then stick to industry standard React or similar.
> >
> > This is something that troubled me a lot. Following the industry-standards
> > would probably be the safest path. But getting new frontend developers may
> > also be challenging and would probably limit their contribution to only the
> > frontend, which is why I think focusing on the current maintainers
> > would be safe
> > alternative too.
> >
> >> I did Swing programming, which is hopefully far from what Compose offers
> >
> > I also developed a UI with Swing a couple years ago, and I am glad Compose
> > does things different.
> >
> >> I like the simplificy of the current UI
> >
> > I was surprised when I saw it. I think it will be difficult to achieve
> > similar results in simplicity with modern frameworks.
> >
> >> I'd like for the UI to be served by a separate servlet/backend that acts
> > as a proxy, so that the Admin UI could be installed separately in a DMZ
> > network
> >
> > This is something I had to give some thought first. I believe that the UI
> > should be implemented against the API, regardless of the deployment method.
> > It should probably not be the responsibility of the UI how or where it is
> > deployed, but it should be flexible enough so that it can work no matter
> > the environment. I agree that there are probably better ways
> > (security-wise) to host it, but this is a matter of abstraction and should
> > be addressed separately (e.g. different environments may use different
> > proxies).
> >
> >> If we managed to separate the new UI as an independent servlet, perhaps
> > with its own /login logic, it would be so much easier to later move the
> > entire UI to a separate repo, should the need arise.
> >
> > I think it's already quite easy to deploy the UI outside or move the source
> > code to a separate project thanks to Gradle modules and the API
> > abstraction. My thoughts from above apply here too and this proxy could be
> > addressed while the new UI is under development (e.g. in combination with a
> > desktop client for example).
> >
> >> just skip the "new" keywork and semicolons, hehe
> >
> > That's a great way to describe Kotlin to Java devs. :D
> > On Mon, 15 Jul 2024, 22:39 Jan Høydahl,  wrote:
> >
> >>> - What technology stack would you consider and why?
> >>> - Would you consider a web-based / javascript-based framework easier to
> >> get
> >>> started with, or a JVM-based / kotlin-based UI framework?
> >>
> >> I consider these related. And it boils down to who will maintain the Admin
> >> UI app.
> >> If frontend devs are to maintain it and we want to attract frontend
> >> professionals, then stick to industry standard React or similar.
> >> However, for the lifetime of the project it's been Java devs who have
> >> maintained the UI with a few huge re-writes being the exceptions, but those
> >> were mostly one-offs or at least one-person.
> >>
> >> So I see why you propse the Kotlin based Compose. In a previous life in
> >> the 90's with Java 1, I did Swing programming, which is hopefully far from
> >> what Compose offers, which I know nothing about.
> >>
> >>> - What was your experience so far with Solr's UI code? What would you
> >> avoid
> >>> doing again, what did you like before?
> >>
> >> I helped patch both the jQuery and the Angular UI. Notable contributions
> >> include the OIDC auth impl, the Nodes screen and the ZK Status page.
> >> On one side I like the simplificy of the current UI, just some JS/HTML/CSS
> >> files, not build, compiling etc.
> >> On the other hand, it makes it hard to add 3rd party modules, upgrade libs
> >> etc.
> >> I dislike the fact that the UI is hosted by the main Solr process and
> >> talks directly to Solr backend APIs. I'd like for the UI to be served by a
> >> separate servlet/backend that acts as a proxy, so that the Admin UI could
> >> be installed separately in a DMZ network and poke a hole in firewalls
> >> between the AdminUI's own backend and the Solr cluster (which would be on a
> >> secure inner network).
> >>
> >> If we managed to separate the new UI as an independent servlet, perhaps
> >> with its own /login logic, it would be so much easier to later mov

JIRA "pull-request-available" label

2024-07-17 Thread David Smiley
Our JIRA issues, according to .asf.yaml configuration, *should have*
been getting a label "pull-request-available".  This wasn't working
because an ASF bot needed committer permissions in JIRA, in accordance
with ASF docs on this.  I addressed that matter last night.  A nice
side-effect of the bot doing this is that it notifies anyone watching
the JIRA issue so that we're aware of the PR.  The PR auto-link
doesn't do that.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org



Re: ZkStateReader.getUpdateLock / ClusterState immutability

2024-07-17 Thread David Smiley
On Wed, Jul 17, 2024 at 2:31 AM Mark Miller  wrote:
> And if you really
> wanted to keep an immutable cluster state object, that still doesn’t mean
> ZkStateReader has to use one for its cluster state structure just because
> that’s what it gives out with getClusterState.

Right -- absolutely.  I _think_ most consumers can deal with
mutability fine; that there are relatively few consumers that require
a snapshot.  They should explicitly ask for that so that the
snapshotting cost isn't needlessly paid.

> The separation of replica states from cluster structure is useful in
> addressing an efficient cluster state structure and update strategy in
> ZkStateReader, if not just so that you know what you actually *need* to
> update. If you get a collection worth of JSON, you have to update it all or
> do some silly gymnastics to reverse engineer what the update actually is.

> For me, a concurrent hashmap in ZkStateReader was better than a cluster
> state object. It mapped a collection to some kind of collection state, and
> in the replica state, I put an atomic integer indicating the state. Then,
> you can throw out the global lock and forget about any lock or object
> creation when updating replica states.

You speak in the past tense as if it worked this way.  Maybe you mean
"would have been" and hopefully you still think so?

Practically speaking, I think "ClusterState" should continue to exist
but simply be mutable (containing a ConcurrentHashMap), effectively
shifting some of ZkStateReader's extensive responsibilities to this
collaborator.  No sense in updating lots of callers needlessly.

> The entire communication path can be made easily 100x+ more scalable with
> changes that are just as simple and straightforward. “Oh, this is crazy.
> And it’s easy to do something that’s not.” And without any brain sweat, you
> end up with a system that works in parallel on independent work, transmits
> state sizes close to the actual state that needs transmitting, doesn’t spam
> updates that are either unnecessary or already outdated, and operates at a
> designed developer drum beat rather than an arbitrary army of drummers.

Then I'm looking forward to seeing you take a stab at this ;-) even if
it's just a draft PR / straw-man.  You can count on me for giving a
code review.

~ David

-
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org