Re: First class support for node roles

2021-10-27 Thread Gus Heck
On Wed, Oct 27, 2021 at 3:34 PM Houston Putman wrote: > I don't think it's unreasonable to want to have nodes that don't accept >> queries. This is just ishan's use case. > > > Maybe I am misunderstanding, and it does deal with your last point about > inter-node communication, but Peer-sync uses

Re: Should v2 API be "experimental"

2021-10-27 Thread David Smiley
I agree that *very* few users use V2. Ok I do at work but it's maybe one API endpoint so whatever. We should free ourselves from the burdensome constraints of back-compat until V2 is sufficiently ready, whenever that is. So I agree with labeling it experimental and we can elaborate on that is th

Re: Should v2 API be "experimental"

2021-10-27 Thread Ishan Chattopadhyaya
I'm +1 to ensuring V2 APIs are ready by everyone's definition before 9.0. Hence I'm +1 to treat it as a blocker. However, if others feel we shouldn't hold it up, I'll likely not care about V2 APIs for another 2 years (by 10.x timeframe). On Thu, 28 Oct, 2021, 2:12 am Ishan Chattopadhyaya, < ichatt

Re: Should v2 API be "experimental"

2021-10-27 Thread Ishan Chattopadhyaya
My main concern is if we don't put it out there as front and center and hence don't allow users to use it, what motivation does that leave for developers to continue working on it? If it is not fully ready, some of us don't want it to be out there prominently for users to use it. Hence, if I can't

Re: First class support for node roles

2021-10-27 Thread Houston Putman
> > I don't think it's unreasonable to want to have nodes that don't accept > queries. This is just ishan's use case. Maybe I am misunderstanding, and it does deal with your last point about inter-node communication, but Peer-sync uses queries when doing replication between replicas. If a node do

Re: First class support for node roles

2021-10-27 Thread Gus Heck
On Wed, Oct 27, 2021 at 2:44 PM Houston Putman wrote: > As for the "query" role, let's name it something better like "compute", > since data nodes are always going to be "querying". > I don't think it's unreasonable to want to have nodes that don't accept queries. This is just ishan's use case.

Re: Should v2 API be "experimental"

2021-10-27 Thread Jason Gerlowski
> For whatever functionality is exposed via V2 APIs, it is ready. You lose me right around here Ishan. IMO exposing functionality makes a particular V2 API "usable", it doesn't necessarily make it "ready". "Ready" implies some confidence that I personally don't have without our test randomization

Re: First class support for node roles

2021-10-27 Thread Gus Heck
Excellent to see what you are thinking there, some of it seems pretty specific to your case but setting aside the specialized aggregation functionality and casting it in my proposed refinement seems to only require: My preference would be as follows: > * I'd like to refer to Layer1 nodes as the "d

Re: First class support for node roles

2021-10-27 Thread Houston Putman
As for the "query" role, let's name it something better like "compute", since data nodes are always going to be "querying". "compute" is for something like the first node for a distributed query, or a StreamingExpressions query. But I agree with the idea that roles should only be "positive", you s

Re: First class support for node roles

2021-10-27 Thread Ishan Chattopadhyaya
bq. In other words, roles are all "positive", but their consequences are only negative (rejecting when the matching positive role is not present). Essentially, yes. A node that doesn't specify any role should be able to do everything. Let me just take a brief detour and mention our thoughts on th

Re: Should v2 API be "experimental"

2021-10-27 Thread Eric Pugh
I thought about fixing SOLR-14795 when I documented the v2 apis, and the amount of change and the “ugh, it will be hard to have back compatibility” led me to not trying to move any of the sub tickets forward. If the V2 version of the API’s is more open to change, then it would make it easier

Re: Should v2 API be "experimental"

2021-10-27 Thread Alexandre Rafalovitch
Hmm, My understanding was that V2 Config API is part of the V2 APIs and relevant discussion, so when the questions about gaps came up, I felt it was relevant. Perhaps it is less relevant than I thought. I will let others judge and apologize if I introduced too much noise. Regards, Alex. On We

Re: First class support for node roles

2021-10-27 Thread Gus Heck
> In other words, roles are all "positive", but their consequences are only > negative (rejecting when the matching positive role is not present). > > Yeah right. to do something the machine needs the role > We can also consider no role defined = all roles allowed. Will make things > simpler. >

Re: First class support for node roles

2021-10-27 Thread Ilan Ginzburg
In other words, roles are all "positive", but their consequences are only negative (rejecting when the matching positive role is not present). We can also consider no role defined = all roles allowed. Will make things simpler. On Wed, Oct 27, 2021 at 6:14 PM Ilan Ginzburg wrote: > How do we exp

Re: Should v2 API be "experimental"

2021-10-27 Thread Gus Heck
> But I think there's a strong case that these users are a tiny > tiny minority out there: see the past lack of v2 docs, SolrJ support, > mailing list traffic or bug reports on v2, etc. > Well I'm guilty of having documented it in documentation I wrote or significantly modified, which ironically

Re: First class support for node roles

2021-10-27 Thread Ilan Ginzburg
How do we expect the roles to be used? One way I see is a node refusing to do anything related to a role it doesn't have. For example if a node does not have role "data", any attempt to create a core on it would fail. A node not having the role "query", will refuse to have anything to do with handl

Re: Should v2 API be "experimental"

2021-10-27 Thread Ishan Chattopadhyaya
Alex, these seem to be issues with config API (and should be solved), while this discussion is about v2 version of all APIs. What is the relevance here? On Wed, 27 Oct, 2021, 9:24 pm Alexandre Rafalovitch, wrote: > I feel that the summary of my umbrella case (SOLR-14795) qualifies for > this: >

Re: Should v2 API be "experimental"

2021-10-27 Thread Alexandre Rafalovitch
I feel that the summary of my umbrella case (SOLR-14795) qualifies for this: " General issues with output being materialized schema: * parameters have already been resolved and are not indicated * empty keys may not be output (e.g. dataDir) * default parameters will be output that are not in solrc

Re: First class support for node roles

2021-10-27 Thread Gus Heck
On Wed, Oct 27, 2021 at 9:55 AM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Hi Gus, > > > I think that we should expand/edit your list of roles to be > > The list can be expanded as and when more isolation and features are > needed. I only listed those roles that we already have a f

Re: Should v2 API be "experimental"

2021-10-27 Thread Ishan Chattopadhyaya
> some aspects of the v2 API are clearly downgrades from the v1 API. Please open a JIRA, and we can discuss there. If there's already any discussion, please point to them. > My big concerns are purely "cosmetic" (HTTP Methods, API paths, HTTP Status Codes, etc). Patches welcome. Or if you open a

Re: Should v2 API be "experimental"

2021-10-27 Thread Ishan Chattopadhyaya
> While that does mean that users will be hesitant to move to it for the time > being, isn't that kind of the point since we are planning on changing things? Exactly, that's the point of major release upgrades. Users will remain sceptical of moving production to 9x during the early few releases an

Re: Should v2 API be "experimental"

2021-10-27 Thread Houston Putman
Personally I think, while it made a lot of improvements, some aspects of the v2 API are clearly downgrades from the v1 API. I do think we should move towards deprecating the v1 API, but not until we have a solid (and documented of course) v2 or v3 API. Instead of worrying about v3, I think we shou

Re: Should v2 API be "experimental"

2021-10-27 Thread Jason Gerlowski
> unfortunately it's been released without that moniker for a long time ... we > may need to go v3 if we want to change things since people will > understandably have written code against it. The idea of breaking "existing v2 users" is definitely a sticking point. But I think there's a strong c

Re: Should v2 API be "experimental"

2021-10-27 Thread Ishan Chattopadhyaya
> While the v2 has been out for a long time, do we actually have evidence that it is widely used and has significant code written against it? I don't think anyone uses V2 APIs. No one wants to use undocumented APIs. > I worry that deciding to go with v3 it going to prevent any forward progress I

Re: Should v2 API be "experimental"

2021-10-27 Thread Eric Pugh
While the v2 has been out for a long time, do we actually have evidence that it is widely used and has significant code written against it? When I look at various components/packages that have been written around Solr, I don’t see the v2 API used. For example, Project Blacklight, a UI for Sol

Re: First class support for node roles

2021-10-27 Thread Ishan Chattopadhyaya
Hi Gus, > I think that we should expand/edit your list of roles to be The list can be expanded as and when more isolation and features are needed. I only listed those roles that we already have a functionality for or is under development. > I would like to recommend that the roles be all positiv

Re: Should v2 API be "experimental"

2021-10-27 Thread Gus Heck
And I think v1/v2 should be split into their own servlets leveraging common code by calling utilities, or composing with other objects rather than inheriting and getting in each other's way. I think v2 could change a lot so experimental seems appropriate, but unfortunately it's been released withou

Re: First class support for node roles

2021-10-27 Thread Gus Heck
The SIP looks like a good start, and I was already thinking of something very similar to this as a follow on to my attempts to split the uber filter (SolrDispatchFilter) into servlets such that roles determine what servlets are deployed, but I would like to recommend that the roles be all positive

First class support for node roles

2021-10-27 Thread Ishan Chattopadhyaya
Hi, Here's an SIP for introducing the concept of node roles: https://issues.apache.org/jira/browse/SOLR-15694 https://cwiki.apache.org/confluence/display/SOLR/SIP-15+Node+roles We also wish to add first class support for Query nodes that are used to process user queries by forwarding to data node