We don't need to know everything about SAI's performance profile to plan and execute some small, reasonable things now for 5.0. I'm going to try to summarize the least controversial package of ideas from the discussion above. I've left out creating any new syntax. For example, I think CREATE LOCAL INDEX, while explicit, is just not necessary. We don't use CREATE LOCAL TABLE, although it has the same locality as our indexes.
Okay, so the proposal for 5.0... 1.) Add a YAML option that specifies a default implementation for CREATE INDEX, and make this the legacy 2i for now. No existing DDL breaks. We don't have to commit to the absolute superiority of SAI. 2.) Add USING...WITH... support to CREATE INDEX, so we don't have to go to market using CREATE CUSTOM INDEX, which feels...not so polished. (The backend for this already exists w/ CREATE CUSTOM INDEX.) 3.) Leave in place but deprecate (client warnings could work?) CREATE CUSTOM INDEX. Support the syntax for the foreseeable future. Can we live w/ this? I don't think any information about SAI we could possibly acquire before a 5.0 release would affect the reasonableness of this much. On Fri, May 12, 2023 at 10:54 AM Benedict <bened...@apache.org> wrote: > if we didn't have copious amounts of (not all public, I know, working on > it) evidence > > > If that’s the assumption on which this proposal is based, let’s discuss > the evidence base first, as given the fundamentally different way they work > (almost diametrically opposite), I would want to see a very high quality of > evidence to support the claim. > > I don’t think we can resolve this conversation effectively until this > question is settled. > > On 12 May 2023, at 16:19, Caleb Rackliffe <calebrackli...@gmail.com> > wrote: > > > > This creates huge headaches for everyone successfully using 2i today > though, and SAI *is not* guaranteed to perform as well or better - it has a > very different performance profile. > > We wouldn't have even advanced it to this point if we didn't have copious > amounts of (not all public, I know, working on it) evidence it did for the > vast majority of workloads. Having said that, I don't strongly agree that > we should make it the default in 5.0, because performance isn't the only > concern. (correctness, DDL back-compat, which we've sort of touched w/ the > YAML default option, etc.) > > This conversation is now going in like 3 different directions, or at least > 3 different "packages" of ideas, so there isn't even a single thing to vote > on. Let me read through again and try to distill into something that we > might be able to do so with... > > On Fri, May 12, 2023 at 7:56 AM Aleksey Yeshchenko <alek...@apple.com> > wrote: > >> This. >> >> I would also consider adding CREATE LEGACY INDEX syntax as an alias for >> today’s CREATE INDEX, the latter to be deprecated and (in very distant >> future) removed. >> >> On 12 May 2023, at 13:14, Benedict <bened...@apache.org> wrote: >> >> This creates huge headaches for everyone successfully using 2i today >> though, and SAI *is not* guaranteed to perform as well or better - it has a >> very different performance profile. >> >> I think we should deprecate CREATE INDEX, and introduce new syntax CREATE >> LOCAL INDEX to make clear that this is not a global index, and that this >> should require the USING syntax to avoid this problem in future. >> >> We should report warnings to the client when CREATE INDEX is used, >> indicating it is deprecated. >> >> >>
