+1 to both of Anil's points. On Fri, Sep 8, 2017 at 3:04 PM, Anilkumar Gingade <aging...@pivotal.io> wrote:
> Indexes are critical for querying; most of the databases doesn't allow > insert/update if there is any failure with index maintenance... > > As Geode OQL supports two ways (sync and async) to maintain the indexes, we > need be careful about the error handling in both cases... > > My take is: > 1. For synchronous index maintenance: > If there is any failure in updating any index (security/auth or logical > error) on the region; throw an exception and rollback the cache update/op > (index management id done under region.entry lock - we should be able to > revert the op). If index or cache is left in bad state, then its a bug that > needs to be addressed. > > Most of the time, If there is any logical error in index, it will be > detected as soon as index is created (on existing data) or when first > update is done to the cache. > > 2. For Asynchronous index maintenance: > As this is async (assuming) user has good understanding of the risk > involved with async, any error with index maintenance, the index should be > invalidated... > > About the security/auth, the user permission with region read/write needs > to be applied for index updates, there should not be different permission > on index. > > -Anil. > > > > On Fri, Sep 8, 2017 at 2:01 PM, Nabarun Nag <n...@pivotal.io> wrote: > > > Hi Mike, > > > > Please do find our answers below: > > *Question:* What if there were multiple indices that were in flight and > > only the third > > one errors out, will they all be marked invalid? > > > > *Answer:* Only the third will be marked invalid and only the third one > will > > not be used for query execution. > > > > *Question/Statement:* If anything goes wrong with the put it should > > probably still throw back to > > the caller. Silent invalidation of the index is probably not desirable. > > > > *Answer: * > > In our current design this the flow of execution of a put operation: > > entry put into region -> update index -> other wan related executions / > > callbacks etc. > > > > If an exception happens while updating the index, the cache gets into a > bad > > state, and we may end up getting different results depending on the index > > we are using. As the failure happens half way in a put operation, the > > regions / cache are now in a bad state. > > -------------------------- > > We are thinking that if index is created over a method invocation in an > > empty region and then we do puts, but method invocation is not allowed as > > per security policies. The puts will now be successful but the index will > > be rendered invalid. Previously the puts will fail with exception and put > > the entire cache in a bad state. > > > > > > > > Regards > > Nabarun > > > > > > > > > > > > On Fri, Sep 8, 2017 at 10:43 AM Michael Stolz <mst...@pivotal.io> wrote: > > > > > Just to help me understand, the index is corrupted in a way beyond just > > the > > > field that errors out? > > > What if there were multiple indices that were in flight and only the > > third > > > one errors out, will they all be marked invalid? > > > If anything goes wrong with the put it should probably still throw back > > to > > > the caller. Silent invalidation of the index is probably not desirable. > > > > > > -- > > > Mike Stolz > > > Principal Engineer, GemFire Product Manager > > > Mobile: +1-631-835-4771 <(631)%20835-4771> > > > > > > On Fri, Sep 8, 2017 at 12:34 PM, Dan Smith <dsm...@pivotal.io> wrote: > > > > > > > +1 > > > > > > > > -Dan > > > > > > > > On Thu, Sep 7, 2017 at 9:14 PM, Nabarun Nag <n...@apache.org> wrote: > > > > > > > > > *Proposal:* > > > > > * Index interface will include an API - isValid() which will return > > > true > > > > if > > > > > the index is still valid / uncorrupted, else will return false if > it > > > > > corrupted / invalid. > > > > > * gfsh command "list index" will have one more column "Is Valid" > > which > > > > will > > > > > state the status as "true" or "false". > > > > > * Invalid indexes will not be used during query executions. > > > > > * In case the index is found to be invalid, the user will be able > to > > > > > remove/destroy the index. > > > > > * When a put operation corrupts an index, it will be logged. > > > > > > > > > > *Reasoning:* > > > > > * Currently if a put operation raises an exception while updating > the > > > > > index, the put operation fails with an exception to the putter. > > > > > * For example, if an index is created on an object method, and that > > > > method > > > > > causes an exception while updating the index as a part of a put > > > > operation, > > > > > then the put operation fails for that particular entry and the > index > > is > > > > > left in a bad state. > > > > > * This may occur also due to lack of permission to update index but > > > have > > > > > permission to do puts. > > > > > * We are proposing that in the above mentioned scenarios, the put > > > > succeeds > > > > > in putting the entry in the region but the index which it was > trying > > to > > > > > update will be marked invalid and will not be used for query > > > executions. > > > > > * This can be justified because the corrupted index will not > > correctly > > > > > represent the region entries. > > > > > > > > > > *Status Quo:* > > > > > * Index creation will still fail if we are trying to create an > index > > > > over a > > > > > region containing an entry/entries which will cause an exception > > while > > > > > loading the entry into the index. > > > > > > > > > > Regards > > > > > Nabarun Nag > > > > > > > > > > > > > > > -- -John john.blum10101 (skype)
