Can anyone help me understand the race condition?
My understanding was that in addition to setting the isInvalid flag, we
fire off a background task to drop the index. This task can use the
plumbing that gfsh uses to drop the index. Even if we schedule more than
one of these tasks, the later tasks
I agree with Dan and Naba. The possible race conditions seem very risky to
me. Memory optimization isn't really needed here because the user has
already budgeted memory for the index. They probably weren't trying to
create an index that would be corrupted and when it does get corrupted, it
won't
Scheduling an asynchronous task to drop the index everywhere seems like
it's adding a lot of complexity and potential for race conditions. Is it
really worth optimizing memory usage in this edge case where we hit an
exception updating an index? The objective of these changes is to fix a
window wher
Thanks for the explanation Naba, please find my reply below:
1. Debugging: If we log a warning, that should get noticed immediately, so
I don't think we need to worry about logs rolling.
2. Performance for a single put: We can always schedule an async task to
drop the index.
On Wed, Sep 20, 2017 a
Hi Swapnil,
There were few factors we considered before going with just invalidating
the index rather than destroying the index.
1. Debugging reasons : If the indexes were destroyed and logs roll over,
and suddenly we see that indexes have disappeared, it will be tough to
differentiate between whet
Agree, invalid index should not be kept...For PRs its tricky, each
bucket-region has its own index, if an index is invalidated on a bucket
region, do we need to remove that index from all the bucket regions (local
and remote)? I guess, yes...
-Anil.
On Wed, Sep 20, 2017 at 2:58 PM, Swapnil Bawas
Sorry for not reading this thread earlier, but I was wondering what is the
point of just invalidating the index and having it lie around if it is not
going to be used?
Can we just drop the index instead, and log a warning message to that
effect? This will free up the memory used by the index and wi
The PR #768 has been created for this issue and also GEODE-3520 has been
changed to reflect this requirement.
Regards
Nabarun
On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag wrote:
> Thanks you guys for the review. I will revert the GEODE-3520 ticket to
> reflect that invalidate should happen for b
Thanks you guys for the review. I will revert the GEODE-3520 ticket to
reflect that invalidate should happen for both synchronous and asynchronous
index maintenance.
Also, I will resend the PR which reflects the changes mentioned in the
ticket
Regards
Nabarun Nag
On Thu, Sep 14, 2017 at 5:55
Dan, you are right...Thanks to Jason, myself and Jason looked into the code
to see if index is updated before the event/change is saved/injected into
the region...It looks like the index update are happening after the region
change/update is saved. Moving the index update before that is not an easy
I'm still going to push that we stick with Naba's original proposal.
The current behavior is clearly broken. If one index update fails, an
exception gets thrown to the user (nice!) but it leaves the put in a
partially completed state - some other indexes may not have been updated,
WAN/AEQs may not
Anil, we actually do have a case where the index is out of sync with the
region currently. It's just not likely to happen but if there is an
exception from an index, the end result is that certain indexes get updated
and the region has already been updated.
However the exception is thrown back to
The other way to look at it is; what happens to a cache op; when there is
an exception after Region.Entry is created? can it happen? In that case, do
we stick the entry into the Cache or not? If an exception is handled, how
is it done, can we look at using the same for Index...
Also previously, on
I’m confused. Once a cache update has been distributed to other members it
can’t be undone. That update could have triggered myriad other application
behaviors.
Anthony
> On Sep 11, 2017, at 2:04 PM, Michael Stolz wrote:
>
> Great, that's exactly the behavior I would expect.
>
> Thanks.
>
Great, that's exactly the behavior I would expect.
Thanks.
--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: +1-631-835-4771
On Mon, Sep 11, 2017 at 4:34 PM, Jason Huynh wrote:
> Hi Mike, I think the concern was less about the security portion but rather
> if any exception occu
Hi Mike, I think the concern was less about the security portion but rather
if any exception occurs during index update, right now, the region gets
updated and the rest of the system (index/wan/callbacks) may or may not be
updated. I think Naba just tried to provide an example where this might
occ
I think there was an intention of having CREATION of an index require a
higher privilege than DATA:WRITE, but it shouldn't affect applying the
index on either of put or get operations.
If we are requiring something like CLUSTER:MANAGE for put on an indexed
region, that is an incorrect requirement.
There's no way to rollback an put/putAll, unless in TX.
On Sun, Sep 10, 2017 at 4:21 PM, Jason Huynh wrote:
> 1.) Does anyone know of a way to do a rollback where the put is already
> reflected in the region? If that is the desired behavior, then perhaps we
> will have to live with the current
1.) Does anyone know of a way to do a rollback where the put is already
reflected in the region? If that is the desired behavior, then perhaps we
will have to live with the current (leaving the region and indexes in a bad
state, wan and other callbacks that occur after index maintenance will not
+1 to both of Anil's points.
On Fri, Sep 8, 2017 at 3:04 PM, Anilkumar Gingade
wrote:
> Indexes are critical for querying; most of the databases doesn't allow
> insert/update if there is any failure with index maintenance...
>
> As Geode OQL supports two ways (sync and async) to maintain the ind
Indexes are critical for querying; most of the databases doesn't allow
insert/update if there is any failure with index maintenance...
As Geode OQL supports two ways (sync and async) to maintain the indexes, we
need be careful about the error handling in both cases...
My take is:
1. For synchrono
Hi Mike,
Please do find our answers below:
*Question:* What if there were multiple indices that were in flight and
only the third
one errors out, will they all be marked invalid?
*Answer:* Only the third will be marked invalid and only the third one will
not be used for query execution.
*Questio
Just to help me understand, the index is corrupted in a way beyond just the
field that errors out?
What if there were multiple indices that were in flight and only the third
one errors out, will they all be marked invalid?
If anything goes wrong with the put it should probably still throw back to
t
+1
-Dan
On Thu, Sep 7, 2017 at 9:14 PM, Nabarun Nag wrote:
> *Proposal:*
> * Index interface will include an API - isValid() which will return true if
> the index is still valid / uncorrupted, else will return false if it
> corrupted / invalid.
> * gfsh command "list index" will have one more c
24 matches
Mail list logo
