Build failed in Jenkins: Geode-nightly #960

2017-09-20 Thread Apache Jenkins Server 
See 


Changes:

[github] GEODE-2818 Document gfsh aliases (#775)

[dschneider] GEODE-3626: Fix relative path support for snapshots

[gosullivan] GEODE-3548 Add logging to new protocol code

[github] GEODE-3609 Small AcceptorImpl refactor (#772)

[gosullivan] GEODE-3083: Fix geode-protobuf stats. - Enhance and rename test of 
basic

--
[...truncated 121.06 KB...]
:geode-junit:spotlessJavaCheck
:geode-junit:spotlessCheck
:geode-junit:test
:geode-junit:check
:geode-junit:build
:geode-junit:distributedTest
:geode-junit:integrationTest
:geode-lucene:assemble
:geode-lucene:compileTestJavaNote: Some input files use or override a 
deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-lucene:processTestResources
:geode-lucene:testClasses
:geode-lucene:checkMissedTests
:geode-lucene:spotlessJavaCheck
:geode-lucene:spotlessCheck
:geode-lucene:test
:geode-lucene:check
:geode-lucene:build
:geode-lucene:distributedTest
:geode-lucene:integrationTest
:geode-old-client-support:assemble
:geode-old-client-support:compileTestJavaNote: 

 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:geode-old-client-support:processTestResources NO-SOURCE
:geode-old-client-support:testClasses
:geode-old-client-support:checkMissedTests
:geode-old-client-support:spotlessJavaCheck
:geode-old-client-support:spotlessCheck
:geode-old-client-support:test
:geode-old-client-support:check
:geode-old-client-support:build
:geode-old-client-support:distributedTest
:geode-old-client-support:integrationTest
:geode-old-versions:distributedTest NO-SOURCE
:geode-old-versions:integrationTest NO-SOURCE
:geode-protobuf:assemble
:geode-protobuf:extractIncludeTestProto
:geode-protobuf:extractTestProto UP-TO-DATE
:geode-protobuf:generateTestProto NO-SOURCE
:geode-protobuf:compileTestJavaNote: 

 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-protobuf:processTestResources
:geode-protobuf:testClasses
:geode-protobuf:checkMissedTests
:geode-protobuf:spotlessJavaCheck
:geode-protobuf:spotlessCheck
:geode-protobuf:test
:geode-protobuf:check
:geode-protobuf:build
:geode-protobuf:distributedTest

org.apache.geode.protocol.acceptance.LocatorConnectionDUnitTest > 
testInvalidOperationReturnsFailure FAILED
java.lang.AssertionError: Suspicious strings were written to the log during 
this run.
Fix the strings or use IgnoredException.addIgnoredException to ignore.
---
Found suspect string in log4j at line 724

[error 2017/09/20 11:54:10.546 UTC  tid=0x4e] 
Invalid execution context found for operation GETREGIONNAMESREQUEST

2 tests completed, 1 failed
:geode-protobuf:distributedTest FAILED
:geode-protobuf:integrationTest

org.apache.geode.protocol.acceptance.CacheConnectionTimeoutJUnitTest > 
testUnresponsiveClientsGetDisconnected FAILED
org.awaitility.core.ConditionTimeoutException: Condition defined as a 
lambda expression in 
org.apache.geode.protocol.acceptance.CacheConnectionTimeoutJUnitTest null 
within 145 milliseconds.
at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:104)
at 
org.awaitility.core.AssertionCondition.await(AssertionCondition.java:117)
at 
org.awaitility.core.AssertionCondition.await(AssertionCondition.java:32)
at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:809)
at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:648)
at 
org.apache.geode.protocol.acceptance.CacheConnectionTimeoutJUnitTest.testUnresponsiveClientsGetDisconnected(CacheConnectionTimeoutJUnitTest.java:138)

14 tests completed, 1 failed
:geode-protobuf:integrationTest FAILED
:geode-pulse:assemble
:geode-pulse:compileTestJavaNote: Some input files use or override a deprecated 
API.
Note: Recompile with -Xlint:deprecation for details.
Note: 

 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-pulse:processTestResources
:geode-pulse:testClasses
:geode-pulse:checkMissedTests
:geode-pulse:spotlessJavaCheck
:geode-pulse:spotlessCheck
:geode-pulse:test
:geode-pulse:check
:geode-pulse:b

Build failed in Jenkins: Geode-nightly-flaky #127

2017-09-20 Thread Apache Jenkins Server 
See 

--
[...truncated 116.08 KB...]
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-swagger-ui/2.6.1/springfox-swagger-ui-2.6.1.pom
Download 
https://repo1.maven.org/maven2/org/springframework/hateoas/spring-hateoas/0.23.0.RELEASE/spring-hateoas-0.23.0.RELEASE.pom
Download 
https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.10.6/scala-reflect-2.10.6.pom
Download 
https://repo1.maven.org/maven2/com/fasterxml/jackson/module/jackson-module-paranamer/2.8.6/jackson-module-paranamer-2.8.6.pom
Download 
https://repo1.maven.org/maven2/com/fasterxml/jackson/module/jackson-modules-base/2.8.6/jackson-modules-base-2.8.6.pom
Download 
https://repo1.maven.org/maven2/com/fasterxml/jackson/jackson-bom/2.8.6/jackson-bom-2.8.6.pom
Download 
https://repo1.maven.org/maven2/io/swagger/swagger-annotations/1.5.10/swagger-annotations-1.5.10.pom
Download 
https://repo1.maven.org/maven2/io/swagger/swagger-project/1.5.10/swagger-project-1.5.10.pom
Download 
https://repo1.maven.org/maven2/io/swagger/swagger-models/1.5.10/swagger-models-1.5.10.pom
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-spi/2.6.1/springfox-spi-2.6.1.pom
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-schema/2.6.1/springfox-schema-2.6.1.pom
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-swagger-common/2.6.1/springfox-swagger-common-2.6.1.pom
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-spring-web/2.6.1/springfox-spring-web-2.6.1.pom
Download 
https://repo1.maven.org/maven2/com/fasterxml/classmate/1.3.1/classmate-1.3.1.pom
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin-core/1.2.0.RELEASE/spring-plugin-core-1.2.0.RELEASE.pom
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin/1.2.0.RELEASE/spring-plugin-1.2.0.RELEASE.pom
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin-metadata/1.2.0.RELEASE/spring-plugin-metadata-1.2.0.RELEASE.pom
Download 
https://repo1.maven.org/maven2/org/mapstruct/mapstruct/1.0.0.Final/mapstruct-1.0.0.Final.pom
Download 
https://repo1.maven.org/maven2/org/mapstruct/mapstruct-parent/1.0.0.Final/mapstruct-parent-1.0.0.Final.pom
Download 
https://repo1.maven.org/maven2/org/springframework/spring-expression/4.3.5.RELEASE/spring-expression-4.3.5.RELEASE.pom
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-core/2.6.1/springfox-core-2.6.1.pom
Download 
https://repo1.maven.org/maven2/com/fasterxml/jackson/module/jackson-module-scala_2.10/2.8.6/jackson-module-scala_2.10-2.8.6.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-swagger2/2.6.1/springfox-swagger2-2.6.1.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-swagger-ui/2.6.1/springfox-swagger-ui-2.6.1.jar
Download 
https://repo1.maven.org/maven2/org/springframework/hateoas/spring-hateoas/0.23.0.RELEASE/spring-hateoas-0.23.0.RELEASE.jar
Download 
https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.10.6/scala-reflect-2.10.6.jar
Download 
https://repo1.maven.org/maven2/com/fasterxml/jackson/module/jackson-module-paranamer/2.8.6/jackson-module-paranamer-2.8.6.jar
Download 
https://repo1.maven.org/maven2/io/swagger/swagger-annotations/1.5.10/swagger-annotations-1.5.10.jar
Download 
https://repo1.maven.org/maven2/io/swagger/swagger-models/1.5.10/swagger-models-1.5.10.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-spi/2.6.1/springfox-spi-2.6.1.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-schema/2.6.1/springfox-schema-2.6.1.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-swagger-common/2.6.1/springfox-swagger-common-2.6.1.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-spring-web/2.6.1/springfox-spring-web-2.6.1.jar
Download 
https://repo1.maven.org/maven2/com/fasterxml/classmate/1.3.1/classmate-1.3.1.jar
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin-core/1.2.0.RELEASE/spring-plugin-core-1.2.0.RELEASE.jar
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin-metadata/1.2.0.RELEASE/spring-plugin-metadata-1.2.0.RELEASE.jar
Download 
https://repo1.maven.org/maven2/org/mapstruct/mapstruct/1.0.0.Final/mapstruct-1.0.0.Final.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-core/2.6.1/springfox-core-2.6.1.jar
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:geode-web-api:processResources
:geode-web-api:classes
:geode-assembly:docs:101:
 warning - Tag @link: reference not found: BasicTypes.Entry

1 warning
:geode-assembly:gfshDepsJar
:geode-common:javadocJar
:geode-common:sources

Re: [DISCUSS] Addition of isValid API to Index interface

2017-09-20 Thread Nabarun Nag 
The PR #768 has been created for this issue and also GEODE-3520 has been
changed to reflect this requirement.

Regards
Nabarun

On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag  wrote:

> Thanks you guys for the review. I will revert the GEODE-3520 ticket to
> reflect that invalidate should happen for both synchronous and asynchronous
> index maintenance.
> Also, I will resend the PR which reflects the changes mentioned in the
> ticket
>
> Regards
> Nabarun Nag
>
>
>
>
>
> On Thu, Sep 14, 2017 at 5:55 PM Anilkumar Gingade 
> wrote:
>
>> Dan, you are right...Thanks to Jason, myself and Jason looked into the
>> code
>> to see if index is updated before the event/change is saved/injected into
>> the region...It looks like the index update are happening after the region
>> change/update is saved. Moving the index update before that is not an easy
>> task...
>>
>> For time, when there is any problem with index update, we can proceed with
>> invalidating the indexes...But we really need to look at making region and
>> index updates in a transactional way, silently invalidating indexes may
>> not
>> be acceptable...
>>
>> -Anil.
>>
>>
>>
>>
>> On Thu, Sep 14, 2017 at 1:12 PM, Dan Smith  wrote:
>>
>> > I'm still going to push that we stick with Naba's original proposal.
>> >
>> > The current behavior is clearly broken. If one index update fails, an
>> > exception gets thrown to the user (nice!) but it leaves the put in a
>> > partially completed state - some other indexes may not have been
>> updated,
>> > WAN/AEQs may not have been notified, etc.
>> >
>> > We should never leave the system in this corrupted state. It would be
>> nice
>> > to be able to cleanly rollback the put, but we don't have that
>> capability
>> > especially considering that cache writers have already been invoked. So
>> the
>> > next best thing is to invalidate the index that failed to update.
>> >
>> > Logging an error an allowing the put to succeed does match what we do
>> with
>> > CacheListeners. Exceptions from CacheListeners do not fail the put.
>> >
>> > -Dan
>> >
>> > On Mon, Sep 11, 2017 at 3:29 PM, Jason Huynh  wrote:
>> >
>> > > Anil, we actually do have a case where the index is out of sync with
>> the
>> > > region currently.  It's just not likely to happen but if there is an
>> > > exception from an index, the end result is that certain indexes get
>> > updated
>> > > and the region has already been updated.
>> > > However the exception is thrown back to the putter, so it becomes very
>> > > obvious something is wrong but I believe Naba has updated the ticket
>> to
>> > > show a test that reproduces the problem...
>> > >
>> > >
>> > > On Mon, Sep 11, 2017 at 2:50 PM Anilkumar Gingade <
>> aging...@pivotal.io>
>> > > wrote:
>> > >
>> > > > The other way to look at it is; what happens to a cache op; when
>> there
>> > is
>> > > > an exception after Region.Entry is created? can it happen? In that
>> > case,
>> > > do
>> > > > we stick the entry into the Cache or not? If an exception is
>> handled,
>> > how
>> > > > is it done, can we look at using the same for Index...
>> > > >
>> > > > Also previously, once the valid index is created (verified during
>> > create
>> > > or
>> > > > first put into the cache); we never had any issue where index is
>> out of
>> > > > sync with cache...If that changes with new futures (security?) then
>> we
>> > > may
>> > > > have to change the expectation with indexing...
>> > > >
>> > > > -Anil.
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Sep 11, 2017 at 2:16 PM, Anthony Baker 
>> > > wrote:
>> > > >
>> > > > > I’m confused.  Once a cache update has been distributed to other
>> > > members
>> > > > > it can’t be undone.  That update could have triggered myriad other
>> > > > > application behaviors.
>> > > > >
>> > > > > Anthony
>> > > > >
>> > > > > > On Sep 11, 2017, at 2:04 PM, Michael Stolz 
>> > > wrote:
>> > > > > >
>> > > > > > Great, that's exactly the behavior I would expect.
>> > > > > >
>> > > > > > Thanks.
>> > > > > >
>> > > > > > --
>> > > > > > Mike Stolz
>> > > > > > Principal Engineer, GemFire Product Manager
>> > > > > > Mobile: +1-631-835-4771 <(631)%20835-4771> <(631)%20835-4771>
>> > > > > >
>> > > > > > On Mon, Sep 11, 2017 at 4:34 PM, Jason Huynh > >
>> > > > wrote:
>> > > > > >
>> > > > > >> Hi Mike, I think the concern was less about the security
>> portion
>> > but
>> > > > > rather
>> > > > > >> if any exception occurs during index update, right now, the
>> region
>> > > > gets
>> > > > > >> updated and the rest of the system (index/wan/callbacks) may or
>> > may
>> > > > not
>> > > > > be
>> > > > > >> updated.  I think Naba just tried to provide an example where
>> this
>> > > > might
>> > > > > >> occur, but that specific scenario is invalid.
>> > > > > >>
>> > > > > >> I believe Nabarun has opened a ticket for rolling back the put
>> > > > operation
>> > > > > >> when an index exception occurs. GEODE-3589.  It can probably be
>> > > > > modified to
>> > > > > >> state any

[RESULT][VOTE] Apache Geode release - v1.2.1 RC4

2017-09-20 Thread Anthony Baker 
The vote passes with 4 +1 votes (3 binding).

+1 Dan Smith (binding)
+1 William Markito Oliveira (binding)
+1 Karen Miller (binding)
+1 Dick Cavender

Vote thread:
http://mail-archives.apache.org/mod_mbox/geode-dev/201709.mbox/%3ccaewge-ft0erojncomtzvrzl8fnxreuhdryunblwt5cft8xq...@mail.gmail.com%3e

Anthony

Passed: apache/geode#3918 (rel/v1.2.1 - 0b881b5)

2017-09-20 Thread Travis CI 
Build Update for apache/geode
-

Build: #3918
Status: Passed

Duration: 9 minutes and 14 seconds
Commit: 0b881b5 (rel/v1.2.1)
Author: Bruce Schuchardt
Message: GEODE-3249 Validate internal client/server messages

This change leaves the security hole in place but allows you to plug
it by setting the system property

geode.disallow-internal-messages-without-credentials=true

Clients must be upgraded to the release containing this change if you
set this system property to true and client/server authentication is
enabled.  Otherwise client messages to register PDX types or
Instantiators will be rejected by the servers.

New tests have been added to perform backward-compatibility testing
with the old security implementation and the internal message command
classes have been modified to perform validation of credentials if
the system property is set to true.

(cherry picked from commit abbb359fe59ea3e74462fe48890918108a0edda3)

View the changeset: https://github.com/apache/geode/compare/rel/v1.2.1

View the full build log and details: 
https://travis-ci.org/apache/geode/builds/277875254?utm_source=email&utm_medium=notification

--

You can configure recipients for build notifications in your .travis.yml file. 
See https://docs.travis-ci.com/user/notifications

Errored: apache/geode-examples#65 (rel/v1.2.1 - 5d034de)

2017-09-20 Thread Travis CI 
Build Update for apache/geode-examples
-

Build: #65
Status: Errored

Duration: 1 minute and 0 seconds
Commit: 5d034de (rel/v1.2.1)
Author: Dan Smith
Message: GEODE-3199: Make signing with a gpg key optional

Make it optional to sign the archives with a gpg key, to avoid annoying
users trying to build the examples. You must specify the property
-PsignArchives to sign with a gpg key.

(cherry picked from commit 76cce858773dd390c27de717a5d7c92a9d7aaf9b)

View the changeset: https://github.com/apache/geode-examples/compare/rel/v1.2.1

View the full build log and details: 
https://travis-ci.org/apache/geode-examples/builds/277875101?utm_source=email&utm_medium=notification

--

You can configure recipients for build notifications in your .travis.yml file. 
See https://docs.travis-ci.com/user/notifications

Re: [DISCUSS] Addition of isValid API to Index interface

2017-09-20 Thread Swapnil Bawaskar 
Sorry for not reading this thread earlier, but I was wondering what is the
point of just invalidating the index and having it lie around if it is not
going to be used?
Can we just drop the index instead, and log a warning message to that
effect? This will free up the memory used by the index and will proactively
let the admin/user know what happened.

On Wed, Sep 20, 2017 at 9:59 AM Nabarun Nag  wrote:

> The PR #768 has been created for this issue and also GEODE-3520 has been
> changed to reflect this requirement.
>
> Regards
> Nabarun
>
> On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag  wrote:
>
> > Thanks you guys for the review. I will revert the GEODE-3520 ticket to
> > reflect that invalidate should happen for both synchronous and
> asynchronous
> > index maintenance.
> > Also, I will resend the PR which reflects the changes mentioned in the
> > ticket
> >
> > Regards
> > Nabarun Nag
> >
> >
> >
> >
> >
> > On Thu, Sep 14, 2017 at 5:55 PM Anilkumar Gingade 
> > wrote:
> >
> >> Dan, you are right...Thanks to Jason, myself and Jason looked into the
> >> code
> >> to see if index is updated before the event/change is saved/injected
> into
> >> the region...It looks like the index update are happening after the
> region
> >> change/update is saved. Moving the index update before that is not an
> easy
> >> task...
> >>
> >> For time, when there is any problem with index update, we can proceed
> with
> >> invalidating the indexes...But we really need to look at making region
> and
> >> index updates in a transactional way, silently invalidating indexes may
> >> not
> >> be acceptable...
> >>
> >> -Anil.
> >>
> >>
> >>
> >>
> >> On Thu, Sep 14, 2017 at 1:12 PM, Dan Smith  wrote:
> >>
> >> > I'm still going to push that we stick with Naba's original proposal.
> >> >
> >> > The current behavior is clearly broken. If one index update fails, an
> >> > exception gets thrown to the user (nice!) but it leaves the put in a
> >> > partially completed state - some other indexes may not have been
> >> updated,
> >> > WAN/AEQs may not have been notified, etc.
> >> >
> >> > We should never leave the system in this corrupted state. It would be
> >> nice
> >> > to be able to cleanly rollback the put, but we don't have that
> >> capability
> >> > especially considering that cache writers have already been invoked.
> So
> >> the
> >> > next best thing is to invalidate the index that failed to update.
> >> >
> >> > Logging an error an allowing the put to succeed does match what we do
> >> with
> >> > CacheListeners. Exceptions from CacheListeners do not fail the put.
> >> >
> >> > -Dan
> >> >
> >> > On Mon, Sep 11, 2017 at 3:29 PM, Jason Huynh 
> wrote:
> >> >
> >> > > Anil, we actually do have a case where the index is out of sync with
> >> the
> >> > > region currently.  It's just not likely to happen but if there is an
> >> > > exception from an index, the end result is that certain indexes get
> >> > updated
> >> > > and the region has already been updated.
> >> > > However the exception is thrown back to the putter, so it becomes
> very
> >> > > obvious something is wrong but I believe Naba has updated the ticket
> >> to
> >> > > show a test that reproduces the problem...
> >> > >
> >> > >
> >> > > On Mon, Sep 11, 2017 at 2:50 PM Anilkumar Gingade <
> >> aging...@pivotal.io>
> >> > > wrote:
> >> > >
> >> > > > The other way to look at it is; what happens to a cache op; when
> >> there
> >> > is
> >> > > > an exception after Region.Entry is created? can it happen? In that
> >> > case,
> >> > > do
> >> > > > we stick the entry into the Cache or not? If an exception is
> >> handled,
> >> > how
> >> > > > is it done, can we look at using the same for Index...
> >> > > >
> >> > > > Also previously, once the valid index is created (verified during
> >> > create
> >> > > or
> >> > > > first put into the cache); we never had any issue where index is
> >> out of
> >> > > > sync with cache...If that changes with new futures (security?)
> then
> >> we
> >> > > may
> >> > > > have to change the expectation with indexing...
> >> > > >
> >> > > > -Anil.
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Sep 11, 2017 at 2:16 PM, Anthony Baker  >
> >> > > wrote:
> >> > > >
> >> > > > > I’m confused.  Once a cache update has been distributed to other
> >> > > members
> >> > > > > it can’t be undone.  That update could have triggered myriad
> other
> >> > > > > application behaviors.
> >> > > > >
> >> > > > > Anthony
> >> > > > >
> >> > > > > > On Sep 11, 2017, at 2:04 PM, Michael Stolz  >
> >> > > wrote:
> >> > > > > >
> >> > > > > > Great, that's exactly the behavior I would expect.
> >> > > > > >
> >> > > > > > Thanks.
> >> > > > > >
> >> > > > > > --
> >> > > > > > Mike Stolz
> >> > > > > > Principal Engineer, GemFire Product Manager
> >> > > > > > Mobile: +1-631-835-4771 <(631)%20835-4771>
> <(631)%20835-4771> <(631)%20835-4771>
> >> > > > > >
> >> > > > > > On Mon, Sep 11, 2017 at 4:34 PM, Jason Huynh <
> jhu...@pivotal.io
> >> >
> >>

Re: [DISCUSS] Addition of isValid API to Index interface

2017-09-20 Thread Anilkumar Gingade 
Agree, invalid index should not be kept...For PRs its tricky, each
bucket-region has its own index, if an index is invalidated on a bucket
region, do we need to remove that index from all the bucket regions (local
and remote)? I guess, yes...

-Anil.


On Wed, Sep 20, 2017 at 2:58 PM, Swapnil Bawaskar 
wrote:

> Sorry for not reading this thread earlier, but I was wondering what is the
> point of just invalidating the index and having it lie around if it is not
> going to be used?
> Can we just drop the index instead, and log a warning message to that
> effect? This will free up the memory used by the index and will proactively
> let the admin/user know what happened.
>
> On Wed, Sep 20, 2017 at 9:59 AM Nabarun Nag  wrote:
>
> > The PR #768 has been created for this issue and also GEODE-3520 has been
> > changed to reflect this requirement.
> >
> > Regards
> > Nabarun
> >
> > On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag  wrote:
> >
> > > Thanks you guys for the review. I will revert the GEODE-3520 ticket to
> > > reflect that invalidate should happen for both synchronous and
> > asynchronous
> > > index maintenance.
> > > Also, I will resend the PR which reflects the changes mentioned in the
> > > ticket
> > >
> > > Regards
> > > Nabarun Nag
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Sep 14, 2017 at 5:55 PM Anilkumar Gingade  >
> > > wrote:
> > >
> > >> Dan, you are right...Thanks to Jason, myself and Jason looked into the
> > >> code
> > >> to see if index is updated before the event/change is saved/injected
> > into
> > >> the region...It looks like the index update are happening after the
> > region
> > >> change/update is saved. Moving the index update before that is not an
> > easy
> > >> task...
> > >>
> > >> For time, when there is any problem with index update, we can proceed
> > with
> > >> invalidating the indexes...But we really need to look at making region
> > and
> > >> index updates in a transactional way, silently invalidating indexes
> may
> > >> not
> > >> be acceptable...
> > >>
> > >> -Anil.
> > >>
> > >>
> > >>
> > >>
> > >> On Thu, Sep 14, 2017 at 1:12 PM, Dan Smith  wrote:
> > >>
> > >> > I'm still going to push that we stick with Naba's original proposal.
> > >> >
> > >> > The current behavior is clearly broken. If one index update fails,
> an
> > >> > exception gets thrown to the user (nice!) but it leaves the put in a
> > >> > partially completed state - some other indexes may not have been
> > >> updated,
> > >> > WAN/AEQs may not have been notified, etc.
> > >> >
> > >> > We should never leave the system in this corrupted state. It would
> be
> > >> nice
> > >> > to be able to cleanly rollback the put, but we don't have that
> > >> capability
> > >> > especially considering that cache writers have already been invoked.
> > So
> > >> the
> > >> > next best thing is to invalidate the index that failed to update.
> > >> >
> > >> > Logging an error an allowing the put to succeed does match what we
> do
> > >> with
> > >> > CacheListeners. Exceptions from CacheListeners do not fail the put.
> > >> >
> > >> > -Dan
> > >> >
> > >> > On Mon, Sep 11, 2017 at 3:29 PM, Jason Huynh 
> > wrote:
> > >> >
> > >> > > Anil, we actually do have a case where the index is out of sync
> with
> > >> the
> > >> > > region currently.  It's just not likely to happen but if there is
> an
> > >> > > exception from an index, the end result is that certain indexes
> get
> > >> > updated
> > >> > > and the region has already been updated.
> > >> > > However the exception is thrown back to the putter, so it becomes
> > very
> > >> > > obvious something is wrong but I believe Naba has updated the
> ticket
> > >> to
> > >> > > show a test that reproduces the problem...
> > >> > >
> > >> > >
> > >> > > On Mon, Sep 11, 2017 at 2:50 PM Anilkumar Gingade <
> > >> aging...@pivotal.io>
> > >> > > wrote:
> > >> > >
> > >> > > > The other way to look at it is; what happens to a cache op; when
> > >> there
> > >> > is
> > >> > > > an exception after Region.Entry is created? can it happen? In
> that
> > >> > case,
> > >> > > do
> > >> > > > we stick the entry into the Cache or not? If an exception is
> > >> handled,
> > >> > how
> > >> > > > is it done, can we look at using the same for Index...
> > >> > > >
> > >> > > > Also previously, once the valid index is created (verified
> during
> > >> > create
> > >> > > or
> > >> > > > first put into the cache); we never had any issue where index is
> > >> out of
> > >> > > > sync with cache...If that changes with new futures (security?)
> > then
> > >> we
> > >> > > may
> > >> > > > have to change the expectation with indexing...
> > >> > > >
> > >> > > > -Anil.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Mon, Sep 11, 2017 at 2:16 PM, Anthony Baker <
> aba...@pivotal.io
> > >
> > >> > > wrote:
> > >> > > >
> > >> > > > > I’m confused.  Once a cache update has been distributed to
> other
> > >> > > members
> > >> > > > > it can’t be undone.  That update could have t

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #684 was SUCCESSFUL (with 2042 tests)

2017-09-20 Thread Spring CI 

---
Spring Data GemFire > Nightly-ApacheGeode > #684 was successful.
---
Scheduled
2044 tests in total.

https://build.spring.io/browse/SGF-NAG-684/





--
This message is automatically generated by Atlassian Bamboo

Re: [DISCUSS] Addition of isValid API to Index interface

2017-09-20 Thread Nabarun Nag 
Hi Swapnil,
There were few factors we considered before going with just invalidating
the index rather than destroying the index.
1. Debugging reasons : If the indexes were destroyed and logs roll over,
and suddenly we see that indexes have disappeared, it will be tough to
differentiate between whether the indexes were created improperly in the
first place [/ restarts] or if a put corrupted it hence it was destroyed.
In case of marking it invalid, we know for sure that a put has corrupted
the index and prevent confusion for the user.

2. Performance perspective  : We were worried that if a put corrupts
multiple indexes and then that put is also responsible for destroying those
indexes may slow down the execution as it will have to acquire/release
locks to destroy indexes [especially in case of PR indexes]. We were also
worried about race conditions arising in that case.

Regards
Nabarun

On Wed, Sep 20, 2017 at 2:59 PM Swapnil Bawaskar 
wrote:

> Sorry for not reading this thread earlier, but I was wondering what is the
> point of just invalidating the index and having it lie around if it is not
> going to be used?
> Can we just drop the index instead, and log a warning message to that
> effect? This will free up the memory used by the index and will proactively
> let the admin/user know what happened.
>
> On Wed, Sep 20, 2017 at 9:59 AM Nabarun Nag  wrote:
>
> > The PR #768 has been created for this issue and also GEODE-3520 has been
> > changed to reflect this requirement.
> >
> > Regards
> > Nabarun
> >
> > On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag  wrote:
> >
> > > Thanks you guys for the review. I will revert the GEODE-3520 ticket to
> > > reflect that invalidate should happen for both synchronous and
> > asynchronous
> > > index maintenance.
> > > Also, I will resend the PR which reflects the changes mentioned in the
> > > ticket
> > >
> > > Regards
> > > Nabarun Nag
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Sep 14, 2017 at 5:55 PM Anilkumar Gingade  >
> > > wrote:
> > >
> > >> Dan, you are right...Thanks to Jason, myself and Jason looked into the
> > >> code
> > >> to see if index is updated before the event/change is saved/injected
> > into
> > >> the region...It looks like the index update are happening after the
> > region
> > >> change/update is saved. Moving the index update before that is not an
> > easy
> > >> task...
> > >>
> > >> For time, when there is any problem with index update, we can proceed
> > with
> > >> invalidating the indexes...But we really need to look at making region
> > and
> > >> index updates in a transactional way, silently invalidating indexes
> may
> > >> not
> > >> be acceptable...
> > >>
> > >> -Anil.
> > >>
> > >>
> > >>
> > >>
> > >> On Thu, Sep 14, 2017 at 1:12 PM, Dan Smith  wrote:
> > >>
> > >> > I'm still going to push that we stick with Naba's original proposal.
> > >> >
> > >> > The current behavior is clearly broken. If one index update fails,
> an
> > >> > exception gets thrown to the user (nice!) but it leaves the put in a
> > >> > partially completed state - some other indexes may not have been
> > >> updated,
> > >> > WAN/AEQs may not have been notified, etc.
> > >> >
> > >> > We should never leave the system in this corrupted state. It would
> be
> > >> nice
> > >> > to be able to cleanly rollback the put, but we don't have that
> > >> capability
> > >> > especially considering that cache writers have already been invoked.
> > So
> > >> the
> > >> > next best thing is to invalidate the index that failed to update.
> > >> >
> > >> > Logging an error an allowing the put to succeed does match what we
> do
> > >> with
> > >> > CacheListeners. Exceptions from CacheListeners do not fail the put.
> > >> >
> > >> > -Dan
> > >> >
> > >> > On Mon, Sep 11, 2017 at 3:29 PM, Jason Huynh 
> > wrote:
> > >> >
> > >> > > Anil, we actually do have a case where the index is out of sync
> with
> > >> the
> > >> > > region currently.  It's just not likely to happen but if there is
> an
> > >> > > exception from an index, the end result is that certain indexes
> get
> > >> > updated
> > >> > > and the region has already been updated.
> > >> > > However the exception is thrown back to the putter, so it becomes
> > very
> > >> > > obvious something is wrong but I believe Naba has updated the
> ticket
> > >> to
> > >> > > show a test that reproduces the problem...
> > >> > >
> > >> > >
> > >> > > On Mon, Sep 11, 2017 at 2:50 PM Anilkumar Gingade <
> > >> aging...@pivotal.io>
> > >> > > wrote:
> > >> > >
> > >> > > > The other way to look at it is; what happens to a cache op; when
> > >> there
> > >> > is
> > >> > > > an exception after Region.Entry is created? can it happen? In
> that
> > >> > case,
> > >> > > do
> > >> > > > we stick the entry into the Cache or not? If an exception is
> > >> handled,
> > >> > how
> > >> > > > is it done, can we look at using the same for Index...
> > >> > > >
> > >> > > > Also previously, once the valid index is created (verified
> during

Re: [DISCUSS] Addition of isValid API to Index interface

2017-09-20 Thread Swapnil Bawaskar 
Thanks for the explanation Naba, please find my reply below:
1. Debugging: If we log a warning, that should get noticed immediately, so
I don't think we need to worry about logs rolling.
2. Performance for a single put: We can always schedule an async task to
drop the index.

On Wed, Sep 20, 2017 at 3:49 PM Nabarun Nag  wrote:

> Hi Swapnil,
> There were few factors we considered before going with just invalidating
> the index rather than destroying the index.
> 1. Debugging reasons : If the indexes were destroyed and logs roll over,
> and suddenly we see that indexes have disappeared, it will be tough to
> differentiate between whether the indexes were created improperly in the
> first place [/ restarts] or if a put corrupted it hence it was destroyed.
> In case of marking it invalid, we know for sure that a put has corrupted
> the index and prevent confusion for the user.
>
> 2. Performance perspective  : We were worried that if a put corrupts
> multiple indexes and then that put is also responsible for destroying those
> indexes may slow down the execution as it will have to acquire/release
> locks to destroy indexes [especially in case of PR indexes]. We were also
> worried about race conditions arising in that case.
>
> Regards
> Nabarun
>
> On Wed, Sep 20, 2017 at 2:59 PM Swapnil Bawaskar 
> wrote:
>
> > Sorry for not reading this thread earlier, but I was wondering what is
> the
> > point of just invalidating the index and having it lie around if it is
> not
> > going to be used?
> > Can we just drop the index instead, and log a warning message to that
> > effect? This will free up the memory used by the index and will
> proactively
> > let the admin/user know what happened.
> >
> > On Wed, Sep 20, 2017 at 9:59 AM Nabarun Nag  wrote:
> >
> > > The PR #768 has been created for this issue and also GEODE-3520 has
> been
> > > changed to reflect this requirement.
> > >
> > > Regards
> > > Nabarun
> > >
> > > On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag  wrote:
> > >
> > > > Thanks you guys for the review. I will revert the GEODE-3520 ticket
> to
> > > > reflect that invalidate should happen for both synchronous and
> > > asynchronous
> > > > index maintenance.
> > > > Also, I will resend the PR which reflects the changes mentioned in
> the
> > > > ticket
> > > >
> > > > Regards
> > > > Nabarun Nag
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Sep 14, 2017 at 5:55 PM Anilkumar Gingade <
> aging...@pivotal.io
> > >
> > > > wrote:
> > > >
> > > >> Dan, you are right...Thanks to Jason, myself and Jason looked into
> the
> > > >> code
> > > >> to see if index is updated before the event/change is saved/injected
> > > into
> > > >> the region...It looks like the index update are happening after the
> > > region
> > > >> change/update is saved. Moving the index update before that is not
> an
> > > easy
> > > >> task...
> > > >>
> > > >> For time, when there is any problem with index update, we can
> proceed
> > > with
> > > >> invalidating the indexes...But we really need to look at making
> region
> > > and
> > > >> index updates in a transactional way, silently invalidating indexes
> > may
> > > >> not
> > > >> be acceptable...
> > > >>
> > > >> -Anil.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Thu, Sep 14, 2017 at 1:12 PM, Dan Smith 
> wrote:
> > > >>
> > > >> > I'm still going to push that we stick with Naba's original
> proposal.
> > > >> >
> > > >> > The current behavior is clearly broken. If one index update fails,
> > an
> > > >> > exception gets thrown to the user (nice!) but it leaves the put
> in a
> > > >> > partially completed state - some other indexes may not have been
> > > >> updated,
> > > >> > WAN/AEQs may not have been notified, etc.
> > > >> >
> > > >> > We should never leave the system in this corrupted state. It would
> > be
> > > >> nice
> > > >> > to be able to cleanly rollback the put, but we don't have that
> > > >> capability
> > > >> > especially considering that cache writers have already been
> invoked.
> > > So
> > > >> the
> > > >> > next best thing is to invalidate the index that failed to update.
> > > >> >
> > > >> > Logging an error an allowing the put to succeed does match what we
> > do
> > > >> with
> > > >> > CacheListeners. Exceptions from CacheListeners do not fail the
> put.
> > > >> >
> > > >> > -Dan
> > > >> >
> > > >> > On Mon, Sep 11, 2017 at 3:29 PM, Jason Huynh 
> > > wrote:
> > > >> >
> > > >> > > Anil, we actually do have a case where the index is out of sync
> > with
> > > >> the
> > > >> > > region currently.  It's just not likely to happen but if there
> is
> > an
> > > >> > > exception from an index, the end result is that certain indexes
> > get
> > > >> > updated
> > > >> > > and the region has already been updated.
> > > >> > > However the exception is thrown back to the putter, so it
> becomes
> > > very
> > > >> > > obvious something is wrong but I believe Naba has updated the
> > ticket
> > > >> to
> > > >> > > show a test that reprodu