date:20170915

Build failed in Jenkins: Geode-nightly #955

2017-09-15 Thread Apache Jenkins Server

See 


Changes:

[lgallinat] GEODE-3185: Fixes CI failures in windows in

[khowe] GEODE-3615: Added CleanupDUnitVMsRule to ConnectCommandWithSSLTest

[bschuchardt] GEODE-3083: New protocol should record statistics

[kohlmu-pivotal] GEODE-3082 Integrate GenericProtocolServerConnection with

--
[...truncated 119.48 KB...]
:geode-json:distributedTest NO-SOURCE
:geode-json:integrationTest NO-SOURCE
:geode-junit:javadoc
:geode-junit:javadocJar
:geode-junit:sourcesJar
:geode-junit:signArchives SKIPPED
:geode-junit:assemble
:geode-junit:compileTestJavaNote: 

 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: 

 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-junit:processTestResources UP-TO-DATE
:geode-junit:testClasses
:geode-junit:checkMissedTests
:geode-junit:spotlessJavaCheck
:geode-junit:spotlessCheck
:geode-junit:test
:geode-junit:check
:geode-junit:build
:geode-junit:distributedTest
:geode-junit:integrationTest
:geode-lucene:assemble
:geode-lucene:compileTestJavaNote: Some input files use or override a 
deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-lucene:processTestResources UP-TO-DATE
:geode-lucene:testClasses
:geode-lucene:checkMissedTests
:geode-lucene:spotlessJavaCheck
:geode-lucene:spotlessCheck
:geode-lucene:test
:geode-lucene:check
:geode-lucene:build
:geode-lucene:distributedTest
:geode-lucene:integrationTest
:geode-old-client-support:assemble
:geode-old-client-support:compileTestJavaNote: 

 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:geode-old-client-support:processTestResources NO-SOURCE
:geode-old-client-support:testClasses
:geode-old-client-support:checkMissedTests
:geode-old-client-support:spotlessJavaCheck
:geode-old-client-support:spotlessCheck
:geode-old-client-support:test
:geode-old-client-support:check
:geode-old-client-support:build
:geode-old-client-support:distributedTest
:geode-old-client-support:integrationTest
:geode-old-versions:distributedTest NO-SOURCE
:geode-old-versions:integrationTest NO-SOURCE
:geode-protobuf:assemble
:geode-protobuf:extractIncludeTestProto
:geode-protobuf:extractTestProto UP-TO-DATE
:geode-protobuf:generateTestProto NO-SOURCE
:geode-protobuf:compileTestJavaNote: Some input files use unchecked or unsafe 
operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-protobuf:processTestResources
:geode-protobuf:testClasses
:geode-protobuf:checkMissedTests
:geode-protobuf:spotlessJavaCheck
:geode-protobuf:spotlessCheck
:geode-protobuf:test
:geode-protobuf:check
:geode-protobuf:build
:geode-protobuf:distributedTest
:geode-protobuf:integrationTest

org.apache.geode.protocol.acceptance.CacheConnectionTimeoutJUnitTest > 
testUnresponsiveClientsGetDisconnected FAILED
org.awaitility.core.ConditionTimeoutException: Condition defined as a 
lambda expression in 
org.apache.geode.protocol.acceptance.CacheConnectionTimeoutJUnitTest null 
within 1120 milliseconds.
at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:104)
at 
org.awaitility.core.AssertionCondition.await(AssertionCondition.java:117)
at 
org.awaitility.core.AssertionCondition.await(AssertionCondition.java:32)
at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:809)
at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:648)
at 
org.apache.geode.protocol.acceptance.CacheConnectionTimeoutJUnitTest.testUnresponsiveClientsGetDisconnected(CacheConnectionTimeoutJUnitTest.java:128)

13 tests completed, 1 failed
:geode-protobuf:integrationTest FAILED
:geode-pulse:assemble
:geode-pulse:compileTestJavaNote: Some input files use or override a deprecated 
API.
Note: Recompile with -Xlint:deprecation for details.
Note: 

 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-pulse:processTestResources UP-TO-DATE
:geode-pulse:testClasses
:geode-pulse:checkMissedTests
:geode-pulse:spotlessJavaCheck
:geode-pulse:spotlessCheck
:geode-pulse:test
:geode-pulse:check
:geode-pulse:build
:geode-pulse:distributedTest
:geode-pulse:integrationTest
:geod

Build failed in Jenkins: Geode-nightly-flaky #123

2017-09-15 Thread Apache Jenkins Server

See 


Changes:

[github] GEODE-3580: patch test to avoid the current failure. (#774)

[github] GEODE-3579 Update gfsh stop locator docs (#765)

[jstewart] GEODE-3544: Fix JSON parsing error

[jstewart] Update javadoc for JarBuilder

[khowe] GEODE-3539: Add test for missing coverage of status locator command

[bschuchardt] GEODE-3562 CI Failure: JSONCodecJUnitTest.testSimpleJSONEncode 
fails on

[kohlmu-pivotal] GEODE-3079 Ensure emergencyClose() closes socket

[lgallinat] GEODE-3185: Fixes CI failures in windows in

[khowe] GEODE-3615: Added CleanupDUnitVMsRule to ConnectCommandWithSSLTest

[bschuchardt] GEODE-3083: New protocol should record statistics

[kohlmu-pivotal] GEODE-3082 Integrate GenericProtocolServerConnection with

--
[...truncated 111.05 KB...]
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-spring-web/2.6.1/springfox-spring-web-2.6.1.jar
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin-core/1.2.0.RELEASE/spring-plugin-core-1.2.0.RELEASE.jar
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin-metadata/1.2.0.RELEASE/spring-plugin-metadata-1.2.0.RELEASE.jar
Download 
https://repo1.maven.org/maven2/org/mapstruct/mapstruct/1.0.0.Final/mapstruct-1.0.0.Final.jar
Download 
https://repo1.maven.org/maven2/com/thoughtworks/paranamer/paranamer/2.8/paranamer-2.8.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-core/2.6.1/springfox-core-2.6.1.jar
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:geode-web-api:processResources
:geode-web-api:classes
:geode-assembly:docs:101:
 warning - Tag @link: reference not found: BasicTypes.Entry

1 warning
:geode-assembly:gfshDepsJar
:geode-common:javadocJar
:geode-common:sourcesJar
:geode-common:signArchives SKIPPED
:geode-core:javadocJar
:geode-core:raJar
:geode-core:jcaJar
:geode-core:sourcesJar
:geode-core:signArchives SKIPPED
:geode-core:webJar
:geode-cq:jar
:geode-cq:javadoc
:geode-cq:javadocJar
:geode-cq:sourcesJar
:geode-cq:signArchives SKIPPED
:geode-json:javadocJar
:geode-json:sourcesJar
:geode-json:signArchives SKIPPED
:geode-lucene:jar
:geode-lucene:javadoc
:geode-lucene:javadocJar
:geode-lucene:sourcesJar
:geode-lucene:signArchives SKIPPED
:geode-old-client-support:jar
:geode-old-client-support:javadoc
:geode-old-client-support:javadocJar
:geode-old-client-support:sourcesJar
:geode-old-client-support:signArchives SKIPPED
:geode-protobuf:jar
:geode-protobuf:javadoc:101:
 warning - Tag @link: reference not found: BasicTypes.Entry

1 warning
:geode-protobuf:javadocJar
:geode-protobuf:sourcesJar
:geode-protobuf:signArchives SKIPPED
:geode-pulse:javadoc
:geode-pulse:javadocJar
:geode-pulse:sourcesJar
:geode-pulse:war
:geode-pulse:signArchives SKIPPED
:geode-rebalancer:jar
:geode-rebalancer:javadoc
:geode-rebalancer:javadocJar
:geode-rebalancer:sourcesJar
:geode-rebalancer:signArchives SKIPPED
:geode-wan:jar
:geode-wan:javadoc
:geode-wan:javadocJar
:geode-wan:sourcesJar
:geode-wan:signArchives SKIPPED
:geode-web:javadoc NO-SOURCE
:geode-web:javadocJar
:geode-web:sourcesJar
:geode-web:war
:geode-web:signArchives SKIPPED
:geode-web-api:javadoc
:geode-web-api:javadocJar
:geode-web-api:sourcesJar
:geode-web-api:war
:geode-web-api:signArchives SKIPPED
:geode-assembly:installDist
:geode-pulse:jar
:geode-assembly:compileTestJava
Download 
https://repo1.maven.org/maven2/org/codehaus/cargo/cargo-core-uberjar/1.6.3/cargo-core-uberjar-1.6.3.pom
Download 
https://repo1.maven.org/maven2/org/codehaus/cargo/cargo-core/1.6.3/cargo-core-1.6.3.pom
Download 
https://repo1.maven.org/maven2/org/codehaus/cargo/codehaus-cargo/1.6.3/codehaus-cargo-1.6.3.pom
Download 
https://repo1.maven.org/maven2/org/codehaus/cargo/cargo-core-uberjar/1.6.3/cargo-core-uberjar-1.6.3.jar
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:geode-assembly:processTestResources
:geode-assembly:testClasses
:geode-assembly:flakyTest
:geode-benchmarks:compileTestJava NO-SOURCE
:geode-benchmarks:processTestResources NO-SOURCE
:geode-benchmarks:testClasses UP-TO-DATE
:geode-benchmarks:flakyTest NO-SOURCE
:geode-common:compileTestJava
:geode-common:processTestResources NO-SOURCE
:geode-common:testClasses
:geode-common:flakyTest
:geode-core:flakyTest

org.apache.geode.management.RegionManagementDUnitTest > testNavigationAPIS 
FAILED
org.apache.geode.test.dunit.RMIExce

Re: [DISCUSS] Clean build takes 10minutes to complete now

2017-09-15 Thread Alexander Murmann

I fully agree with Udo here. The main build should be for Unit tests. Our
"Unit Tests" are already exercising much more of the system than they
should. Adding unit tests that not only too much or our current code but
also old code is moving us in the wrong direction. Let's keep the tests,
but please appropriately mark them as IntegrationTest.

On Tue, Sep 12, 2017 at 9:30 AM, Udo Kohlmeyer 
wrote:

> My apologies, I might gotten the commit reason incorrect. I just know that
> downloading the older product version every time is becoming painful.
> Yes, sometimes it is faster than other times, but imo, this is not
> something that should be part of the main build path.
>
> Backwards compat or integration testing should not be running as part of
> the main build task.
>
> --Udo
>
> On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag  wrote:
>
> > As we are working on fixing this issue, some extra parameters may help
> the
> > build to get bit quicker on your machine.
> >
> > using -xjavadoc -xdoc
> > Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> > BUILD SUCCESSFUL
> > Total time: 2 mins 2.729 secs
> >
> >
> > Also, I think as Jason mentioned that the slow down is due to full
> product
> > download for session state tests. LuceneSearchWithRollingUpgradeDUnit
> > tests
> > were added  in July. Please do correct me if I am wrong.
> >
> > Regards
> > Nabarun
> >
> >
> > On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann 
> > wrote:
> >
> > > Could we make it so that these tests for now are only run as part of
> > > pre-checkin till we got this ironed out and then revisit this?
> > >
> > > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> > bschucha...@pivotal.io>
> > > wrote:
> > >
> > > > The geode-old-versions module was originally created to pull in old
> > > > version jar files into your gradle cache.  This happened only once
> and
> > > you
> > > > were good to go.  I don't think that part should be backed out as it
> > has
> > > > minimal impact and is not affecting build time.
> > > >
> > > > The recent changes for lucene testing seem to be pulling in full
> > > > installations of old versions and these are deleted as part of the
> > > "clean"
> > > > gradle task.  That's causing them to be downloaded again each time
> you
> > > do a
> > > > clean&build.  Dan put changes in place so that the files aren't
> > > downloaded
> > > > again if you build without cleaning but clearly more needs to be done
> > in
> > > > this area.
> > > >
> > > >
> > > >
> > > > On 9/11/17 11:23 AM, Jacob Barrett wrote:
> > > >
> > > >> Agreed, integration tests should not be part of the build process.
> > This
> > > >> is clearly an integration test.
> > > >>
> > > >> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
> > > >>>
> > > >>> Hi there,
> > > >>>
> > > >>> With a recent addition to the build scripts, to test lucene
> backwards
> > > >>> compatibility, a step was added to download a previous version of
> > > GEODE.
> > > >>>
> > > >>> This is causing longer build times now, which is a real
> distraction.
> > In
> > > >>> cases where one would like to work on a branch, rebase that on
> > develop
> > > and
> > > >>> merge that, this step becomes a real time hog.
> > > >>>
> > > >>> I request that we remove this default behavior from a clean build
> > until
> > > >>> we have a better solution to this issue.
> > > >>>
> > > >>> I also believe that if anyone wants to add behavior like this into
> > the
> > > >>> default build, that it at least is discussed on the dev list before
> > > >>> implementing this.
> > > >>>
> > > >>> --Udo
> > > >>>
> > > >>>
> > > >
> > >
> >
>
>
>
> --
> Kindest Regards
> -
> *Udo Kohlmeyer* | *Snr Solutions Architect* |*Pivotal*
> *Mobile:* +61 409-279-160 | ukohlme...@pivotal.io
> 
> www.pivotal.io
>

Re: [DISCUSS] Clean build takes 10minutes to complete now

2017-09-15 Thread Kirk Lund

The actual tests marked with UnitTest category are actually pretty good.
They all run in just over one minute and almost all of them use Mockito to
isolate one class. I remember seeing one newer Lucene UnitTest that touches
File System which should be recategorized as IntegrationTest.

If we could move the pulling down of previous versions of Geode out of the
main build+unit-test target, that would help a lot.

Even prior to the pulling down of previous versions for backwards compat
testing, the main build (without unit-test) was too slow and I think it's
because our project is a little too complex for what Gradle is designed to
handle.

Code generation and javadocs are two of the tasks in our main build
(without unit-test) that contributes to it taking too long.

Also, the way Gradle handles junit categories is designed and coded very
inefficiently -- if we could change their junit runner to use
FastClasspathScanner to find all tests containing the targeted junit
category annotation then that would speed up all of our testing targets
immensely. Any testing target that forks JVMs runs super slow due to the
way they handle categories (this effects IntegrationTest, DistributedTest,
FlakyTest).

On Fri, Sep 15, 2017 at 8:44 AM, Alexander Murmann 
wrote:

> I fully agree with Udo here. The main build should be for Unit tests. Our
> "Unit Tests" are already exercising much more of the system than they
> should. Adding unit tests that not only too much or our current code but
> also old code is moving us in the wrong direction. Let's keep the tests,
> but please appropriately mark them as IntegrationTest.
>
> On Tue, Sep 12, 2017 at 9:30 AM, Udo Kohlmeyer 
> wrote:
>
> > My apologies, I might gotten the commit reason incorrect. I just know
> that
> > downloading the older product version every time is becoming painful.
> > Yes, sometimes it is faster than other times, but imo, this is not
> > something that should be part of the main build path.
> >
> > Backwards compat or integration testing should not be running as part of
> > the main build task.
> >
> > --Udo
> >
> > On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag  wrote:
> >
> > > As we are working on fixing this issue, some extra parameters may help
> > the
> > > build to get bit quicker on your machine.
> > >
> > > using -xjavadoc -xdoc
> > > Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> > > BUILD SUCCESSFUL
> > > Total time: 2 mins 2.729 secs
> > >
> > >
> > > Also, I think as Jason mentioned that the slow down is due to full
> > product
> > > download for session state tests. LuceneSearchWithRollingUpgradeDUnit
> > > tests
> > > were added  in July. Please do correct me if I am wrong.
> > >
> > > Regards
> > > Nabarun
> > >
> > >
> > > On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann <
> amurm...@pivotal.io>
> > > wrote:
> > >
> > > > Could we make it so that these tests for now are only run as part of
> > > > pre-checkin till we got this ironed out and then revisit this?
> > > >
> > > > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> > > bschucha...@pivotal.io>
> > > > wrote:
> > > >
> > > > > The geode-old-versions module was originally created to pull in old
> > > > > version jar files into your gradle cache.  This happened only once
> > and
> > > > you
> > > > > were good to go.  I don't think that part should be backed out as
> it
> > > has
> > > > > minimal impact and is not affecting build time.
> > > > >
> > > > > The recent changes for lucene testing seem to be pulling in full
> > > > > installations of old versions and these are deleted as part of the
> > > > "clean"
> > > > > gradle task.  That's causing them to be downloaded again each time
> > you
> > > > do a
> > > > > clean&build.  Dan put changes in place so that the files aren't
> > > > downloaded
> > > > > again if you build without cleaning but clearly more needs to be
> done
> > > in
> > > > > this area.
> > > > >
> > > > >
> > > > >
> > > > > On 9/11/17 11:23 AM, Jacob Barrett wrote:
> > > > >
> > > > >> Agreed, integration tests should not be part of the build process.
> > > This
> > > > >> is clearly an integration test.
> > > > >>
> > > > >> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer 
> wrote:
> > > > >>>
> > > > >>> Hi there,
> > > > >>>
> > > > >>> With a recent addition to the build scripts, to test lucene
> > backwards
> > > > >>> compatibility, a step was added to download a previous version of
> > > > GEODE.
> > > > >>>
> > > > >>> This is causing longer build times now, which is a real
> > distraction.
> > > In
> > > > >>> cases where one would like to work on a branch, rebase that on
> > > develop
> > > > and
> > > > >>> merge that, this step becomes a real time hog.
> > > > >>>
> > > > >>> I request that we remove this default behavior from a clean build
> > > until
> > > > >>> we have a better solution to this issue.
> > > > >>>
> > > > >>> I also believe that if anyone wants to add behavior like this
> into
> > > the
> > > > >>> default build, th

[VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Mark Hanson

So we have two approaches as their has been a veto on w_string, so it will not 
be used.

Approach 1) std::string str = object.to_string();
Approach 2) std::string str = std::to_string(object);

Please vote of your preferred.

Thanks,
Mark


> On Sep 14, 2017, at 12:29 PM, Jacob Barrett  wrote:
> 
> Don't use std::wstring EVER. The std::wstring has a platform dependent
> storage class. This means on Windows it is 32bits and Linux it is 16bits.
> None of the std::string types, and there are more than std::wstring, define
> an encoding, so would the wstring be UTF-16, UC-2, UTF-32, UC-4, or some
> other local non-unicode codepage. It gets nasty fast!
> 
> If you only support std::string and require the all std::string's be
> encoded with UTF-8 then things get real clear really fast. Yes there is
> some overhead in converting to UC-4 for calling windows system calls that
> want Unicode 32-bit std::wstring but we already had that overhead since
> almost all strings in NC were actually 8-bit UTF-8 std::string. And since
> almost all those are ASCII strings and UTF-8 can encode ASCII without
> overhead we are good to go!
> 
> So to simply we would change Serializable::toString to only a std::string.
> 
> class Serializable {
> ...
> virtual const std::string& toString() const noexcept;
> }
> 
> So the example use would be:
> 
> auto myObject = region.get("myObject"); // auto = std::shared_ptr
> auto myString = myObject->toString(); // auto = std::string
> std::cout << myString;
> 
> Even nicer may be to implement a default << operator for Serializable the
> outputs the toString so we can do:
> 
> std::cout << region.get("myObject");
> 
> As for exporting a to_string into the std namespace, I think that is one
> that is actually forbidden unlike std::hash/equal_to. I feel like I found
> that when trying to output std::chrono::duration objects to stream. Can you
> double check that it is allowed?
> 
> -Jake
> 
> 
> 
> On Thu, Sep 14, 2017 at 11:30 AM David Kimura  wrote:
> 
>> Seems like a good idea.
>> 
>> Here's a quick thought - I think c++11 is not really an obj.toString() kind
>> of language (like Java or C#).  Would it make sense to add std::to_string()
>> and std::to_wstring() equivalents for CacheableString?  This might mean
>> implementing following functions:
>> 
>>std::string to_string(CacheableString value);
>>std::wstring to_wstring(CacheableString value);
>> 
>> So that we can do something like:
>> 
>>std::cout << std::to_wstring(pdxser);
>> 
>> Thanks,
>> David
>> 
>> On Thu, Sep 14, 2017 at 11:10 AM, Michael William Dodge >> 
>> wrote:
>> 
>>> +1 for std::string and std::wstring.
>>> 
>>> Sarge
>>> 
 On 14 Sep, 2017, at 11:10, Mark Hanson  wrote:
 
 Hi All,
 
 I wanted to broach the subject of moving away from moving away from
>>> CacheableStringPtrs for the toString representation of Serializable. It
>>> would seem desirable to move to std::string and std::wstring to use more
>>> basic types that would be faster to log and the code would be simpler
>> for a
>>> user.
 
 Are there any opinions on this subject?
 
 Here is a before and after look at a chunk of code
 
 Before
 
 CacheableStringPtr ptr = pdxser->toString();
 if (ptr->isWideString()) {
 printf(" query idx %d pulled object %S  :: \n", i,
ptr->asWChar());
 } else {
 printf(" query idx %d pulled object %s  :: \n", i,
ptr->asChar());
 }
 
 After
 
 
 if (pdxser->isWideString()) {
  std::cout << " query idx “ << i << "pulled object ” <<
>>> pdxser->toWString() << std::endl;
 } else {
  std::cout << " query idx “ << i << "pulled object ” <<
>>> pdxser->toString() << std::endl;
 }
 
 
 Thanks,
 Mark
>>> 
>>> 
>>

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Ernest Burghardt

+1 Approach 1) std::string str = object.to_string();

On Fri, Sep 15, 2017 at 11:18 AM, Mark Hanson  wrote:

> So we have two approaches as their has been a veto on w_string, so it will
> not be used.
>
> Approach 1) std::string str = object.to_string();
> Approach 2) std::string str = std::to_string(object);
>
> Please vote of your preferred.
>
> Thanks,
> Mark
>
>
> > On Sep 14, 2017, at 12:29 PM, Jacob Barrett  wrote:
> >
> > Don't use std::wstring EVER. The std::wstring has a platform dependent
> > storage class. This means on Windows it is 32bits and Linux it is 16bits.
> > None of the std::string types, and there are more than std::wstring,
> define
> > an encoding, so would the wstring be UTF-16, UC-2, UTF-32, UC-4, or some
> > other local non-unicode codepage. It gets nasty fast!
> >
> > If you only support std::string and require the all std::string's be
> > encoded with UTF-8 then things get real clear really fast. Yes there is
> > some overhead in converting to UC-4 for calling windows system calls that
> > want Unicode 32-bit std::wstring but we already had that overhead since
> > almost all strings in NC were actually 8-bit UTF-8 std::string. And since
> > almost all those are ASCII strings and UTF-8 can encode ASCII without
> > overhead we are good to go!
> >
> > So to simply we would change Serializable::toString to only a
> std::string.
> >
> > class Serializable {
> > ...
> > virtual const std::string& toString() const noexcept;
> > }
> >
> > So the example use would be:
> >
> > auto myObject = region.get("myObject"); // auto =
> std::shared_ptr
> > auto myString = myObject->toString(); // auto = std::string
> > std::cout << myString;
> >
> > Even nicer may be to implement a default << operator for Serializable the
> > outputs the toString so we can do:
> >
> > std::cout << region.get("myObject");
> >
> > As for exporting a to_string into the std namespace, I think that is one
> > that is actually forbidden unlike std::hash/equal_to. I feel like I found
> > that when trying to output std::chrono::duration objects to stream. Can
> you
> > double check that it is allowed?
> >
> > -Jake
> >
> >
> >
> > On Thu, Sep 14, 2017 at 11:30 AM David Kimura 
> wrote:
> >
> >> Seems like a good idea.
> >>
> >> Here's a quick thought - I think c++11 is not really an obj.toString()
> kind
> >> of language (like Java or C#).  Would it make sense to add
> std::to_string()
> >> and std::to_wstring() equivalents for CacheableString?  This might mean
> >> implementing following functions:
> >>
> >>std::string to_string(CacheableString value);
> >>std::wstring to_wstring(CacheableString value);
> >>
> >> So that we can do something like:
> >>
> >>std::cout << std::to_wstring(pdxser);
> >>
> >> Thanks,
> >> David
> >>
> >> On Thu, Sep 14, 2017 at 11:10 AM, Michael William Dodge <
> mdo...@pivotal.io
> >>>
> >> wrote:
> >>
> >>> +1 for std::string and std::wstring.
> >>>
> >>> Sarge
> >>>
>  On 14 Sep, 2017, at 11:10, Mark Hanson  wrote:
> 
>  Hi All,
> 
>  I wanted to broach the subject of moving away from moving away from
> >>> CacheableStringPtrs for the toString representation of Serializable. It
> >>> would seem desirable to move to std::string and std::wstring to use
> more
> >>> basic types that would be faster to log and the code would be simpler
> >> for a
> >>> user.
> 
>  Are there any opinions on this subject?
> 
>  Here is a before and after look at a chunk of code
> 
>  Before
> 
>  CacheableStringPtr ptr = pdxser->toString();
>  if (ptr->isWideString()) {
>  printf(" query idx %d pulled object %S  :: \n", i,
> ptr->asWChar());
>  } else {
>  printf(" query idx %d pulled object %s  :: \n", i,
> ptr->asChar());
>  }
> 
>  After
> 
> 
>  if (pdxser->isWideString()) {
>   std::cout << " query idx “ << i << "pulled object ” <<
> >>> pdxser->toWString() << std::endl;
>  } else {
>   std::cout << " query idx “ << i << "pulled object ” <<
> >>> pdxser->toString() << std::endl;
>  }
> 
> 
>  Thanks,
>  Mark
> >>>
> >>>
> >>
>
>

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Mark Hanson

+1 for Approach 1 based on the fact that it will have a compile error if you 
don’t implement it, and it is more object oriented.

Thanks,
Mark
> On Sep 15, 2017, at 10:18 AM, Mark Hanson  wrote:
> 
> So we have two approaches as their has been a veto on w_string, so it will 
> not be used.
> 
> Approach 1) std::string str = object.to_string();
> Approach 2) std::string str = std::to_string(object);
> 
> Please vote of your preferred.
> 
> Thanks,
> Mark
> 
> 
>> On Sep 14, 2017, at 12:29 PM, Jacob Barrett  wrote:
>> 
>> Don't use std::wstring EVER. The std::wstring has a platform dependent
>> storage class. This means on Windows it is 32bits and Linux it is 16bits.
>> None of the std::string types, and there are more than std::wstring, define
>> an encoding, so would the wstring be UTF-16, UC-2, UTF-32, UC-4, or some
>> other local non-unicode codepage. It gets nasty fast!
>> 
>> If you only support std::string and require the all std::string's be
>> encoded with UTF-8 then things get real clear really fast. Yes there is
>> some overhead in converting to UC-4 for calling windows system calls that
>> want Unicode 32-bit std::wstring but we already had that overhead since
>> almost all strings in NC were actually 8-bit UTF-8 std::string. And since
>> almost all those are ASCII strings and UTF-8 can encode ASCII without
>> overhead we are good to go!
>> 
>> So to simply we would change Serializable::toString to only a std::string.
>> 
>> class Serializable {
>> ...
>> virtual const std::string& toString() const noexcept;
>> }
>> 
>> So the example use would be:
>> 
>> auto myObject = region.get("myObject"); // auto = std::shared_ptr
>> auto myString = myObject->toString(); // auto = std::string
>> std::cout << myString;
>> 
>> Even nicer may be to implement a default << operator for Serializable the
>> outputs the toString so we can do:
>> 
>> std::cout << region.get("myObject");
>> 
>> As for exporting a to_string into the std namespace, I think that is one
>> that is actually forbidden unlike std::hash/equal_to. I feel like I found
>> that when trying to output std::chrono::duration objects to stream. Can you
>> double check that it is allowed?
>> 
>> -Jake
>> 
>> 
>> 
>> On Thu, Sep 14, 2017 at 11:30 AM David Kimura  wrote:
>> 
>>> Seems like a good idea.
>>> 
>>> Here's a quick thought - I think c++11 is not really an obj.toString() kind
>>> of language (like Java or C#).  Would it make sense to add std::to_string()
>>> and std::to_wstring() equivalents for CacheableString?  This might mean
>>> implementing following functions:
>>> 
>>>   std::string to_string(CacheableString value);
>>>   std::wstring to_wstring(CacheableString value);
>>> 
>>> So that we can do something like:
>>> 
>>>   std::cout << std::to_wstring(pdxser);
>>> 
>>> Thanks,
>>> David
>>> 
>>> On Thu, Sep 14, 2017 at 11:10 AM, Michael William Dodge >>> 
>>> wrote:
>>> 
 +1 for std::string and std::wstring.
 
 Sarge
 
> On 14 Sep, 2017, at 11:10, Mark Hanson  wrote:
> 
> Hi All,
> 
> I wanted to broach the subject of moving away from moving away from
 CacheableStringPtrs for the toString representation of Serializable. It
 would seem desirable to move to std::string and std::wstring to use more
 basic types that would be faster to log and the code would be simpler
>>> for a
 user.
> 
> Are there any opinions on this subject?
> 
> Here is a before and after look at a chunk of code
> 
> Before
> 
> CacheableStringPtr ptr = pdxser->toString();
> if (ptr->isWideString()) {
> printf(" query idx %d pulled object %S  :: \n", i,
>   ptr->asWChar());
> } else {
> printf(" query idx %d pulled object %s  :: \n", i,
>   ptr->asChar());
> }
> 
> After
> 
> 
> if (pdxser->isWideString()) {
> std::cout << " query idx “ << i << "pulled object ” <<
 pdxser->toWString() << std::endl;
> } else {
> std::cout << " query idx “ << i << "pulled object ” <<
 pdxser->toString() << std::endl;
> }
> 
> 
> Thanks,
> Mark
 
 
>>> 
>

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Jacob Barrett

-1 to all, comments below.

> On Sep 15, 2017, at 10:18 AM, Mark Hanson  wrote:
> 
> So we have two approaches as their has been a veto on w_string, so it will 
> not be used.

Who said anything about veto? Nobody has veto authority. I simply stated and 
opinion on the use of wstring based on my research in the past.

> Approach 1) std::string str = object.to_string();
To remain consistent with the APIs use of camel case this should be toString();

> Approach 2) std::string str = std::to_string(object);

Nobody countered my assertion that this is not legal to export into std names 
space. 

-Jake

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Mark Hanson

Comments inline….
> On Sep 15, 2017, at 10:25 AM, Jacob Barrett  wrote:
> 
> -1 to all, comments below.
> 
> 
>> On Sep 15, 2017, at 10:18 AM, Mark Hanson  wrote:
>> 
>> So we have two approaches as their has been a veto on w_string, so it will 
>> not be used.
> 
> Who said anything about veto? Nobody has veto authority. I simply stated and 
> opinion on the use of wstring based on my research in the past.
> 

Ok, not vetoed.

>> Approach 1) std::string str = object.to_string();
> To remain consistent with the APIs use of camel case this should be 
> toString();
> 

Uh, yeah. Concept not code...

> 
>> Approach 2) std::string str = std::to_string(object);
> 
> Nobody countered my assertion that this is not legal to export into std names 
> space. 
> 

class blobject
{
  bool blobBool;
};


namespace std {
  std::string to_String(blobject * blob)
  {
return std::string("works");
  }
}


int main() {

  blobject * blob = new blobject();
  std::string str = std::to_String(blob);
  std::cout << "Hello, World! " <<  "value = " << str << std::endl;
  return 0;
}
/Users/mhanson/CLionProjects/untitled1/cmake-build-debug/untitled1
Hello, World! value = works

Process finished with exit code 0


> -Jake
> 
>

Re: 1.3.0 release

2017-09-15 Thread Anthony Baker

Excellent!  Can you review the open issues currently tagged for 1.3.0 (I think 
it’s probably not accurate) and gather consensus on any remaining changes 
needed?

Anthony

> On Sep 12, 2017, at 2:40 PM, Swapnil Bawaskar  wrote:
> 
> Sound good.
> 
> I would like to volunteer to be the release manager.
> 
> Thanks!
> 
> 
> On Tue, Sep 12, 2017 at 2:24 PM Anthony Baker  wrote:
> 
>> Hi all,
>> 
>> I think we should begin discussing scope and timeline for the 1.3.0
>> release.  I know we’re still finalizing 1.2.1, but we released 1.2.0 almost
>> two months ago and we’ve fixed almost 200 issues in that time.  IMO, we
>> should complete 1.2.1 and then immediately turn around 1.3.0.
>> 
>> Thoughts?  Any volunteers for release manager?
>> 
>> Anthony
>> 
>>

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Ernest Burghardt

cool that we could export into the std namespace, but as a library I don't
think it is a good idea as a user of our library might have done the same
thing and that is a situation we can easily/pro-actively avoid

+1 object::toString()

On Fri, Sep 15, 2017 at 11:33 AM, Mark Hanson  wrote:

> Comments inline….
> > On Sep 15, 2017, at 10:25 AM, Jacob Barrett  wrote:
> >
> > -1 to all, comments below.
> >
> >
> >> On Sep 15, 2017, at 10:18 AM, Mark Hanson  wrote:
> >>
> >> So we have two approaches as their has been a veto on w_string, so it
> will not be used.
> >
> > Who said anything about veto? Nobody has veto authority. I simply stated
> and opinion on the use of wstring based on my research in the past.
> >
>
> Ok, not vetoed.
>
> >> Approach 1) std::string str = object.to_string();
> > To remain consistent with the APIs use of camel case this should be
> toString();
> >
>
> Uh, yeah. Concept not code...
>
> >
> >> Approach 2) std::string str = std::to_string(object);
> >
> > Nobody countered my assertion that this is not legal to export into std
> names space.
> >
>
> class blobject
> {
>   bool blobBool;
> };
>
>
> namespace std {
>   std::string to_String(blobject * blob)
>   {
> return std::string("works");
>   }
> }
>
>
> int main() {
>
>   blobject * blob = new blobject();
>   std::string str = std::to_String(blob);
>   std::cout << "Hello, World! " <<  "value = " << str << std::endl;
>   return 0;
> }
> /Users/mhanson/CLionProjects/untitled1/cmake-build-debug/untitled1
> Hello, World! value = works
>
> Process finished with exit code 0
>
>
> > -Jake
> >
> >
>
>

[Vote] virtual void fromData(PdxReaderPtr input) to virtual void fromData(const PdxReaderPtr input);

2017-09-15 Thread Mark Hanson

Note the const on PdxReaderPtr virtual void fromData(const PdxReaderPtr input);

Hi All,

So there is a question outstanding about whether or not we want to add const to 
PdxReaderPtr for fromData calls. It seems reasonable and plausible.

Proposal: Change to virtual void fromData(const PdxReaderPtr input) from 
virtual void fromData(PdxReaderPtr input) 

Get your +1’s and -1’s in to vote.

Thanks,
Mark

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread David Kimura

Actually, it looks like Jake is right.  According to documentation it's
undefined behavior since it's not a template specialization.

http://en.cppreference.com/w/cpp/language/extending_std

Thanks,
David

On Fri, Sep 15, 2017 at 10:37 AM, Ernest Burghardt 
wrote:

> cool that we could export into the std namespace, but as a library I don't
> think it is a good idea as a user of our library might have done the same
> thing and that is a situation we can easily/pro-actively avoid
>
> +1 object::toString()
>
> On Fri, Sep 15, 2017 at 11:33 AM, Mark Hanson  wrote:
>
> > Comments inline….
> > > On Sep 15, 2017, at 10:25 AM, Jacob Barrett 
> wrote:
> > >
> > > -1 to all, comments below.
> > >
> > >
> > >> On Sep 15, 2017, at 10:18 AM, Mark Hanson  wrote:
> > >>
> > >> So we have two approaches as their has been a veto on w_string, so it
> > will not be used.
> > >
> > > Who said anything about veto? Nobody has veto authority. I simply
> stated
> > and opinion on the use of wstring based on my research in the past.
> > >
> >
> > Ok, not vetoed.
> >
> > >> Approach 1) std::string str = object.to_string();
> > > To remain consistent with the APIs use of camel case this should be
> > toString();
> > >
> >
> > Uh, yeah. Concept not code...
> >
> > >
> > >> Approach 2) std::string str = std::to_string(object);
> > >
> > > Nobody countered my assertion that this is not legal to export into std
> > names space.
> > >
> >
> > class blobject
> > {
> >   bool blobBool;
> > };
> >
> >
> > namespace std {
> >   std::string to_String(blobject * blob)
> >   {
> > return std::string("works");
> >   }
> > }
> >
> >
> > int main() {
> >
> >   blobject * blob = new blobject();
> >   std::string str = std::to_String(blob);
> >   std::cout << "Hello, World! " <<  "value = " << str << std::endl;
> >   return 0;
> > }
> > /Users/mhanson/CLionProjects/untitled1/cmake-build-debug/untitled1
> > Hello, World! value = works
> >
> > Process finished with exit code 0
> >
> >
> > > -Jake
> > >
> > >
> >
> >
>

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Jacob Barrett

On Fri, Sep 15, 2017 at 10:33 AM Mark Hanson  wrote:

> class blobject
> {
>   bool blobBool;
> };
>
>
> namespace std {
>   std::string to_String(blobject * blob)
>   {
> return std::string("works");
>   }
> }
>
>
> int main() {
>
>   blobject * blob = new blobject();
>   std::string str = std::to_String(blob);
>   std::cout << "Hello, World! " <<  "value = " << str << std::endl;
>   return 0;
> }
> /Users/mhanson/CLionProjects/untitled1/cmake-build-debug/untitled1
> Hello, World! value = works
>

Yes, this asserts it is possible, which was not in question. My assertion
is that the specification forbids exporting code into the std namespace
with very few exceptions.

http://en.cppreference.com/w/cpp/language/extending_std

In another review of the specification extending the std::to_string is
allowed assuming that it is only done for user defined types and none of
the std types. In our use case it would be allowed.

I would suggest however that we keep Serializable::toString for consistency
with previous API and implement:
namespace std {
  std::string to_string(const Serializable& object) {
return object.toString();
  }
  std::string to_string(const Serializable* object) {
return object->toString();
  }
  std::string to_string(const std::shared_ptr& object) {
return object->toString();
  }
  std::string to_string(const std::unique_ptr& object) {
return object->toString();
  }
}

-Jake

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Jacob Barrett

On Fri, Sep 15, 2017 at 10:40 AM David Kimura  wrote:

> Actually, it looks like Jake is right.  According to documentation it's
> undefined behavior since it's not a template specialization.
>
> http://en.cppreference.com/w/cpp/language/extending_std
>
>
Oh yeah, I know I rejected the idea for a reason the first time. Thanks for
pointing out the template specialization issue.

-Jake

Re: [DISCUSS] authorizing function execution

2017-09-15 Thread Michael Stolz

Yeah that's the right level of authorization. The Function Author should
get to decide.
Listeners, Loaders and Writers should only require DATA:READ and DATA:WRITE
as appropriate.
It is up to the authors what they do inside of those.

--
Mike Stolz
Principal Engineer, GemFire Product Manager
Mobile: +1-631-835-4771

On Thu, Sep 14, 2017 at 2:29 PM, Swapnil Bawaskar 
wrote:

> Indeed, the function author has a higher level of privilege than someone
> who is invoking a function, that is why the proposal here is to let the
> function author choose what level of permissions are required to invoke a
> function.
>
> On Thu, Sep 14, 2017 at 11:20 AM Anilkumar Gingade 
> wrote:
>
> > >> from reaching into internal classes
> > If thats the case; one could do anything, even with read
> permission...Isn't
> > it...
> >
> >
> >
> > On Thu, Sep 14, 2017 at 10:43 AM, Jared Stewart 
> > wrote:
> >
> > > There is nothing to prevent code in a function that's executing on a
> > > server from reaching into internal classes and bypassing the public
> > region
> > > APIs. I think a function's author should ultimately determine the
> > > permissions required to execute it.
> > >
> > > > On Sep 14, 2017, at 10:34 AM, Anilkumar Gingade  >
> > > wrote:
> > > >
> > > > When a function is accessing/modifying region; the function will be
> > doing
> > > > so by region apis, don't we have credential check with region apis;
> if
> > > not
> > > > can we add those checks here...instead of having it in the
> function...
> > > >
> > > > -Anil.
> > > >
> > > >
> > > >
> > > > On Wed, Sep 13, 2017 at 11:22 AM, Jared Stewart  >
> > > wrote:
> > > >
> > > >> After some more investigation into the implementation details, here
> is
> > > our
> > > >> updated proposal to add to the Function interface:
> > > >>
> > > >> default Collection getRequiredPermissions(
> > > Optional
> > > >> onRegion) {
> > > >>  return Collections.singletonList(ResourcePermissions.DATA_WRITE);
> > > >> }
> > > >>
> > > >> This method can be overridden by Function authors who want to
> require
> > > >> permissions other than DATA:WRITE.. The onRegion parameter will be
> > > present
> > > >> only when a Function is executed via FunctionService.onRegion, and
> is
> > > >> intended to allow Function authors to require different permissions
> > > >> depending on the Region which Function will be executed on.  We pass
> > the
> > > >> region name into this method rather than the full FunctionContext
> > > because
> > > >> the latter would be much more expansive to implement.
> > > >>
> > > >> Any feedback is appreciated.
> > > >>
> > > >> Thanks,
> > > >> Jared
> > > >>
> > > >>> On Aug 17, 2017, at 1:42 AM, Swapnil Bawaskar <
> sbawas...@pivotal.io>
> > > >> wrote:
> > > >>>
> > > >>> Discuss fix for GEODE-2817
> > > >>> 
> > > >>>
> > > >>> Currently to execute a function, you will need "data:write"
> > permission,
> > > >> but
> > > >>> it really depends on what the function is doing. For example, if a
> > > >> function
> > > >>> is just reading data, the function author might want users with
> > > DATA:READ
> > > >>> permissions to execute the function. The two options mentioned in
> the
> > > >>> ticket are:
> > > >>>
> > > >>> 1) externalize SecurityService so that function author can use it
> in
> > > the
> > > >>> function.execute code to check authorization.
> > > >>> 2) add a method to function interface to tell the framework what
> > > >> permission
> > > >>> this function needs to execute, so that the framework will check
> the
> > > >>> permission before executing the function.
> > > >>>
> > > >>> I vote for #2 because, I think, a function author will be able to
> > > easily
> > > >>> discover a method on the Function interface, rather than trying to
> > look
> > > >> for
> > > >>> SecurityService.
> > > >>>
> > > >>> I propose that we add the following new method to Function:
> > > >>>
> > > >>> default public List requiredPermissions() {
> > > >>>  // default DATA:WRITE
> > > >>> }
> > > >>>
> > > >>> In order to preserve existing behavior, the default required
> > permission
> > > >>> would be DATA:WRITE.
> > > >>
> > > >>
> > >
> > >
> >
>

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Mark Hanson

My vote is unchanged…
+1 for on the object.

Thanks,
Mark
> On Sep 15, 2017, at 10:47 AM, Jacob Barrett  wrote:
> 
> On Fri, Sep 15, 2017 at 10:40 AM David Kimura  wrote:
> 
>> Actually, it looks like Jake is right.  According to documentation it's
>> undefined behavior since it's not a template specialization.
>> 
>> http://en.cppreference.com/w/cpp/language/extending_std
>> 
>> 
> Oh yeah, I know I rejected the idea for a reason the first time. Thanks for
> pointing out the template specialization issue.
> 
> -Jake

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Ernest Burghardt

Not convinced or grok'g doing both...  what is the benefit of adding the
std::to_string()'s?

On Fri, Sep 15, 2017 at 11:46 AM, Jacob Barrett  wrote:

> On Fri, Sep 15, 2017 at 10:33 AM Mark Hanson  wrote:
>
> > class blobject
> > {
> >   bool blobBool;
> > };
> >
> >
> > namespace std {
> >   std::string to_String(blobject * blob)
> >   {
> > return std::string("works");
> >   }
> > }
> >
> >
> > int main() {
> >
> >   blobject * blob = new blobject();
> >   std::string str = std::to_String(blob);
> >   std::cout << "Hello, World! " <<  "value = " << str << std::endl;
> >   return 0;
> > }
> > /Users/mhanson/CLionProjects/untitled1/cmake-build-debug/untitled1
> > Hello, World! value = works
> >
>
> Yes, this asserts it is possible, which was not in question. My assertion
> is that the specification forbids exporting code into the std namespace
> with very few exceptions.
>
> http://en.cppreference.com/w/cpp/language/extending_std
>
> In another review of the specification extending the std::to_string is
> allowed assuming that it is only done for user defined types and none of
> the std types. In our use case it would be allowed.
>
> I would suggest however that we keep Serializable::toString for consistency
> with previous API and implement:
> namespace std {
>   std::string to_string(const Serializable& object) {
> return object.toString();
>   }
>   std::string to_string(const Serializable* object) {
> return object->toString();
>   }
>   std::string to_string(const std::shared_ptr& object) {
> return object->toString();
>   }
>   std::string to_string(const std::unique_ptr& object) {
> return object->toString();
>   }
> }
>
> -Jake
>

Re: [Vote] virtual void fromData(PdxReaderPtr input) to virtual void fromData(const PdxReaderPtr input);

2017-09-15 Thread Jacob Barrett

-1

PdxReader holds state. All calls to read the state on PdxReader also result
in a change in that state.

On Fri, Sep 15, 2017 at 10:40 AM Mark Hanson  wrote:

> Note the const on PdxReaderPtr virtual void fromData(const PdxReaderPtr
> input);
>
> Hi All,
>
> So there is a question outstanding about whether or not we want to add
> const to PdxReaderPtr for fromData calls. It seems reasonable and plausible.
>
> Proposal: Change to virtual void fromData(const PdxReaderPtr input) from
> virtual void fromData(PdxReaderPtr input)
>
> Get your +1’s and -1’s in to vote.
>
> Thanks,
> Mark

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Jacob Barrett

On Fri, Sep 15, 2017 at 10:50 AM Ernest Burghardt 
wrote:

> Not convinced or grok'g doing both...  what is the benefit of adding the
> std::to_string()'s?
>

Convenience only, but as David pointed out it is not allowed.

-Jake

Re: [Vote] virtual void fromData(PdxReaderPtr input) to virtual void fromData(const PdxReaderPtr input);

2017-09-15 Thread Ernest Burghardt

+1 for const PdxReader


- remember, vote early and vote often...

On Fri, Sep 15, 2017 at 11:40 AM, Mark Hanson  wrote:

> Note the const on PdxReaderPtr virtual void fromData(const PdxReaderPtr
> input);
>
> Hi All,
>
> So there is a question outstanding about whether or not we want to add
> const to PdxReaderPtr for fromData calls. It seems reasonable and plausible.
>
> Proposal: Change to virtual void fromData(const PdxReaderPtr input) from
> virtual void fromData(PdxReaderPtr input)
>
> Get your +1’s and -1’s in to vote.
>
> Thanks,
> Mark

Re: [Vote] virtual void fromData(PdxReaderPtr input) to virtual void fromData(const PdxReaderPtr input);

2017-09-15 Thread Mark Hanson

Readers do in fact hold state, but they do not appear to modify the state 
during normal reads. I could be wrong on that, but that is what I just read. 

Thanks,
Mark
> On Sep 15, 2017, at 10:51 AM, Jacob Barrett  wrote:
> 
> -1
> 
> PdxReader holds state. All calls to read the state on PdxReader also result
> in a change in that state.
> 
> On Fri, Sep 15, 2017 at 10:40 AM Mark Hanson  wrote:
> 
>> Note the const on PdxReaderPtr virtual void fromData(const PdxReaderPtr
>> input);
>> 
>> Hi All,
>> 
>> So there is a question outstanding about whether or not we want to add
>> const to PdxReaderPtr for fromData calls. It seems reasonable and plausible.
>> 
>> Proposal: Change to virtual void fromData(const PdxReaderPtr input) from
>> virtual void fromData(PdxReaderPtr input)
>> 
>> Get your +1’s and -1’s in to vote.
>> 
>> Thanks,
>> Mark

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Ernest Burghardt

okay so we have +1s on object.toString()

any dissenting opinions?

On Fri, Sep 15, 2017 at 11:52 AM, Jacob Barrett  wrote:

> On Fri, Sep 15, 2017 at 10:50 AM Ernest Burghardt 
> wrote:
>
> > Not convinced or grok'g doing both...  what is the benefit of adding the
> > std::to_string()'s?
> >
>
> Convenience only, but as David pointed out it is not allowed.
>
> -Jake
>

Re: [VOTE] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

2017-09-15 Thread Jacob Barrett

+1 for:

std::string toString() const noexcept;





On Fri, Sep 15, 2017 at 10:52 AM Jacob Barrett  wrote:

> On Fri, Sep 15, 2017 at 10:50 AM Ernest Burghardt 
> wrote:
>
>> Not convinced or grok'g doing both...  what is the benefit of adding the
>> std::to_string()'s?
>>
>
> Convenience only, but as David pointed out it is not allowed.
>
> -Jake
>

Re: [Vote] virtual void fromData(PdxReaderPtr input) to virtual void fromData(const PdxReaderPtr input);

2017-09-15 Thread Jacob Barrett

PdxReader advances the cursor of the internal buffer on each call. If we
consider that each call moves the cursor to the requested field position we
could make the cursor mutable but we also need to specify that all accessor
fields are const as well. If we do this then I could support the approach.
The use of PdxReader in this way, while const, is not thread safe because
the object member variable is mutated on each call non-atomically.

-Jake

On Fri, Sep 15, 2017 at 10:53 AM Mark Hanson  wrote:

> Readers do in fact hold state, but they do not appear to modify the state
> during normal reads. I could be wrong on that, but that is what I just read.
>
> Thanks,
> Mark
> > On Sep 15, 2017, at 10:51 AM, Jacob Barrett  wrote:
> >
> > -1
> >
> > PdxReader holds state. All calls to read the state on PdxReader also
> result
> > in a change in that state.
> >
> > On Fri, Sep 15, 2017 at 10:40 AM Mark Hanson  wrote:
> >
> >> Note the const on PdxReaderPtr virtual void fromData(const PdxReaderPtr
> >> input);
> >>
> >> Hi All,
> >>
> >> So there is a question outstanding about whether or not we want to add
> >> const to PdxReaderPtr for fromData calls. It seems reasonable and
> plausible.
> >>
> >> Proposal: Change to virtual void fromData(const PdxReaderPtr input) from
> >> virtual void fromData(PdxReaderPtr input)
> >>
> >> Get your +1’s and -1’s in to vote.
> >>
> >> Thanks,
> >> Mark
>
>

New client/server protocol - seeking feedback

2017-09-15 Thread Brian Baynes

Greetings, friends of Geode.

Work has been progressing on the new client/server protocol for Geode and
we're approaching a GA v1.0.  Completed work/features include put/get,
putAll, getAll, remove, one-way SSL, authentication and authorization, and
support for primitive types and JSON documents as values (saved in PDX on
the server).

Invite you to review the road map toward GA v1.0, the features proposed for
post-v1.0, and give us your feedback!  (Directly in Confluence or here on
dev@geode.apache.org)

New Client/Server Protocol - Road Map, Proposed



Thanks for your input,

Brian

[DISCUSS] Framework for concurrency tests

2017-09-15 Thread Dan Smith

Hi Geode devs,

I've been messing around with an open source tool called Java
Pathfinder for writing tests of multithreaded code. Java Pathfinder is
a special JVM which among other things tries to execute your code
using all possible thread interleavings.

I'd like to propose two things:

1) We introduce a framework for writing unit tests of code that is
supposed to be thread safe. This framework should let a developer
easily write a test with multiple things going on in parallel. The
framework can then take that code and try to run it with different
thread interleavings.

Here's an example of what this could look like:

@RunWith(ConcurrentTestRunner.class)
public class AtomicIntegerTest {

  @Test
  public void parallelIncrementReturns2(ParallelExecutor executor)
  throws ExecutionException, InterruptedException {
AtomicInteger atomicInteger = new AtomicInteger();
executor.inParallel(() -> atomicInteger.incrementAndGet());
executor.inParallel(() -> atomicInteger.incrementAndGet());
executor.execute();
assertEquals(2, atomicInteger.get());
  }


2) We implement this framework initially using Java Pathfinder, but
allow for other methods of testing the code to be plugged in for
example just running the test in the loop. Java pathfinder is cool
because it can run the code with different interleavings but it does
have some serious limitations.

I've put together some code for this proposal which is available in
this github PR:

https://github.com/apache/geode/pull/787

What do you think?

-Dan

Re: [DISCUSS] Framework for concurrency tests

2017-09-15 Thread Jacob Barrett

Love it! See my comments in your pull request.

On Fri, Sep 15, 2017 at 12:08 PM Dan Smith  wrote:

> Hi Geode devs,
>
> I've been messing around with an open source tool called Java
> Pathfinder for writing tests of multithreaded code. Java Pathfinder is
> a special JVM which among other things tries to execute your code
> using all possible thread interleavings.
>
> I'd like to propose two things:
>
> 1) We introduce a framework for writing unit tests of code that is
> supposed to be thread safe. This framework should let a developer
> easily write a test with multiple things going on in parallel. The
> framework can then take that code and try to run it with different
> thread interleavings.
>
> Here's an example of what this could look like:
>
> @RunWith(ConcurrentTestRunner.class)
> public class AtomicIntegerTest {
>
>   @Test
>   public void parallelIncrementReturns2(ParallelExecutor executor)
>   throws ExecutionException, InterruptedException {
> AtomicInteger atomicInteger = new AtomicInteger();
> executor.inParallel(() -> atomicInteger.incrementAndGet());
> executor.inParallel(() -> atomicInteger.incrementAndGet());
> executor.execute();
> assertEquals(2, atomicInteger.get());
>   }
>
>
> 2) We implement this framework initially using Java Pathfinder, but
> allow for other methods of testing the code to be plugged in for
> example just running the test in the loop. Java pathfinder is cool
> because it can run the code with different interleavings but it does
> have some serious limitations.
>
> I've put together some code for this proposal which is available in
> this github PR:
>
> https://github.com/apache/geode/pull/787
>
> What do you think?
>
> -Dan
>

Re: [DISCUSS] Clean build takes 10minutes to complete now

2017-09-15 Thread Jason Huynh

For the original issue, where the old version is pulling down the old
versions during compilation time, we have a pull request to use the maven
repo: https://github.com/apache/geode/pull/790

The rest of the tests added for the session state are marked as @Category
({DistributedTest.class, BackwardCompatibilityTest.class})
 geode-old-versions happens to be required at compile time.

The old versions will now (after the pull request is checked in) pull down
the zip into the local repo once.  Unzipping will still be required but
that should be a lot shorter than downloading.

Whether we should move the old versions out of the compile task is a
different issue...



On Fri, Sep 15, 2017 at 8:55 AM Kirk Lund  wrote:

> The actual tests marked with UnitTest category are actually pretty good.
> They all run in just over one minute and almost all of them use Mockito to
> isolate one class. I remember seeing one newer Lucene UnitTest that touches
> File System which should be recategorized as IntegrationTest.
>
> If we could move the pulling down of previous versions of Geode out of the
> main build+unit-test target, that would help a lot.
>
> Even prior to the pulling down of previous versions for backwards compat
> testing, the main build (without unit-test) was too slow and I think it's
> because our project is a little too complex for what Gradle is designed to
> handle.
>
> Code generation and javadocs are two of the tasks in our main build
> (without unit-test) that contributes to it taking too long.
>
> Also, the way Gradle handles junit categories is designed and coded very
> inefficiently -- if we could change their junit runner to use
> FastClasspathScanner to find all tests containing the targeted junit
> category annotation then that would speed up all of our testing targets
> immensely. Any testing target that forks JVMs runs super slow due to the
> way they handle categories (this effects IntegrationTest, DistributedTest,
> FlakyTest).
>
> On Fri, Sep 15, 2017 at 8:44 AM, Alexander Murmann 
> wrote:
>
> > I fully agree with Udo here. The main build should be for Unit tests. Our
> > "Unit Tests" are already exercising much more of the system than they
> > should. Adding unit tests that not only too much or our current code but
> > also old code is moving us in the wrong direction. Let's keep the tests,
> > but please appropriately mark them as IntegrationTest.
> >
> > On Tue, Sep 12, 2017 at 9:30 AM, Udo Kohlmeyer 
> > wrote:
> >
> > > My apologies, I might gotten the commit reason incorrect. I just know
> > that
> > > downloading the older product version every time is becoming painful.
> > > Yes, sometimes it is faster than other times, but imo, this is not
> > > something that should be part of the main build path.
> > >
> > > Backwards compat or integration testing should not be running as part
> of
> > > the main build task.
> > >
> > > --Udo
> > >
> > > On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag  wrote:
> > >
> > > > As we are working on fixing this issue, some extra parameters may
> help
> > > the
> > > > build to get bit quicker on your machine.
> > > >
> > > > using -xjavadoc -xdoc
> > > > Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> > > > BUILD SUCCESSFUL
> > > > Total time: 2 mins 2.729 secs
> > > >
> > > >
> > > > Also, I think as Jason mentioned that the slow down is due to full
> > > product
> > > > download for session state tests. LuceneSearchWithRollingUpgradeDUnit
> > > > tests
> > > > were added  in July. Please do correct me if I am wrong.
> > > >
> > > > Regards
> > > > Nabarun
> > > >
> > > >
> > > > On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann <
> > amurm...@pivotal.io>
> > > > wrote:
> > > >
> > > > > Could we make it so that these tests for now are only run as part
> of
> > > > > pre-checkin till we got this ironed out and then revisit this?
> > > > >
> > > > > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> > > > bschucha...@pivotal.io>
> > > > > wrote:
> > > > >
> > > > > > The geode-old-versions module was originally created to pull in
> old
> > > > > > version jar files into your gradle cache.  This happened only
> once
> > > and
> > > > > you
> > > > > > were good to go.  I don't think that part should be backed out as
> > it
> > > > has
> > > > > > minimal impact and is not affecting build time.
> > > > > >
> > > > > > The recent changes for lucene testing seem to be pulling in full
> > > > > > installations of old versions and these are deleted as part of
> the
> > > > > "clean"
> > > > > > gradle task.  That's causing them to be downloaded again each
> time
> > > you
> > > > > do a
> > > > > > clean&build.  Dan put changes in place so that the files aren't
> > > > > downloaded
> > > > > > again if you build without cleaning but clearly more needs to be
> > done
> > > > in
> > > > > > this area.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 9/11/17 11:23 AM, Jacob Barrett wrote:
> > > > > >
> > > > > >> Agreed, integration tests shou

Re: 1.3.0 release

2017-09-15 Thread Swapnil Bawaskar

I took preliminary look and tagged some issues for 1.3.0.
Looks like we have 11 issues remaining:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-2788&versions=visible&selectedVersion=12340669

Please take a look at these issues to see which are not critical to fix in
1.3 and also look at issues assigned to you/reported by you to see if they
must be tagged for 1.3.

Thanks!

On Fri, Sep 15, 2017 at 10:36 AM Anthony Baker  wrote:

> Excellent!  Can you review the open issues currently tagged for 1.3.0 (I
> think it’s probably not accurate) and gather consensus on any remaining
> changes needed?
>
> Anthony
>
> > On Sep 12, 2017, at 2:40 PM, Swapnil Bawaskar 
> wrote:
> >
> > Sound good.
> >
> > I would like to volunteer to be the release manager.
> >
> > Thanks!
> >
> >
> > On Tue, Sep 12, 2017 at 2:24 PM Anthony Baker  wrote:
> >
> >> Hi all,
> >>
> >> I think we should begin discussing scope and timeline for the 1.3.0
> >> release.  I know we’re still finalizing 1.2.1, but we released 1.2.0
> almost
> >> two months ago and we’ve fixed almost 200 issues in that time.  IMO, we
> >> should complete 1.2.1 and then immediately turn around 1.3.0.
> >>
> >> Thoughts?  Any volunteers for release manager?
> >>
> >> Anthony
> >>
> >>
>
>

Re: [DISCUSS] Clean build takes 10minutes to complete now

2017-09-15 Thread Udo Kohlmeyer

I think that the moving the old version out of the compilation step is
paramount. Could the download of the artifact be made a dependency of the
BackwardCompatibilityTest?

--Udo

On Fri, Sep 15, 2017 at 1:08 PM, Jason Huynh  wrote:

> For the original issue, where the old version is pulling down the old
> versions during compilation time, we have a pull request to use the maven
> repo: https://github.com/apache/geode/pull/790
>
> The rest of the tests added for the session state are marked as @Category
> ({DistributedTest.class, BackwardCompatibilityTest.class})
>  geode-old-versions happens to be required at compile time.
>
> The old versions will now (after the pull request is checked in) pull down
> the zip into the local repo once.  Unzipping will still be required but
> that should be a lot shorter than downloading.
>
> Whether we should move the old versions out of the compile task is a
> different issue...
>
>
>
> On Fri, Sep 15, 2017 at 8:55 AM Kirk Lund  wrote:
>
> > The actual tests marked with UnitTest category are actually pretty good.
> > They all run in just over one minute and almost all of them use Mockito
> to
> > isolate one class. I remember seeing one newer Lucene UnitTest that
> touches
> > File System which should be recategorized as IntegrationTest.
> >
> > If we could move the pulling down of previous versions of Geode out of
> the
> > main build+unit-test target, that would help a lot.
> >
> > Even prior to the pulling down of previous versions for backwards compat
> > testing, the main build (without unit-test) was too slow and I think it's
> > because our project is a little too complex for what Gradle is designed
> to
> > handle.
> >
> > Code generation and javadocs are two of the tasks in our main build
> > (without unit-test) that contributes to it taking too long.
> >
> > Also, the way Gradle handles junit categories is designed and coded very
> > inefficiently -- if we could change their junit runner to use
> > FastClasspathScanner to find all tests containing the targeted junit
> > category annotation then that would speed up all of our testing targets
> > immensely. Any testing target that forks JVMs runs super slow due to the
> > way they handle categories (this effects IntegrationTest,
> DistributedTest,
> > FlakyTest).
> >
> > On Fri, Sep 15, 2017 at 8:44 AM, Alexander Murmann 
> > wrote:
> >
> > > I fully agree with Udo here. The main build should be for Unit tests.
> Our
> > > "Unit Tests" are already exercising much more of the system than they
> > > should. Adding unit tests that not only too much or our current code
> but
> > > also old code is moving us in the wrong direction. Let's keep the
> tests,
> > > but please appropriately mark them as IntegrationTest.
> > >
> > > On Tue, Sep 12, 2017 at 9:30 AM, Udo Kohlmeyer 
> > > wrote:
> > >
> > > > My apologies, I might gotten the commit reason incorrect. I just know
> > > that
> > > > downloading the older product version every time is becoming painful.
> > > > Yes, sometimes it is faster than other times, but imo, this is not
> > > > something that should be part of the main build path.
> > > >
> > > > Backwards compat or integration testing should not be running as part
> > of
> > > > the main build task.
> > > >
> > > > --Udo
> > > >
> > > > On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag 
> wrote:
> > > >
> > > > > As we are working on fixing this issue, some extra parameters may
> > help
> > > > the
> > > > > build to get bit quicker on your machine.
> > > > >
> > > > > using -xjavadoc -xdoc
> > > > > Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> > > > > BUILD SUCCESSFUL
> > > > > Total time: 2 mins 2.729 secs
> > > > >
> > > > >
> > > > > Also, I think as Jason mentioned that the slow down is due to full
> > > > product
> > > > > download for session state tests. LuceneSearchWithRollingUpgrade
> DUnit
> > > > > tests
> > > > > were added  in July. Please do correct me if I am wrong.
> > > > >
> > > > > Regards
> > > > > Nabarun
> > > > >
> > > > >
> > > > > On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann <
> > > amurm...@pivotal.io>
> > > > > wrote:
> > > > >
> > > > > > Could we make it so that these tests for now are only run as part
> > of
> > > > > > pre-checkin till we got this ironed out and then revisit this?
> > > > > >
> > > > > > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> > > > > bschucha...@pivotal.io>
> > > > > > wrote:
> > > > > >
> > > > > > > The geode-old-versions module was originally created to pull in
> > old
> > > > > > > version jar files into your gradle cache.  This happened only
> > once
> > > > and
> > > > > > you
> > > > > > > were good to go.  I don't think that part should be backed out
> as
> > > it
> > > > > has
> > > > > > > minimal impact and is not affecting build time.
> > > > > > >
> > > > > > > The recent changes for lucene testing seem to be pulling in
> full
> > > > > > > installations of old versions and these are deleted as part of
> > the
>

Re: [DISCUSS] Framework for concurrency tests

2017-09-15 Thread Michael William Dodge

+1 for unit tests for multithreaded code.

High fives to Dan.

Sarge

> On 15 Sep, 2017, at 12:08, Dan Smith  wrote:
> 
> Hi Geode devs,
> 
> I've been messing around with an open source tool called Java
> Pathfinder for writing tests of multithreaded code. Java Pathfinder is
> a special JVM which among other things tries to execute your code
> using all possible thread interleavings.
> 
> I'd like to propose two things:
> 
> 1) We introduce a framework for writing unit tests of code that is
> supposed to be thread safe. This framework should let a developer
> easily write a test with multiple things going on in parallel. The
> framework can then take that code and try to run it with different
> thread interleavings.
> 
> Here's an example of what this could look like:
> 
> @RunWith(ConcurrentTestRunner.class)
> public class AtomicIntegerTest {
> 
>  @Test
>  public void parallelIncrementReturns2(ParallelExecutor executor)
>  throws ExecutionException, InterruptedException {
>AtomicInteger atomicInteger = new AtomicInteger();
>executor.inParallel(() -> atomicInteger.incrementAndGet());
>executor.inParallel(() -> atomicInteger.incrementAndGet());
>executor.execute();
>assertEquals(2, atomicInteger.get());
>  }
> 
> 
> 2) We implement this framework initially using Java Pathfinder, but
> allow for other methods of testing the code to be plugged in for
> example just running the test in the loop. Java pathfinder is cool
> because it can run the code with different interleavings but it does
> have some serious limitations.
> 
> I've put together some code for this proposal which is available in
> this github PR:
> 
> https://github.com/apache/geode/pull/787
> 
> What do you think?
> 
> -Dan

Re: [DISCUSS] Framework for concurrency tests

2017-09-15 Thread Galen O'Sullivan

+1 This is great! I'll take a look at your PR when I get the time.

We may want to think carefully about how often we run these tests, because
unlike regular unit tests, they will take forever to run.

On Fri, Sep 15, 2017 at 1:42 PM, Michael William Dodge 
wrote:

> +1 for unit tests for multithreaded code.
>
> High fives to Dan.
>
> Sarge
>
> > On 15 Sep, 2017, at 12:08, Dan Smith  wrote:
> >
> > Hi Geode devs,
> >
> > I've been messing around with an open source tool called Java
> > Pathfinder for writing tests of multithreaded code. Java Pathfinder is
> > a special JVM which among other things tries to execute your code
> > using all possible thread interleavings.
> >
> > I'd like to propose two things:
> >
> > 1) We introduce a framework for writing unit tests of code that is
> > supposed to be thread safe. This framework should let a developer
> > easily write a test with multiple things going on in parallel. The
> > framework can then take that code and try to run it with different
> > thread interleavings.
> >
> > Here's an example of what this could look like:
> >
> > @RunWith(ConcurrentTestRunner.class)
> > public class AtomicIntegerTest {
> >
> >  @Test
> >  public void parallelIncrementReturns2(ParallelExecutor executor)
> >  throws ExecutionException, InterruptedException {
> >AtomicInteger atomicInteger = new AtomicInteger();
> >executor.inParallel(() -> atomicInteger.incrementAndGet());
> >executor.inParallel(() -> atomicInteger.incrementAndGet());
> >executor.execute();
> >assertEquals(2, atomicInteger.get());
> >  }
> >
> >
> > 2) We implement this framework initially using Java Pathfinder, but
> > allow for other methods of testing the code to be plugged in for
> > example just running the test in the loop. Java pathfinder is cool
> > because it can run the code with different interleavings but it does
> > have some serious limitations.
> >
> > I've put together some code for this proposal which is available in
> > this github PR:
> >
> > https://github.com/apache/geode/pull/787
> >
> > What do you think?
> >
> > -Dan
>
>

Re: [DISCUSS] Framework for concurrency tests

2017-09-15 Thread Jacob Barrett

What? You don’t think Travis can run these fast? 

> On Sep 15, 2017, at 2:07 PM, Galen O'Sullivan  wrote:
> 
> +1 This is great! I'll take a look at your PR when I get the time.
> 
> We may want to think carefully about how often we run these tests, because
> unlike regular unit tests, they will take forever to run.
> 
> On Fri, Sep 15, 2017 at 1:42 PM, Michael William Dodge 
> wrote:
> 
>> +1 for unit tests for multithreaded code.
>> 
>> High fives to Dan.
>> 
>> Sarge
>> 
>>> On 15 Sep, 2017, at 12:08, Dan Smith  wrote:
>>> 
>>> Hi Geode devs,
>>> 
>>> I've been messing around with an open source tool called Java
>>> Pathfinder for writing tests of multithreaded code. Java Pathfinder is
>>> a special JVM which among other things tries to execute your code
>>> using all possible thread interleavings.
>>> 
>>> I'd like to propose two things:
>>> 
>>> 1) We introduce a framework for writing unit tests of code that is
>>> supposed to be thread safe. This framework should let a developer
>>> easily write a test with multiple things going on in parallel. The
>>> framework can then take that code and try to run it with different
>>> thread interleavings.
>>> 
>>> Here's an example of what this could look like:
>>> 
>>> @RunWith(ConcurrentTestRunner.class)
>>> public class AtomicIntegerTest {
>>> 
>>> @Test
>>> public void parallelIncrementReturns2(ParallelExecutor executor)
>>> throws ExecutionException, InterruptedException {
>>>   AtomicInteger atomicInteger = new AtomicInteger();
>>>   executor.inParallel(() -> atomicInteger.incrementAndGet());
>>>   executor.inParallel(() -> atomicInteger.incrementAndGet());
>>>   executor.execute();
>>>   assertEquals(2, atomicInteger.get());
>>> }
>>> 
>>> 
>>> 2) We implement this framework initially using Java Pathfinder, but
>>> allow for other methods of testing the code to be plugged in for
>>> example just running the test in the loop. Java pathfinder is cool
>>> because it can run the code with different interleavings but it does
>>> have some serious limitations.
>>> 
>>> I've put together some code for this proposal which is available in
>>> this github PR:
>>> 
>>> https://github.com/apache/geode/pull/787
>>> 
>>> What do you think?
>>> 
>>> -Dan
>> 
>>

Re: [DISCUSS] Framework for concurrency tests

2017-09-15 Thread Jason Huynh

+1 to Dan's Changes but also +1 to Galen's suggestion.  JPF looks like it
might take a bit to run all the different states even for a small
interleaving of code (maybe we can tune/configure it though).  Or we can
mark these as a different category and not run as a "UnitTest"

On Fri, Sep 15, 2017 at 2:22 PM Jacob Barrett  wrote:

> What? You don’t think Travis can run these fast?
>
> > On Sep 15, 2017, at 2:07 PM, Galen O'Sullivan 
> wrote:
> >
> > +1 This is great! I'll take a look at your PR when I get the time.
> >
> > We may want to think carefully about how often we run these tests,
> because
> > unlike regular unit tests, they will take forever to run.
> >
> > On Fri, Sep 15, 2017 at 1:42 PM, Michael William Dodge <
> mdo...@pivotal.io>
> > wrote:
> >
> >> +1 for unit tests for multithreaded code.
> >>
> >> High fives to Dan.
> >>
> >> Sarge
> >>
> >>> On 15 Sep, 2017, at 12:08, Dan Smith  wrote:
> >>>
> >>> Hi Geode devs,
> >>>
> >>> I've been messing around with an open source tool called Java
> >>> Pathfinder for writing tests of multithreaded code. Java Pathfinder is
> >>> a special JVM which among other things tries to execute your code
> >>> using all possible thread interleavings.
> >>>
> >>> I'd like to propose two things:
> >>>
> >>> 1) We introduce a framework for writing unit tests of code that is
> >>> supposed to be thread safe. This framework should let a developer
> >>> easily write a test with multiple things going on in parallel. The
> >>> framework can then take that code and try to run it with different
> >>> thread interleavings.
> >>>
> >>> Here's an example of what this could look like:
> >>>
> >>> @RunWith(ConcurrentTestRunner.class)
> >>> public class AtomicIntegerTest {
> >>>
> >>> @Test
> >>> public void parallelIncrementReturns2(ParallelExecutor executor)
> >>> throws ExecutionException, InterruptedException {
> >>>   AtomicInteger atomicInteger = new AtomicInteger();
> >>>   executor.inParallel(() -> atomicInteger.incrementAndGet());
> >>>   executor.inParallel(() -> atomicInteger.incrementAndGet());
> >>>   executor.execute();
> >>>   assertEquals(2, atomicInteger.get());
> >>> }
> >>>
> >>>
> >>> 2) We implement this framework initially using Java Pathfinder, but
> >>> allow for other methods of testing the code to be plugged in for
> >>> example just running the test in the loop. Java pathfinder is cool
> >>> because it can run the code with different interleavings but it does
> >>> have some serious limitations.
> >>>
> >>> I've put together some code for this proposal which is available in
> >>> this github PR:
> >>>
> >>> https://github.com/apache/geode/pull/787
> >>>
> >>> What do you think?
> >>>
> >>> -Dan
> >>
> >>
>

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #679 was SUCCESSFUL (with 2042 tests)

2017-09-15 Thread Spring CI


---
Spring Data GemFire > Nightly-ApacheGeode > #679 was successful.
---
Scheduled
2044 tests in total.

https://build.spring.io/browse/SGF-NAG-679/





--
This message is automatically generated by Atlassian Bamboo

Re: New client/server protocol - seeking feedback

2017-09-15 Thread Dan Smith

What's the best place to look for more details on the specific protocol for
the v1.0 messages? The other pages on https://cwiki.apache.org/
confluence/display/GEODE/New+Client+Server+Protocol? Or directly in the
code somewhere?

-Dan

On Fri, Sep 15, 2017 at 11:15 AM, Brian Baynes  wrote:

> Greetings, friends of Geode.
>
> Work has been progressing on the new client/server protocol for Geode and
> we're approaching a GA v1.0.  Completed work/features include put/get,
> putAll, getAll, remove, one-way SSL, authentication and authorization, and
> support for primitive types and JSON documents as values (saved in PDX on
> the server).
>
> Invite you to review the road map toward GA v1.0, the features proposed for
> post-v1.0, and give us your feedback!  (Directly in Confluence or here on
> dev@geode.apache.org)
>
> New Client/Server Protocol - Road Map, Proposed
> 
>
>
> Thanks for your input,
>
> Brian
>

Re: New client/server protocol - seeking feedback

2017-09-15 Thread Brian Baynes

You can find them in the code, but we'll be providing better documentation
on the messages shortly. The proto files have the message definitions and
they're pretty straightforward, but we'll have a more user-friendly
write-up soon.

On Sep 15, 2017 5:27 PM, "Dan Smith"  wrote:

What's the best place to look for more details on the specific protocol for
the v1.0 messages? The other pages on https://cwiki.apache.org/
confluence/display/GEODE/New+Client+Server+Protocol? Or directly in the
code somewhere?

-Dan

On Fri, Sep 15, 2017 at 11:15 AM, Brian Baynes  wrote:

> Greetings, friends of Geode.
>
> Work has been progressing on the new client/server protocol for Geode and
> we're approaching a GA v1.0.  Completed work/features include put/get,
> putAll, getAll, remove, one-way SSL, authentication and authorization, and
> support for primitive types and JSON documents as values (saved in PDX on
> the server).
>
> Invite you to review the road map toward GA v1.0, the features proposed
for
> post-v1.0, and give us your feedback!  (Directly in Confluence or here on
> dev@geode.apache.org)
>
> New Client/Server Protocol - Road Map, Proposed
> 
>
>
> Thanks for your input,
>
> Brian
>

38 matches

Mail list logo