Help With ClusterStartupRuleCanSpecifyOlderVersionsDUnitTest

[Ju@N]

Hello all,

Working through GEODE-5739
 I've modified
*ServerStarterRule* and *LocatorStarterRule* to use the public
*ServerLauncher* and *LocatorLauncher* classes respectively, instead of the
internal APIs to start up the members. As part of the changes, I've
modified the accessibility of some methods to be *public* instead of
*private/protected* (like *getLogFile* so member's logs are redirected to
console) and made use of some methods not present at all in older versions
(like *setDeletePidFileOnStop*). This causes the test of the subject to fail

when trying to invoke those if the user requested  using older versions of
Geode (*IllegalAccessError*, *NoSuchMethodError*, etc.)...
Considering that these changes don't actually break the backward
compatibility for the product itself, what would be the best solution here?:

   - Discard the changes and leave the rules as they currently are (rules
   will continue to use internal APIs instead of launchers).
   - Extend the current rules, deprecate the old ones, and tweak the
   *ClusterStartupRule* to choose the correct rule instance when starting
   members based on the requested version.
   - Modify the rules to use the old approach (internal APIs) whenever they
   detect that the user has requested to start a member using an older version
   of the product.
   - Other?.

Best regards.

-- 
Ju@N

Re: [Discuss] Where should simple classes for tests belong?

[Jens Deppe]

For testing functionality like 'gfsh deploy' one has to be aware of
classloading issues. Specifically, when writing dunit tests, classes are
compiled from java resource files so that they are guaranteed not to be on
the classpath during testing. This is less of an issue for gfsh acceptance
tests where there is more control over the classpath and what is launched
as part of the test.

For the dunit deploy testing scenario (and any similar) I would prefer to
see the java test resource as exactly that - a package-equivalent resource
file that is explicitly compiled as part of the test. It makes the test
clearer and doesn't leave anyone wondering where/how resources are built or
injected.

--Jens

On Fri, Oct 12, 2018 at 4:22 PM Kirk Lund  wrote:

> I usually create a private static inner class and keep it as small as
> possible at the end of the test class that uses it. Some of these these
> such as MyCacheListener can be replaced by spy(CacheListener.class) and I
> always try to convert these where possible.
>
> If it's a simple class that only tests in one package will use (like a
> custom AssertJ class that nothing outside that package will use) then I
> make it package-protected and stick it in the same package and srcSet as
> the tests that will use it.
>
> I think something like MonthBasedPartitionResolver is simple enough that it
> should exist within the same src set and in the same package as the test or
> a few tests that use it. And if another test in some other src set or in
> another package needs something similar, then too bad, that other test
> should create its own simple small class that is similar. This sort of
> class does NOT belong in a src/main of some module like geode-dunit. Better
> yet make it a static inner class in every test that uses it. I know some
> people object to this, but here's why...
>
> If I want to modify one test that uses MonthBasedPartitionResolver and this
> modification involves changing MonthBasedPartitionResolver (which remember
> it's a simple class) then if each test has its own copy I won't risk
> breaking other tests. When I'm fixing up, cleaning up and refactoring
> tests, the biggest nightmares I run into are shared utility classes such
> that if I try to cleanup one test, I can't -- I have to clean up
> potentially dozens of tests because they linked some some stupid utility
> class because "code sharing" is great. MonthBasedPartitionResolver is a
> small example, there are plenty of other examples that are true
> monstrosities.
>
> Sharing code between tests is NOT great unless you devise a Rule or custom
> AssertJ Assertion that is truly very reusable across many tests -- and only
> in this case it belongs in geode-junit or geode-dunit where it should live
> in a src/main and there should be tests for it under src/test or
> src/integrationTest. I also highly recommend that you keep shared testing
> objects like this small in scope with a single high-level responsibility.
> And please, never create something that simply creates a new improved API
> so that you don't have to use the Geode API -- this just promotes leaving
> crappy product APIs in place without changing them because by using some
> fancy testing API, we don't feel the pain that Users experience with the
> product APIs.
>
> On Fri, Oct 12, 2018 at 3:32 PM, Patrick Rhomberg 
> wrote:
>
> > Hello, all!
> >
> >   There are a number of classes that require some number of "toy" classes
> > as part of their testing framework, e.g., to be used as custom data
> > containers.  Many of these classes involve testing or use of the `deploy
> > jar` command, and so are packaged into jars for this purpose.
> >   In an effort to promote good coding habits and, as importantly,
> consensus
> > throughout this community and our codebase, I would like to ask which of
> > these are considered the preferred method to maintain these additional
> > classes.
> >   I realize a priori that none of these options will be applicable in all
> > cases, but am curious of how the prioritization of these options are
> > ordered among you all.
> >
> > -- Class definition --
> > For definition of the classes consumed, we observe all of the following
> in
> > the Geode codebase.
> >
> > Option C-1:  Toy classes are defined as a proper class in a reasonable
> > package.
> > -- suboption (a): The class is defined in the module in which it is
> > consumed, as a resource.  [1]
> > -- suboption (b): The class is defined in the module in which it is
> > consumed, as a neighboring file.  [2]
> > -- suboption (c): The class is defined in geode-dunit or geode-junit.
> [3]
> >
> > Option C-2:  Toy classes are defined as inner classes of the test class.
> > [4]
> >
> > Option C-3:  Sufficiently small toy classes are defined as raw String
> > fields in the test class that consumed it and is compiled by the test JVM
> > via the JavaCompiler class.  [5]
> >
> > -- Resource exposure --
> > For those tests that require classes placed in Jars f

Re: [Discuss] Where should simple classes for tests belong?

[Jinmei Liao]

+1 to what Jens said. The test jar is very specific to the test itself.
Does not need to be in the src at all. It should disappear when the test
finishes. There is no need for it to be in the build process at all.
Producing it inside the test itself is a much cleaner way for the
developer/maintainer of the code to know what the intent of the test/code
is for rather having to hunt around to find out where the test code lives.

On Mon, Oct 15, 2018 at 8:38 AM Jens Deppe  wrote:

> For testing functionality like 'gfsh deploy' one has to be aware of
> classloading issues. Specifically, when writing dunit tests, classes are
> compiled from java resource files so that they are guaranteed not to be on
> the classpath during testing. This is less of an issue for gfsh acceptance
> tests where there is more control over the classpath and what is launched
> as part of the test.
>
> For the dunit deploy testing scenario (and any similar) I would prefer to
> see the java test resource as exactly that - a package-equivalent resource
> file that is explicitly compiled as part of the test. It makes the test
> clearer and doesn't leave anyone wondering where/how resources are built or
> injected.
>
> --Jens
>
> On Fri, Oct 12, 2018 at 4:22 PM Kirk Lund  wrote:
>
> > I usually create a private static inner class and keep it as small as
> > possible at the end of the test class that uses it. Some of these these
> > such as MyCacheListener can be replaced by spy(CacheListener.class) and I
> > always try to convert these where possible.
> >
> > If it's a simple class that only tests in one package will use (like a
> > custom AssertJ class that nothing outside that package will use) then I
> > make it package-protected and stick it in the same package and srcSet as
> > the tests that will use it.
> >
> > I think something like MonthBasedPartitionResolver is simple enough that
> it
> > should exist within the same src set and in the same package as the test
> or
> > a few tests that use it. And if another test in some other src set or in
> > another package needs something similar, then too bad, that other test
> > should create its own simple small class that is similar. This sort of
> > class does NOT belong in a src/main of some module like geode-dunit.
> Better
> > yet make it a static inner class in every test that uses it. I know some
> > people object to this, but here's why...
> >
> > If I want to modify one test that uses MonthBasedPartitionResolver and
> this
> > modification involves changing MonthBasedPartitionResolver (which
> remember
> > it's a simple class) then if each test has its own copy I won't risk
> > breaking other tests. When I'm fixing up, cleaning up and refactoring
> > tests, the biggest nightmares I run into are shared utility classes such
> > that if I try to cleanup one test, I can't -- I have to clean up
> > potentially dozens of tests because they linked some some stupid utility
> > class because "code sharing" is great. MonthBasedPartitionResolver is a
> > small example, there are plenty of other examples that are true
> > monstrosities.
> >
> > Sharing code between tests is NOT great unless you devise a Rule or
> custom
> > AssertJ Assertion that is truly very reusable across many tests -- and
> only
> > in this case it belongs in geode-junit or geode-dunit where it should
> live
> > in a src/main and there should be tests for it under src/test or
> > src/integrationTest. I also highly recommend that you keep shared testing
> > objects like this small in scope with a single high-level responsibility.
> > And please, never create something that simply creates a new improved API
> > so that you don't have to use the Geode API -- this just promotes leaving
> > crappy product APIs in place without changing them because by using some
> > fancy testing API, we don't feel the pain that Users experience with the
> > product APIs.
> >
> > On Fri, Oct 12, 2018 at 3:32 PM, Patrick Rhomberg 
> > wrote:
> >
> > > Hello, all!
> > >
> > >   There are a number of classes that require some number of "toy"
> classes
> > > as part of their testing framework, e.g., to be used as custom data
> > > containers.  Many of these classes involve testing or use of the
> `deploy
> > > jar` command, and so are packaged into jars for this purpose.
> > >   In an effort to promote good coding habits and, as importantly,
> > consensus
> > > throughout this community and our codebase, I would like to ask which
> of
> > > these are considered the preferred method to maintain these additional
> > > classes.
> > >   I realize a priori that none of these options will be applicable in
> all
> > > cases, but am curious of how the prioritization of these options are
> > > ordered among you all.
> > >
> > > -- Class definition --
> > > For definition of the classes consumed, we observe all of the following
> > in
> > > the Geode codebase.
> > >
> > > Option C-1:  Toy classes are defined as a proper class in a reasonable
> > > package.
>

Re: [Discuss] Where should simple classes for tests belong?

[Udo Kohlmeyer]

+1 to what Jens & Jinmei & Kirk have said. It is far easier to have a 
local definition of the "toy" classes.


Yes, it makes the code base "larger" and most good editors would flag it 
as duplicate code, BUT it is about localization and making sure that one 
can easily see what classes are part of a test and what its definition is.


In addition to this, having a set of "toy" definition lying around 
means, that as more tests use these classes, the test scenario 
inevitably starts seeping into the toy classes, eventually causing 
copies of these classes to be made in any case, as the original does not 
meet the requirement of the new test.


--Udo


On 10/15/18 09:40, Jinmei Liao wrote:

+1 to what Jens said. The test jar is very specific to the test itself.
Does not need to be in the src at all. It should disappear when the test
finishes. There is no need for it to be in the build process at all.
Producing it inside the test itself is a much cleaner way for the
developer/maintainer of the code to know what the intent of the test/code
is for rather having to hunt around to find out where the test code lives.

On Mon, Oct 15, 2018 at 8:38 AM Jens Deppe  wrote:


For testing functionality like 'gfsh deploy' one has to be aware of
classloading issues. Specifically, when writing dunit tests, classes are
compiled from java resource files so that they are guaranteed not to be on
the classpath during testing. This is less of an issue for gfsh acceptance
tests where there is more control over the classpath and what is launched
as part of the test.

For the dunit deploy testing scenario (and any similar) I would prefer to
see the java test resource as exactly that - a package-equivalent resource
file that is explicitly compiled as part of the test. It makes the test
clearer and doesn't leave anyone wondering where/how resources are built or
injected.

--Jens

On Fri, Oct 12, 2018 at 4:22 PM Kirk Lund  wrote:


I usually create a private static inner class and keep it as small as
possible at the end of the test class that uses it. Some of these these
such as MyCacheListener can be replaced by spy(CacheListener.class) and I
always try to convert these where possible.

If it's a simple class that only tests in one package will use (like a
custom AssertJ class that nothing outside that package will use) then I
make it package-protected and stick it in the same package and srcSet as
the tests that will use it.

I think something like MonthBasedPartitionResolver is simple enough that

it

should exist within the same src set and in the same package as the test

or

a few tests that use it. And if another test in some other src set or in
another package needs something similar, then too bad, that other test
should create its own simple small class that is similar. This sort of
class does NOT belong in a src/main of some module like geode-dunit.

Better

yet make it a static inner class in every test that uses it. I know some
people object to this, but here's why...

If I want to modify one test that uses MonthBasedPartitionResolver and

this

modification involves changing MonthBasedPartitionResolver (which

remember

it's a simple class) then if each test has its own copy I won't risk
breaking other tests. When I'm fixing up, cleaning up and refactoring
tests, the biggest nightmares I run into are shared utility classes such
that if I try to cleanup one test, I can't -- I have to clean up
potentially dozens of tests because they linked some some stupid utility
class because "code sharing" is great. MonthBasedPartitionResolver is a
small example, there are plenty of other examples that are true
monstrosities.

Sharing code between tests is NOT great unless you devise a Rule or

custom

AssertJ Assertion that is truly very reusable across many tests -- and

only

in this case it belongs in geode-junit or geode-dunit where it should

live

in a src/main and there should be tests for it under src/test or
src/integrationTest. I also highly recommend that you keep shared testing
objects like this small in scope with a single high-level responsibility.
And please, never create something that simply creates a new improved API
so that you don't have to use the Geode API -- this just promotes leaving
crappy product APIs in place without changing them because by using some
fancy testing API, we don't feel the pain that Users experience with the
product APIs.

On Fri, Oct 12, 2018 at 3:32 PM, Patrick Rhomberg 
wrote:


Hello, all!

   There are a number of classes that require some number of "toy"

classes

as part of their testing framework, e.g., to be used as custom data
containers.  Many of these classes involve testing or use of the

`deploy

jar` command, and so are packaged into jars for this purpose.
   In an effort to promote good coding habits and, as importantly,

consensus

throughout this community and our codebase, I would like to ask which

of

these are considered the preferred method to maintain these additional
cl

Re: Queries on key fields

[anjana_nair]

Jason,

We are using the Map feature of GemFire and update throws CommitComflict
exceptions when the same key is updated from multiple places at the same
time, how can this be resolved ? This happens only at high contention
though. We are  retrying , but still even after a couple of retries it does
not get successful.

What can be done ?



--
Sent from: http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/

Re: Queries on key fields

[Jason Huynh]

Hello Siby,

I think we should keep this conversation based on Geode if possible.  I
think GemFire has their own support channels that you might be able to
contact for help specific to GemFire...
As for the commit conflicts, I assume you are using transactions?  Also, I
am not aware of a specific Map feature, would you be able to provide the
api call and maybe a bit more information about the usage?

Is this related to index/queries at this point or should this be spun into
another thread?




On Mon, Oct 15, 2018 at 12:22 PM anjana_nair 
wrote:

> Jason,
>
> We are using the Map feature of GemFire and update throws CommitComflict
> exceptions when the same key is updated from multiple places at the same
> time, how can this be resolved ? This happens only at high contention
> though. We are  retrying , but still even after a couple of retries it does
> not get successful.
>
> What can be done ?
>
>
>
> --
> Sent from:
> http://apache-geode-incubating-developers-forum.70738.x6.nabble.com/
>

Pipeline names changing shortly

[Sean Goller]

In order to standardize CI deployment and to improve quality of life around
running pipelines for forks, we've merged a number of changes to the ci/
directory which will create new pipelines for the develop branch. If you
have direct links to specific pipelines (specifically develop ones) those
links will eventually break. Once we've deployed the new set of pipelines,
(real soon now) we will pause the legacy pipelines and leave them up for a
week or so.

-Sean.

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #1071 was SUCCESSFUL (with 2456 tests). Change made by Mark Paluch.

[Spring CI]

---
Spring Data GemFire > Nightly-ApacheGeode > #1071 was successful.
---
Scheduled with changes by Mark Paluch.
2458 tests in total.

https://build.spring.io/browse/SGF-NAG-1071/




--
Code Changes
--
Mark Paluch (6534c0b26f167c0fc68bf5a5894de1743bd1c9bd):

>DATAGEODE-147 - Updated changelog.



--
This message is automatically generated by Atlassian Bamboo

Re: Pipeline names changing shortly

[Sean Goller]

New pipelines for develop (apache-develop-main) and PR (apache-develop-pr)
are up. We'll bring up the rest of the pipelines (metrics, examples, etc.)
tomorrow.

On Mon, Oct 15, 2018 at 3:03 PM Sean Goller  wrote:

> In order to standardize CI deployment and to improve quality of life
> around running pipelines for forks, we've merged a number of changes to the
> ci/ directory which will create new pipelines for the develop branch. If
> you have direct links to specific pipelines (specifically develop ones)
> those links will eventually break. Once we've deployed the new set of
> pipelines, (real soon now) we will pause the legacy pipelines and leave
> them up for a week or so.
>
> -Sean.
>