[PROPOSAL]: Adding declarable support in Gfsh command
Currently, when you want to specify a call-back in gfsh command option, you
can only pass in the class name, e.g.:
create region --name=regionA --type=PARTITION --cache-loader=my.CacheLoader
But these callbacks all implements Declarable (CacheLoader, CacheWriter,
CacheListener, CustomExpiry etc.), i.e they can initialized with extra
properties that can be set when they are declared in cache.xml, but
currently, our gfsh command doesn't support that.
We are proposing to add the support to configure Declarables in gfsh
commands by adding json strings at the end of the class name. like this:
create region --name=regionA --type=PARTITION
--cache-loader=my.CacheLoader?{"key":"value,"key2":"value2"}
(of course, if you don't need to configure your Declarable, you can still
only specify a className as before).
Comments/thoughts?
--
Cheers
Jinmei
Re: [PROPOSAL]: Adding declarable support in Gfsh command
This also avoids the other option of implementing this by having associated
'params' options for each option which can take a Declarable, thus reducing
the proliferation of options - in particular for 'create region'.
i.e. --cache-listener AND --cache-listener-params.
Further, this json parameter would not allow for arbitrary json but would
be restricted to a simple key/value mapping so that there would be a direct
translation to Declarable parameters in the cache.xml.
--Jens
On Fri, Jan 26, 2018 at 8:07 AM, Jinmei Liao wrote:
> Currently, when you want to specify a call-back in gfsh command option, you
> can only pass in the class name, e.g.:
>
> create region --name=regionA --type=PARTITION --cache-loader=my.CacheLoader
>
> But these callbacks all implements Declarable (CacheLoader, CacheWriter,
> CacheListener, CustomExpiry etc.), i.e they can initialized with extra
> properties that can be set when they are declared in cache.xml, but
> currently, our gfsh command doesn't support that.
>
> We are proposing to add the support to configure Declarables in gfsh
> commands by adding json strings at the end of the class name. like this:
>
> create region --name=regionA --type=PARTITION
> --cache-loader=my.CacheLoader?{"key":"value,"key2":"value2"}
>
> (of course, if you don't need to configure your Declarable, you can still
> only specify a className as before).
>
> Comments/thoughts?
>
> --
> Cheers
>
> Jinmei
>
jacocoTestReport
gradle/code-analysis.gradle contains configuration for the jacoco plugin, but if I attempt to execute the standard jacoco task jacocoTestReport, our build just fails and says it doesn't know what that is. How do we run jacoco on the command line? I tried this: ./gradlew geode-core:test --tests *RegionMapDestroyTest jacocoTestReport Thanks, Kirk
Re: jacocoTestReport
Specify `-PcodeCoverage=true` on your build command. > On Jan 26, 2018, at 9:42 AM, Kirk Lund wrote: > > gradle/code-analysis.gradle contains configuration for the jacoco plugin, > but if I attempt to execute the standard jacoco task jacocoTestReport, our > build just fails and says it doesn't know what that is. > > How do we run jacoco on the command line? > > I tried this: > > ./gradlew geode-core:test --tests *RegionMapDestroyTest jacocoTestReport > > Thanks, > Kirk
Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found
Jake, the requirement seems like users doesn't have to set the read-serialized flag; system needs to take care of that based on availability of domain-class in classpath. -Anil. On Thu, Jan 25, 2018 at 2:01 PM, Jacob Barrett wrote: > It is no different really than how the Java client works. If on a Java > client I put an object who's class implements PdxSerializable the object is > serialized as PDX. If form the Java client I call a function with that > object as an argument then the server will try to deserialize the PDX to > the named domain class. If the domain class has not been deployed in the > class path then a ClassNotFoundException is throw. So if from the .NET, > C++, whatever language, I have a domain class that implements some language > specific version of PdxSerializable (IPdxSerializable in .NET) the object > is sent as PDX. Since I haven't created a corresponding Java domain object > and deployed it then I will get a ClassNotFoundException. Yes, I could > write a Java domain object and deploy it, but I shouldn't have to. The > crappy work around we have for this is the pdx read serializable flag and > with it set to true my non-Java client can run server side functions and > queries without also having to write and deploy matching Java domain > objects. The downside is that it global and changes all the behavior of PDX > serialized objects. > > > On Thu, Jan 25, 2018 at 1:43 PM Galen O'Sullivan > wrote: > > > Hi Jake, > > > > I'm not really familiar with how the .NET client works - from what you're > > saying, it sounds like there is a way to serialize PDX objects without > > having access to a domain class? Can the .NET client tag PDX-serialized > > objects with the name of the domain class that they represent, so that > the > > Java classes will get the objects they expect? > > > > Thanks, > > Galen > > > > On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett > > wrote: > > > > > Anil, this wouldn't fix a function execution where a parameter sent to > > the > > > function is a PDX serialized object. The function would be invoked on > the > > > server with the current global setting and file to deserialize the PDX > to > > > domain object. This is why the ask is for it to fall back to > PdxInstance > > if > > > the domain object is not deployed. > > > > > > -Jake > > > > > > On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade > > > > wrote: > > > > > > > Internally, there is an option to override read-serialized flag (to > > > true); > > > > the query engine and other components uses this to keep the data in > > > > serialized form and work with PdxInstance... > > > > > > > > public static void setPdxReadSerialized(Cache cache, boolean > > > > readSerialized); > > > > > > > > We had discussed, making this as a public api...so that any thread > that > > > can > > > > work on PdxInstance can take advantage of it... > > > > > > > > -Anil. > > > > > > > > > > > > On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett > > > > wrote: > > > > > > > > > Bruce, the flag only applies to values serialized with PDX, > > > > > DataSerializable objects are not effected by this property. > > > > > > > > > > I think there is some real value here as a stop gap until we have a > > > > better > > > > > solution in Geode 2 where the user can have a per request context > > that > > > > > specifies what return type they would like. Consider the user that > > has > > > an > > > > > existing application that uses domain objects but wants to deploy > > > another > > > > > application that doesn't to the same Geode cluster. The only option > > is > > > to > > > > > either have all PDX deserialize to domain objects or all returned > as > > > > > PdxInstance. One of the two applications will not work without > > > > > modification. Changing the behavior described by Addison splits the > > > > > difference. If the application is, like it is by default, configure > > to > > > > > deserialize PDX to the domain object but the domain object is not > > > > deployed > > > > > it will now give back the PDX instance rather than failing. > > > > > > > > > > An explicit use case is user that has both a Java and .NET > > application. > > > > The > > > > > .NET application does not have any Java domain objects to deploy to > > the > > > > > server but does want to query or run server side functions. The > Java > > > > > application has deployed the domain objects to the server and > > > distributed > > > > > functions are written expecting those domain objects on the server. > > The > > > > > user would have to create Java domain objects for the .NET > > application > > > or > > > > > modify their Java application to expect PdxInstance. > > > > > > > > > > > > > > > -Jake > > > > > > > > > > > > > > > On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt < > > > bschucha...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > I've found the current read-serialized property to be pretty > > useless. > > > >
Re: [PROPOSAL]: Adding declarable support in Gfsh command
+1
Currently passing properties is supported with AsyncEventListener with
"-listener-param" option, we need make it consistent across the call-backs.
-Anil.
On Fri, Jan 26, 2018 at 8:43 AM, Jens Deppe wrote:
> This also avoids the other option of implementing this by having associated
> 'params' options for each option which can take a Declarable, thus reducing
> the proliferation of options - in particular for 'create region'.
>
> i.e. --cache-listener AND --cache-listener-params.
>
> Further, this json parameter would not allow for arbitrary json but would
> be restricted to a simple key/value mapping so that there would be a direct
> translation to Declarable parameters in the cache.xml.
>
> --Jens
>
> On Fri, Jan 26, 2018 at 8:07 AM, Jinmei Liao wrote:
>
> > Currently, when you want to specify a call-back in gfsh command option,
> you
> > can only pass in the class name, e.g.:
> >
> > create region --name=regionA --type=PARTITION
> --cache-loader=my.CacheLoader
> >
> > But these callbacks all implements Declarable (CacheLoader, CacheWriter,
> > CacheListener, CustomExpiry etc.), i.e they can initialized with extra
> > properties that can be set when they are declared in cache.xml, but
> > currently, our gfsh command doesn't support that.
> >
> > We are proposing to add the support to configure Declarables in gfsh
> > commands by adding json strings at the end of the class name. like this:
> >
> > create region --name=regionA --type=PARTITION
> > --cache-loader=my.CacheLoader?{"key":"value,"key2":"value2"}
> >
> > (of course, if you don't need to configure your Declarable, you can still
> > only specify a className as before).
> >
> > Comments/thoughts?
> >
> > --
> > Cheers
> >
> > Jinmei
> >
>
Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found
Correct, that is what this ticket is asking for. > On Jan 26, 2018, at 10:09 AM, Anilkumar Gingade wrote: > > Jake, the requirement seems like users doesn't have to set the > read-serialized flag; system needs to take care of that based on > availability of domain-class in classpath. > > -Anil. > > > >> On Thu, Jan 25, 2018 at 2:01 PM, Jacob Barrett wrote: >> >> It is no different really than how the Java client works. If on a Java >> client I put an object who's class implements PdxSerializable the object is >> serialized as PDX. If form the Java client I call a function with that >> object as an argument then the server will try to deserialize the PDX to >> the named domain class. If the domain class has not been deployed in the >> class path then a ClassNotFoundException is throw. So if from the .NET, >> C++, whatever language, I have a domain class that implements some language >> specific version of PdxSerializable (IPdxSerializable in .NET) the object >> is sent as PDX. Since I haven't created a corresponding Java domain object >> and deployed it then I will get a ClassNotFoundException. Yes, I could >> write a Java domain object and deploy it, but I shouldn't have to. The >> crappy work around we have for this is the pdx read serializable flag and >> with it set to true my non-Java client can run server side functions and >> queries without also having to write and deploy matching Java domain >> objects. The downside is that it global and changes all the behavior of PDX >> serialized objects. >> >> >> On Thu, Jan 25, 2018 at 1:43 PM Galen O'Sullivan >> wrote: >> >>> Hi Jake, >>> >>> I'm not really familiar with how the .NET client works - from what you're >>> saying, it sounds like there is a way to serialize PDX objects without >>> having access to a domain class? Can the .NET client tag PDX-serialized >>> objects with the name of the domain class that they represent, so that >> the >>> Java classes will get the objects they expect? >>> >>> Thanks, >>> Galen >>> >>> On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett >>> wrote: >>> Anil, this wouldn't fix a function execution where a parameter sent to >>> the function is a PDX serialized object. The function would be invoked on >> the server with the current global setting and file to deserialize the PDX >> to domain object. This is why the ask is for it to fall back to >> PdxInstance >>> if the domain object is not deployed. -Jake On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade >> wrote: > Internally, there is an option to override read-serialized flag (to true); > the query engine and other components uses this to keep the data in > serialized form and work with PdxInstance... > > public static void setPdxReadSerialized(Cache cache, boolean > readSerialized); > > We had discussed, making this as a public api...so that any thread >> that can > work on PdxInstance can take advantage of it... > > -Anil. > > > On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett > wrote: > >> Bruce, the flag only applies to values serialized with PDX, >> DataSerializable objects are not effected by this property. >> >> I think there is some real value here as a stop gap until we have a > better >> solution in Geode 2 where the user can have a per request context >>> that >> specifies what return type they would like. Consider the user that >>> has an >> existing application that uses domain objects but wants to deploy another >> application that doesn't to the same Geode cluster. The only option >>> is to >> either have all PDX deserialize to domain objects or all returned >> as >> PdxInstance. One of the two applications will not work without >> modification. Changing the behavior described by Addison splits the >> difference. If the application is, like it is by default, configure >>> to >> deserialize PDX to the domain object but the domain object is not > deployed >> it will now give back the PDX instance rather than failing. >> >> An explicit use case is user that has both a Java and .NET >>> application. > The >> .NET application does not have any Java domain objects to deploy to >>> the >> server but does want to query or run server side functions. The >> Java >> application has deployed the domain objects to the server and distributed >> functions are written expecting those domain objects on the server. >>> The >> user would have to create Java domain objects for the .NET >>> application or >> modify their Java application to expect PdxInstance. >> >> >> -Jake >> >> >> On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt < bschucha...@apache.org >> >> wrote: >> >>> +1 >>> >>> I've found the current read-serialized property to be pr
Re: [PROPOSAL]: Adding declarable support in Gfsh command
This would solve the problem of specifying the parameters for a Declarable,
but if you provided support for any valid json, you could cover other
situations as well, including those with more complicated and possibly
nested configuration. If we would ever support such scenarios in the
future, I assume that we would want a generic solution that would cover all
configuration. Is this something that we anticipate needing and if so, how
would the current proposal cover such situations?
On Fri, Jan 26, 2018 at 8:43 AM, Jens Deppe wrote:
> This also avoids the other option of implementing this by having associated
> 'params' options for each option which can take a Declarable, thus reducing
> the proliferation of options - in particular for 'create region'.
>
> i.e. --cache-listener AND --cache-listener-params.
>
> Further, this json parameter would not allow for arbitrary json but would
> be restricted to a simple key/value mapping so that there would be a direct
> translation to Declarable parameters in the cache.xml.
>
> --Jens
>
> On Fri, Jan 26, 2018 at 8:07 AM, Jinmei Liao wrote:
>
> > Currently, when you want to specify a call-back in gfsh command option,
> you
> > can only pass in the class name, e.g.:
> >
> > create region --name=regionA --type=PARTITION
> --cache-loader=my.CacheLoader
> >
> > But these callbacks all implements Declarable (CacheLoader, CacheWriter,
> > CacheListener, CustomExpiry etc.), i.e they can initialized with extra
> > properties that can be set when they are declared in cache.xml, but
> > currently, our gfsh command doesn't support that.
> >
> > We are proposing to add the support to configure Declarables in gfsh
> > commands by adding json strings at the end of the class name. like this:
> >
> > create region --name=regionA --type=PARTITION
> > --cache-loader=my.CacheLoader?{"key":"value,"key2":"value2"}
> >
> > (of course, if you don't need to configure your Declarable, you can still
> > only specify a className as before).
> >
> > Comments/thoughts?
> >
> > --
> > Cheers
> >
> > Jinmei
> >
>
Re: [PROPOSAL]: Adding declarable support in Gfsh command
+1 for ability to pass in simple key-value pairs.
Would it be possible to loose the "?" between the class name and the json
though?
Also, since we are essentially trying to pass a java.util.Properties object
to the Declarable, would it be better if we ask the users to pass-in a
string representation of the properties object? something like:
--cache-loader=my.CacheLoader{key2=value2, key1=value1}
A parenthesis around the values may also help in conveying the intention:
--cache-loader=my.CacheLoader({key2=value2, key1=value1})
On Fri, Jan 26, 2018 at 10:24 AM Nick Reich wrote:
> This would solve the problem of specifying the parameters for a Declarable,
> but if you provided support for any valid json, you could cover other
> situations as well, including those with more complicated and possibly
> nested configuration. If we would ever support such scenarios in the
> future, I assume that we would want a generic solution that would cover all
> configuration. Is this something that we anticipate needing and if so, how
> would the current proposal cover such situations?
>
> On Fri, Jan 26, 2018 at 8:43 AM, Jens Deppe wrote:
>
> > This also avoids the other option of implementing this by having
> associated
> > 'params' options for each option which can take a Declarable, thus
> reducing
> > the proliferation of options - in particular for 'create region'.
> >
> > i.e. --cache-listener AND --cache-listener-params.
> >
> > Further, this json parameter would not allow for arbitrary json but would
> > be restricted to a simple key/value mapping so that there would be a
> direct
> > translation to Declarable parameters in the cache.xml.
> >
> > --Jens
> >
> > On Fri, Jan 26, 2018 at 8:07 AM, Jinmei Liao wrote:
> >
> > > Currently, when you want to specify a call-back in gfsh command option,
> > you
> > > can only pass in the class name, e.g.:
> > >
> > > create region --name=regionA --type=PARTITION
> > --cache-loader=my.CacheLoader
> > >
> > > But these callbacks all implements Declarable (CacheLoader,
> CacheWriter,
> > > CacheListener, CustomExpiry etc.), i.e they can initialized with extra
> > > properties that can be set when they are declared in cache.xml, but
> > > currently, our gfsh command doesn't support that.
> > >
> > > We are proposing to add the support to configure Declarables in gfsh
> > > commands by adding json strings at the end of the class name. like
> this:
> > >
> > > create region --name=regionA --type=PARTITION
> > > --cache-loader=my.CacheLoader?{"key":"value,"key2":"value2"}
> > >
> > > (of course, if you don't need to configure your Declarable, you can
> still
> > > only specify a className as before).
> > >
> > > Comments/thoughts?
> > >
> > > --
> > > Cheers
> > >
> > > Jinmei
> > >
> >
>
Re: [PROPOSAL]: Adding declarable support in Gfsh command
Yes, we are debating the necessity of the "?".
-1 on the non-json syntax though. we don't want to implement a new parser
ourselves to gather the properties.
As to Nick's point about a generic solution that would cover initializing
not just Declarable, yes, the syntax itself does not lock down to the class
being a Declarable. However you want to configure the object using the
parsed Property, it's up to how we want to interpret this property. Our
current implementation only support using this property if the className
specified is a Declarable, but we can enhance it to support other types as
well.
On Fri, Jan 26, 2018 at 12:15 PM, Swapnil Bawaskar
wrote:
> +1 for ability to pass in simple key-value pairs.
> Would it be possible to loose the "?" between the class name and the json
> though?
>
> Also, since we are essentially trying to pass a java.util.Properties object
> to the Declarable, would it be better if we ask the users to pass-in a
> string representation of the properties object? something like:
> --cache-loader=my.CacheLoader{key2=value2, key1=value1}
> A parenthesis around the values may also help in conveying the intention:
> --cache-loader=my.CacheLoader({key2=value2, key1=value1})
>
> On Fri, Jan 26, 2018 at 10:24 AM Nick Reich wrote:
>
> > This would solve the problem of specifying the parameters for a
> Declarable,
> > but if you provided support for any valid json, you could cover other
> > situations as well, including those with more complicated and possibly
> > nested configuration. If we would ever support such scenarios in the
> > future, I assume that we would want a generic solution that would cover
> all
> > configuration. Is this something that we anticipate needing and if so,
> how
> > would the current proposal cover such situations?
> >
> > On Fri, Jan 26, 2018 at 8:43 AM, Jens Deppe wrote:
> >
> > > This also avoids the other option of implementing this by having
> > associated
> > > 'params' options for each option which can take a Declarable, thus
> > reducing
> > > the proliferation of options - in particular for 'create region'.
> > >
> > > i.e. --cache-listener AND --cache-listener-params.
> > >
> > > Further, this json parameter would not allow for arbitrary json but
> would
> > > be restricted to a simple key/value mapping so that there would be a
> > direct
> > > translation to Declarable parameters in the cache.xml.
> > >
> > > --Jens
> > >
> > > On Fri, Jan 26, 2018 at 8:07 AM, Jinmei Liao
> wrote:
> > >
> > > > Currently, when you want to specify a call-back in gfsh command
> option,
> > > you
> > > > can only pass in the class name, e.g.:
> > > >
> > > > create region --name=regionA --type=PARTITION
> > > --cache-loader=my.CacheLoader
> > > >
> > > > But these callbacks all implements Declarable (CacheLoader,
> > CacheWriter,
> > > > CacheListener, CustomExpiry etc.), i.e they can initialized with
> extra
> > > > properties that can be set when they are declared in cache.xml, but
> > > > currently, our gfsh command doesn't support that.
> > > >
> > > > We are proposing to add the support to configure Declarables in gfsh
> > > > commands by adding json strings at the end of the class name. like
> > this:
> > > >
> > > > create region --name=regionA --type=PARTITION
> > > > --cache-loader=my.CacheLoader?{"key":"value,"key2":"value2"}
> > > >
> > > > (of course, if you don't need to configure your Declarable, you can
> > still
> > > > only specify a className as before).
> > > >
> > > > Comments/thoughts?
> > > >
> > > > --
> > > > Cheers
> > > >
> > > > Jinmei
> > > >
> > >
> >
>
--
Cheers
Jinmei
Geode unit tests completed in 'develop/IntegrationTest' with non-zero exit code
Pipeline results can be found at: Concourse: https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/IntegrationTest/builds/146
[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #809 was SUCCESSFUL (with 2324 tests)
--- Spring Data GemFire > Nightly-ApacheGeode > #809 was successful. --- Scheduled 2326 tests in total. https://build.spring.io/browse/SGF-NAG-809/ -- This message is automatically generated by Atlassian Bamboo
[PROPOSAL] Feature flag for multiple caches in the same JVM
We're trying to remove singletons from the system as part of GEODE-3583. I propose adding a feature flag to allow multiple geode caches in the same JVM, and using that to build out tests that our features work without relying on a singletons. I made a PR to this affect - https://github.com/apache/geode/pull/1353. With this PR we can create multiple caches and do a basic replicated region operation. With this approach we can keep writing more tests that prove that we have eliminated singletons from various feature areas until we have completely eliminated them. -Dan
Geode unit tests completed in 'develop/FlakyTest' with non-zero exit code
Pipeline results can be found at: Concourse: https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/FlakyTest/builds/158
Geode unit tests completed in 'develop/IntegrationTest' with non-zero exit code
Pipeline results can be found at: Concourse: https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/IntegrationTest/builds/147
Geode unit tests completed in 'release-1.4.0/DistributedTest' with non-zero exit code
Pipeline results can be found at: Concourse: https://concourse.apachegeode-ci.info/teams/main/pipelines/release-1.4.0/jobs/DistributedTest/builds/15
Geode unit tests completed in 'develop/IntegrationTest' with non-zero exit code
Pipeline results can be found at: Concourse: https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/IntegrationTest/builds/148
Geode unit tests completed in 'release-1.4.0/DistributedTest' with non-zero exit code
Pipeline results can be found at: Concourse: https://concourse.apachegeode-ci.info/teams/main/pipelines/release-1.4.0/jobs/DistributedTest/builds/16
Geode unit tests completed in 'develop/DistributedTest' with non-zero exit code
Pipeline results can be found at: Concourse: https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/DistributedTest/builds/105
