Finer grained security

[Swapnil Bawaskar]

In our current security model, a user with DATA:MANAGE can create regions,
create disk stores, WAN gateways etc. I think this is a very wide scope,
because an administrator may want to give create region privilege to a
developer, but not necessarily give them the ability to create disk stores
or send the data in that region over WAN. I propose that we refine the
security model to make it finer grained.

I propose that Disk, WAN and AsyncQueue be treated as resources in the
security framework. These resources will be controlled at the CLUSTER
level. As with any other resource, admins will be able to grant READ, WRITE
and MANAGE permissions to these resources. In terms of shiro, this will
take the form: CLUSTER:READ/WRITE/MANAGE:DISK,WAN,ASYNCQUEUE.

Here is how it will work out for each resource:
DISK
1. CLUSTER:MANAGE:DISK - allows users to create/manage disk stores
2. CLUSTER:WRITE:DISK - allows users to create regions that write/overflow
to disk stores
3. CLUSTER:READ:DISK - should be covered by DATA:READ, does not make sense
here

WAN:
1. CLUSTER:MANAGE:WAN - allows users to create gateway senders and receivers
2. CLUSTER:WRITE:WAN - allows users to create regions that write data to
gateway senders
3. CLUSTER:READ:WAN - allows users to create regions that read data from
gateway receivers

We can add other things like functions here as well.

Thoughts?

Re: Simple Java Client

[Swapnil Bawaskar]

I had looked at the JCache in the past and here are some of the things I
noted:

Naming convention: Geode's region is a Cache in JSR-107, and Geode's Cache
is a CacheManager. I think it would be too confusing to change the meaning
of cache. Also, how do you document this given that Cache means different
things if you are talking JCache vs Geode.

The way to create a Cache in JSR-107 is:
Cache cache = manager.createCache(cacheName, Configuration
c);
Where it is upto the implementation to extend Configuration. Given this,
users will not be able to switch from an existing implementation to ours;
will have to write new code specially Configuration, making callbacks
serializable etc.

JSR-107 will not be limited to the client. Server side callbacks like
CacheLoader, CacheListener etc. will need handle on jsr-107 “cache”.

JSR-107 supports features like an EntryProcessor, which is a function
invoked atomically on an entry operation. We will have to make invasive
changes to Geode to support this.

Given these, I don't think supporting JSR-107 is trivial.

On Tue, Apr 25, 2017 at 2:55 PM Dan Smith  wrote:

> What transport are you planning on using? REST, or the current binary
> protocol? Or is this just a wrapper around the existing java client APIs?
>
> If this about creating a new API, I agree with what John is saying that we
> need reduce the number of APIs we have to do the same thing in java. It
> seems especially confusing if we end up with different APIs that support
> distinct features - like being able to create a region on the server with
> this new API but only being able to invoke a function with the old API.
>
> The other thing I think we need to avoid is being able to do things from
> the client (or gfsh, or xml, ...) that don't have a java API on the server.
> I'm assuming your getOrCreateRegion region is supposed behave like the gfsh
> command and update the cluster configuration? +++1 to Wes's suggestion have
> a public API for executing these gfsh operations.
>
> I really think we need to work on having *one* authoritative public API
> that contains everything that we support on the server. The protocols we
> support (REST, binary, ...) and the client drivers that use those protocols
> should just be ways of accessing that API. The Java API on the server right
> now the closest thing we have to this, but there are things you can do with
> gfsh and things you can do with the current client that have no java API
> equivalent. I think we really need to fix that!
>
> Also, preferably we won't have to hand code a bunch of stuff in every
> client driver and protocol whenever we add a new feature. For example if
> were to add a geode function to invoke each new API we add, that new API
> would be accessible from REST, gfsh, the java client etc. Later we could
> add syntatic sugar to make the new API prettier, but if we had a low level
> API that automatically exposed all new features that would make adding new
> features much less painful.
>
> I do like the idea of adding a reactive API.
>
>  Supporting JSR-107 might be nice - but that could probably be written just
> as a wrapper around the current API without too much work. I don't think we
> should do anything for JSR-107 other than provide a JSR-107 compatible
> wrapper - if someone wants additional geode specific features they should
> drop into the existing API.
>
> I also like the idea of a smaller client jar and dependencies, but I think
> there is a huge amount of room for improvement in our existing client just
> by refactoring the code a bit more and shinking geode-core into a bare
> minimum.
>
> -Dan
>
> On Mon, Apr 24, 2017 at 8:20 PM, Fred Krone  wrote:
>
> > That's great, Wes.  I will take a look.  Thanks.
> >
> > John -- good feedback to consider and I was hoping this would come up.
> >
> > "In my mind, there really are only 2 approaches... *Spring* and
> > non-*Spring*,
> > or rather PCF and non-PCF, and since PCF is primarily based on Boot
> (given
> > Microservices/applications are the new concurrency), then I am really
> > saying the same thing."
> >
> > This would be cover non spring and attempt to give the community
> something
> > that had the same standardized experience as JSR 107 -- starting with the
> > Cache interface itself.  Although we don't necessarily have to implement
> > Cache, it's method signatures are essentially a (non spring) caching
> > standard for Java developers at this point.   We considered full blown
> JSR
> > 107 implementation but thought it was too robust for the needs mentioned
> > (that's not to say we couldn't get there incrementally).  I think those
> > needs still exist open-source outside of PCF and don't cannibalize.
> >
> >
> >
> >
> >
> >
> > On Mon, Apr 24, 2017 at 7:44 PM, Real Wes 
> wrote:
> >
> > >
> > > Here is a simple Java client _for enterprise use_ that I developed for
> > > Geode and distributed to several enterprises. It has similarities and
> > > differences for your goal. This pro

Re: Simple Java Client

[Swapnil Bawaskar]

 by saying "must have Geode API knowledge"? I can find people
> >>>>>>>
> >>>>>> with
> >>>>
> >>>>> JCache knowledge though. So from in implementor's perspective having
> >>>>>>> standards is a must. Now I don't think the JSR-107 interface should
> >>>>>>>
> >>>>>> be
> >>>>
> >>>>> the
> >>>>>>
> >>>>>>> root interface but rather a facade on the "raw" Geode proprietary
> bad
> >>>>>>>
> >>>>>> ass
> >>>>>
> >>>>>> API. That API should be 100% asynchronous (reactive, SEDA, whatever
> >>>>>>>
> >>>>>> the
> >>>>
> >>>>> current buzzword for asynchronous is today). Around that API we can
> >>>>>>>
> >>>>>> provide
> >>>>>>
> >>>>>>> facades for JSR 107, ConcurrentMap (our current yet not so well
> >>>>>>>
> >>>>>> behaving
> >>>>>
> >>>>>> API), List, Queue, etc. Maybe even JPA, JCA, etc. The thought of
> >>>>>>>
> >>>>>> putting
> >>>>>
> >>>>>> all those features into a single API makes my head exploded and they
> >>>>>>>
> >>>>>> don't
> >>>>>>
> >>>>>>> need to be like they are today.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Apr 25, 2017 at 8:25 PM Wes Williams  >
> >>>>>>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> A couple of points to interact with John's points.
> >>>>>>>>
> >>>>>>>> GFSH as API
> >>>>>>>>
> >>>>>>>> We agree that GFSH is a DSL, and a really good and useful one. We
> >>>>>>>>
> >>>>>>> agree
> >>>>>
> >>>>>> that we don't want our API tied to GFSH syntax. I view the valuable
> >>>>>>>>
> >>>>>>> part
> >>>>>>
> >>>>>>> of
> >>>>>>>
> >>>>>>>> GemFire admin as the Java code underneath GFSH, or the "Commands."
> >>>>>>>>
> >>>>>>>> For example, to create a JAVA API to "Create Region",  why not
> >>>>>>>>
> >>>>>>> create a
> >>>>>
> >>>>>> Java API around CreateAlterDestroyRegionCommands? In this way,
> >>>>>>>>
> >>>>>>> GFSH
> >>>>
> >>>>> and
> >>>>>
> >>>>>> the
> >>>>>>>
> >>>>>>>> JAVA API share the same code. It seems too obvious yet I don't see
> >>>>>>>>
> >>>>>>> it
> >>>>
> >>>>> being
> >>>>>>>
> >>>>>>>> recommended.  Why not?  (Note: the hard-coded formatting would
> need
> >>>>>>>>
> >>>>>>> to
> >>>>>
> >>>>>> be
> >>>>>>
> >>>>>>> removed).
> >>>>>>>>
> >>>>>>>> Once you have the Java/GFSH/REST API as common code, you then
> >>>>>>>>
> >>>>>>> refactor
> >>>>>
> >>>>>> it.
> >>>>>>>
> >>>>>>>> What's the objection to this approach? Once you open Java API's to
> >>>>>>>>
> >>>>>>> do
> >>>>
> >>>>> everything that GFSH does, then you have now unshackled the power
> >>>>>>>>
> >>>>>>> of
> >>>>
> >>>>> the
> >>>>>>
> >>>>>>> developer (the JAVA API) and the admin (GFSH API).
> >>>>>>>>
> >>>>>>>>
> >

Re: Finer grained security

[Swapnil Bawaskar]

Thanks for feedback! I have tried to incorporate this on our wiki:
https://cwiki.apache.org/confluence/display/GEODE/Finer+grained+security.
Comments welcome.

On Thu, Apr 27, 2017 at 1:33 PM John Blum  wrote:

> +1 to Jake's comments, and is a fundamental property of Java's security
> internally.
>
> On Thu, Apr 27, 2017 at 1:09 PM, Jacob Barrett 
> wrote:
>
> > Typical solution to the X service needs to create something it service Y
> > where user has permission to X but not to Y is to treat the actions on Y
> > performed by X to be trusted. Often I have seen this implemented such
> that
> > after asserting permission on "create" on X that X performs actions on Y
> as
> > a trusted principal, like a "SYSTEM" user. The other option is to give
> each
> > service a trusted account and elevate to "SERVICE-X" where Y would allow
> > "SERVICE-X" to perform some set of operations.
> >
> > See
> > https://docs.oracle.com/javase/7/docs/api/javax/
> > security/auth/Subject.html#doAs(javax.security.auth.
> > Subject,%20java.security.PrivilegedAction
> > )
> >
> > -Jake
> >
> >
> > On Thu, Apr 27, 2017 at 11:28 AM Michael Stolz 
> wrote:
> >
> > > We have seen users who need per-Region permission for Data read/write,
> so
> > > there is precedent there at least.
> > >
> > > --
> > > Mike Stolz
> > > Principal Engineer, GemFire Product Manager
> > > Mobile: +1-631-835-4771 <(631)%20835-4771> <(631)%20835-4771>
> > >
> > > On Thu, Apr 27, 2017 at 2:11 PM, pulkit chandra <
> > pulkit.chan...@gmail.com>
> > > wrote:
> > >
> > > > For per instance permission, I would say look for the evidence. Do we
> > > have
> > > > evidence that customers want per instance permission? If not may be
> > > > implement minimally in the first cut and validate with customers if
> > they
> > > > want per instance model?
> > > >
> > > > About Lucene concern, It is in fact good to provide permission for
> per
> > > > logical operation that implementation is doing. This will align the
> > > > security permission with operations performed and provide better
> design
> > > of
> > > > a role. I would argue if you are able to perform 2 logically separate
> > > > operations with single permission is perhaps a smell that you might
> not
> > > > have enough permissions.
> > > >
> > > > my $0.02
> > > >
> > > > On Wed, 26 Apr 2017 at 16:54 Dan Smith  wrote:
> > > >
> > > > > I agree that async event queues seem like a different case than wan
> > or
> > > > > disk. In that case you are not using anything that creating a
> region
> > > > > doesn't do.
> > > > >
> > > > > Shouldn't creating a region be DATA:MANAGE:DISK? Requiring DATA
> > > > privileges
> > > > > for a region without disk and CLUSTER privileges for a region with
> > disk
> > > > > seems weird. Same issues with creating a region that uses WAN.
> > > > >
> > > > > -Dan
> > > > >
> > > > > On Wed, Apr 26, 2017 at 12:53 PM, Diane Rose Hardman <
> > > > dhard...@pivotal.io>
> > > > > wrote:
> > > > >
> > > > > > One more possible complication is that creating a Lucene index
> will
> > > > also
> > > > > > create an AsyncEventQueue. Today the required permission to
> create
> > > the
> > > > > AEQ
> > > > > > is DATA:MANAGE which coincidentally nicely matches the permission
> > > > > required
> > > > > > to create an OQL index.
> > > > > > Pulling out the AEQ as a separate resource will affect Lucene
> index
> > > > > > creation. FYI.
> > > > > >
> > > > > > On Tue, Apr 25, 2017 at 3:10 PM, Jinmei Liao 
> > > > wrote:
> > > > > >
> > > > > > > DATA:*:RegionA would allow you to only operate that region but
> > not
> > > > all
> > > > > of
> > > > > > > them.
> > > > > > >
> > > > > > > if we want to control a specific wan, maybe we add a fourth
> > > > parameter:
> > > > > > > cluster:*:wan:wanName, same goes for Disk etc.
> > >

Re: Last call on 1.2.0 (and fixing test failures)

[Swapnil Bawaskar]

I think we should also wait for GEODE-2836

On Mon, May 15, 2017 at 8:58 AM Karen Miller  wrote:

> Let's finish GEODE-2913, the documentation for improvements made to the
> Lucene integration and include it with the 1.2.0 release!
>
>
> On Sun, May 14, 2017 at 6:47 PM, Anthony Baker  wrote:
>
> > Hi everyone,
> >
> > Our last release was v1.1.1 in March.  We have made a lot of great
> > progress on the develop branch with over 250 issues fixed.  It would be
> > great to get those changes into a release.  What’s left before we are
> ready
> > to release 1.2.0?
> >
> > Note that we need a clean test run before releasing (except for “flaky"
> > tests).  We haven’t had one of those in awhile [1].
> >
> > Anthony
> >
> > [1] https://builds.apache.org/job/Geode-nightly/
> > lastCompletedBuild/testReport/
> >
> >
>

Re: [jira] [Updated] (GEODE-3005) A developer can create a Region with Partition Resolution using Java API

[Swapnil Bawaskar]

For this reason:
But since the user has to always add this delimiter to their string keys
perhaps it is better for them to have to explicitly configure what it is.

I would vote for a single configuration.



On Wed, May 31, 2017 at 4:17 PM Darrel Schneider 
wrote:

> The partition-by-prefix feature can be configured with gfsh, cache.xml, or
> apis.
>
> We did not want it to be configured as a PartitionResolver because that
> ends up with overly verbose configuration.
> You end up with something like: gfsh create region "foo"
> --partition-resolver=org.apache.geode.resolvers.PartitionByPrefix.class
> instead of: gfsh create region "foo" --partition-by-prefix=true
>
> The reason for two configuration attributes (a boolean to enable the
> feature, and a string to customize the default delimiter) was for clarity.
> We thought that most use cases would just use the boolean and set it to
> true because they could use the default delimiter.
> The configuration of the delimiter is only needed when your existing prefix
> string contains the default delimiter.
> If we have just one configuration attribute (the delimiter) then it needs
> to default to the empty string which means the feature is disabled.
> I originally was pushing for the single configuration attribute but was
> convinced by others that the two config attribute was better. I like the
> idea of geode having a predefined delimiter like ":" instead of it always
> needed to be configured if you are going to use it. But since the customer
> has to always add this delimiter to their string keys perhaps it is better
> for them to have to explicitly configure what it is.
>
> I do not like the idea of returning the entire key if no delimiter is
> found. This seems error prone. If a developer forgets to put the correct
> delimiter in a key I think it is better for it to fail fast then to just
> instead use the default hash of the entire key.
> In the example of a Customer and Customer.Orders region you would just use
> the normal "partition-by-key" policy on the Customer region and then
> configure "partition-by-prefix" on the Customer.Orders region. You have no
> need for both regions to be configured with "partition-by-prefix".
>
> If someone does want to have a mix of keys, some with prefixes and some
> without, on the same region then they could just implement their own
> PartitionResolver and have it support this mix. I don't see that
> "partition-by-prefix" needs to support this use case unless it is very
> common.
>
> On Wed, May 31, 2017 at 10:15 AM, Michael Stolz  wrote:
>
> > I don't really see a need for new Region api for this except to be able
> to
> > specify the delimeter.
> >
> > A developer can just specify the provided PartionByPrefix class for his
> > PartitionResolver during region creation?
> >
> > Providing a String key with no delimiter should just return the entire
> key,
> > NOT throw an error. That way the same PartitionResolver can be used for
> > both the Customer region and the Customer.Orders region and it just
> works.
> >
> > One more thing, we need to be able to specify a partition resolver via
> > gfsh.
> >
> > --
> > Mike Stolz
> > Principal Engineer - Gemfire Product Manager
> > Mobile: 631-835-4771 <(631)%20835-4771> <(631)%20835-4771>
> >
> > On May 30, 2017 3:26 PM, "Fred Krone (JIRA)"  wrote:
> >
> > >
> > >  [ https://issues.apache.org/jira/browse/GEODE-3005?page=com.at
> > > lassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
> > >
> > > Fred Krone updated GEODE-3005:
> > > --
> > > Description:
> > > A user should be able to set partition by prefix programmatically when
> > > creating a partitioned region.
> > >
> > > This can only be done when creating a Region type PARTITION
> > >
> > > Acceptance:
> > > Partitioned region can be created using new 'partition by prefix'
> > > attributes (on/off, delimiter)
> > > Only String keys work -- all other keys throw an error
> > > Providing a key with the correct delimiter routes the entry to the
> > correct
> > > node
> > > Providing a key with no delimiter throws an error
> > >
> > >
> > >   was:
> > > A user should be able to set partition by prefix programmatically when
> > > creating a partitioned region.
> > >
> > > This can only be done when creating a Region type PARTITION
> > >
> > >
> > > > A developer can create a Region with Partition Resolution using Java
> > API
> > > > 
> > 
> > > >
> > > > Key: GEODE-3005
> > > > URL:
> https://issues.apache.org/jira/browse/GEODE-3005
> > > > Project: Geode
> > > >  Issue Type: Wish
> > > >  Components: regions
> > > >Reporter: Fred Krone
> > > >Assignee: Fred Krone
> > > >
> > > > A user should be able to set partition by prefix programmatically
> when
> > > creating a partitioned region.
> > > > This can only be done when creating

Re: [jira] [Updated] (GEODE-3005) A developer can create a Region with Partition by Prefix using Java API

[Swapnil Bawaskar]

I don't see how this RoutableKey approach, which is a Java interface will
work from non-java clients. So, in the interest of cross-language support
and keeping the usage simple, I think we should just restrict this to work
with utf-8 encoded strings with a delimiter.

I agree however that this should just be an implementation of the
PartitionResolver and should be configured as such. i.e. while creating the
region users could say:
gfsh>create region --name=foo --type=PARTITION
--partition-resolver=org.apache.geode.StringPartitionResolver
which will just use a default delimiter (say ":"). In the future we can try
to figure out how to pass parameters to the partition resolver to make the
delimiter configurable. And may be also provide the RoutableKey for java
clients.


On Fri, Jun 2, 2017 at 10:54 AM Michael Stolz  wrote:

> +1 to the RoutableKey. Does need to be supported on any non-java clients
> too though.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: +1-631-835-4771 <(631)%20835-4771>
>
> On Thu, Jun 1, 2017 at 8:31 PM, Darrel Schneider 
> wrote:
>
> > +1 to the RoutableKey idea.
> > It also gets rid of any need for a delimiter since the two args passed
> when
> > constructing this key define which part is for resolving routing and
> which
> > part is the key.
> >
> > As far as memory overhead goes we could do some optimization of how this
> > key is stored by not exposing the actual class we use to implement
> > RoutableKey but instead RoutableKey would just be an interface and we
> would
> > provide a factory method on Cache for creating a new key.
> >
> >
> > On Thu, Jun 1, 2017 at 4:59 PM, Fred Krone  wrote:
> >
> > > Going this route we could do something like: RoutableKey
> implements
> > > PartitionResolver and pass it in as a key (this was batted around in
> the
> > > feature narrative).
> > >
> > > Here RO is the routing object, K is the original key.
> > >
> > > Region.put already looks for key to implement PartitonResolver and if
> it
> > > does will use the routing object.
> > >
> > > This way we don't need any API changes, you don't even need to set this
> > > PartitionResolver on gfsh create region and if we do this with generics
> > you
> > > can make whatever compound key you like.
> > >
> > > Basically we're just doing the implementation and making it generic.
> One
> > > caveat is that this neglects .Net clients and will use more memory
> > > overhead.
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Jun 1, 2017 at 2:20 PM, Michael Stolz 
> wrote:
> > >
> > > > I think that implementing a pre-built PartitionResolver that is
> bundled
> > > > with Geode would enable users to leverage that PartitionResolver
> > without
> > > > writing any code just by mentioning its class name during region
> > > creation.
> > > >
> > > > That's a huge win. Co-location without writing code. Period. Yay!
> > > >
> > > > Adding the ability to specify the delimiter is a nice add-on to that
> > and
> > > > could probably be done by parameterizing the PartitionResolver much
> > like
> > > > the way Listeners can be parameterized but its not there in the
> tooling
> > > at
> > > > this moment.
> > > >
> > > > What's bothering me is the fact that we already have a way to
> specify a
> > > > PartitionResolver generically built into all our tooling, and now we
> > are
> > > > adding another way to specify this particular PartitionResolver that
> > > could
> > > > cause confusion with the current generic way, and could even run into
> > > > conflicts that we need no handle specially.
> > > >
> > > > If we have already taken care of ensuring that the class is on the
> > > > classpath, it is probably not really much easier to say
> > > > partition-by-prefix=true
> > > > than it is to say
> > > > partition-resolver=org.apache.geode.somepackage.some.class
> > > > and if we don't add "partition-by-prefix" into the API and tooling
> then
> > > > there's no temptation to ever say both.
> > > >
> > > >
> > > >
> > > > --
> > > > Mike Stolz
> > > > Principal Engineer, GemFire Product Manager
> > > > Mobile: +1-631-835-4771 <(631)%20835-4771>
> > > >
> > > > On Thu, Jun 1, 2017 at 4:54 PM, Darrel Schneider <
> > dschnei...@pivotal.io>
> > > > wrote:
> > > >
> > > > > We already have a general purpose mechanism that, if you are
> willing
> > to
> > > > > write java code, handles all the permutations you mention. That
> > > mechanism
> > > > > is the PartitionResolver interface.
> > > > >
> > > > > This new feature is intended to give users a way to control how
> their
> > > > keys
> > > > > are partitioned without writing any java code. I think by
> > "unnecessary
> > > > API"
> > > > > you mean "unnecessary feature". But our current features do not
> give
> > > you
> > > > a
> > > > > way to control partitioning without implementing a
> PartitionResolver.
> > > > This
> > > > > new feature allows you to just configure your region (using gfsh,
> > xml,
> > > or
> > > > > apis) and then format your s

Re: Do you want to Tweet for @ApacheGeode?

[Swapnil Bawaskar]

So can I.

On Tue, Jun 6, 2017 at 4:40 PM Michael Stolz  wrote:

> Hey Greg,
>
> I could tweet now and then.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: +1-631-835-4771 <(631)%20835-4771>
>
> On Tue, Jun 6, 2017 at 12:28 PM, Greg Chase  wrote:
>
> > Greetings Apache Geode community,
> > Tweeting for the Apache Geode project in the past has been a community
> > effort.
> >
> > However, our most prolific tweeters are currently not spending much time
> > with the project.
> >
> > As a result @ApacheGeode has fallen silent.
> >
> > Per ASF recommendations, we manage access to the @ApacheGeode Twitter
> > handle through Tweet Deck. Any contributor is welcome to Tweet on behalf
> of
> > the project.  You simply need to ask for access, such as replying to this
> > thread.
> >
> > Presuming no concerns raised by the PMC, access is granted to your
> personal
> > Twitter credentials to access @ApacheGeode using Tweet Deck.
> >
> > Administrative rights to @ApacheGeode are maintained by several PMC
> > members.
> >
> > The guideline for Tweeting for @ApacheGeode is to promote the project,
> > announce any important releases, events or topics, and to help answer
> > questions or direct askers to u...@geode.apache.org.
> >
> > You are also welcome to engage in technical topics of interest to Apache
> > Geode users, such as event driven architectures, caching patterns,
> > in-memory computing, etc.
> >
> > We recommend not engaging in bashing other projects or products, or
> > engaging in off-topic discussions.
> >
> > If you are interested in helping Tweet for @ApacheGeode, I invite you to
> > volunteer by replying to this thread.
> >
> > Regards,
> >
> > -Greg
> >
>

Re: SecurityService versus Cluster Config

[Swapnil Bawaskar]

I think we should revisit connecting to the distributed system with
properties. In addition to what Kirk mentioned below, I think one of the
big limitation of this is the inability to change properties on a running
cluster. If there is a property that a user needs to change, they will have
to bounce the cluster as opposed to performing a rolling restart of the
cluster with that property changed. I think the quickest way for us to get
there is to make the connection a two step process, connect just to get the
properties from the cluster config avoiding any validation, then reconnect
using the new set of properties.

On Fri, Jun 9, 2017 at 2:54 PM Kirk Lund  wrote:

> The usage of CacheFactory#setSecurityManager (and setPostProcessor) is
> working as expected, both before and after my changes!
>
> The problem is that Cluster Config is requested during Cache initialization
> which means that any gemfire.properties stored in Cluster Config will not
> be used by DistributedSystem -- only the properties that affect the Cache
> (such as cache-xml-file) are used as expected in Cluster Config. This
> problem exists with/without my changes but my changes exposed this fact by
> moving the use of the security-manager property to DistributedSystem. I'm
> working on a short-term fix specific to security-manager. Correcting the
> problem for all gemfire.properties will be a long-term fix.
>
> On Fri, Jun 9, 2017 at 1:48 PM, John Blum  wrote:
>
> > I am not sure I follow everything that is happening here quite yet as I
> > have not had time to go over this in more detail and digest it.
> >
> > But, I certainly hope that the security-manager property in Geode is not
> > impacted by this in anyway since *Spring Data Geode* builds on this and,
> > well, this
> >  > src/main/java/org/apache/geode/cache/CacheFactory.java#L355-L368>
> > [1],
> > as way to configure Geode (Integrated) Security.
> >
> > Thanks,
> > John
> >
> > [1]
> > https://github.com/apache/geode/blob/develop/geode-core/
> > src/main/java/org/apache/geode/cache/CacheFactory.java#L355-L368
> >
> >
> > On Fri, Jun 9, 2017 at 1:11 PM, Kirk Lund  wrote:
> >
> > > Yeah I think we need #2 in the long run. Right now nearly all of the
> > > gemfire.properties are ignored if you use Cluster Config. The few
> > > gemfire.properties that are mutable by RuntimeDistributionConfigImpl
> will
> > > work when used in Cluster Config -- I believe all other
> > gemfire.properties
> > > would be ignored until we complete #2.
> > >
> > > On Fri, Jun 9, 2017 at 1:02 PM, Udo Kohlmeyer 
> > > wrote:
> > >
> > > > +1 for #2. It does seem more correct
> > > >
> > > >
> > > >
> > > > On 6/9/17 12:45, Kirk Lund wrote:
> > > >
> > > >> The new changes for SecurityService don't work with Cluster Config.
> We
> > > >> only
> > > >> had one test that would've failed but it was annotated with @Ignore.
> > > >>
> > > >> The problem is this:
> > > >>
> > > >> The security-manager is a gemfire property and is now used by the
> > > >> InternalDistributedSystem because membership needs SecurityService
> to
> > > >> handle peer-to-peer authentication. So with my changes, the
> > > >> InternalDistributedSystem now creates SecurityService and passes it
> > into
> > > >> the constructor of GemFireCacheImpl.
> > > >>
> > > >> Next, GemFireCacheImpl#initialize requests Cluster Config. If this
> is
> > a
> > > >> new
> > > >> Server with no gemfire properties, it will have already joined the
> > > Cluster
> > > >> (before and after my changes). It then gets Cluster Config that has
> > > >> gemfire.properties including security-manager. But it's too late,
> the
> > > >> immutable SecurityService has already been created by
> > > >> InternalDistributedSystem.
> > > >>
> > > >> In theory, Cluster Config should be requested before the creation of
> > > >> InternalDistributedSystem so that other gemfire.properties in
> Cluster
> > > >> Config can be applied (such as member-timeout). In fact most of the
> > > >> gemfire.properties control aspects of InternalDistributedSystem.
> > > >> Presently,
> > > >> all gemfire.properties that would configure
> InternalDistributedSystem
> > > are
> > > >> ignored because Cluster Config is loaded after
> > InternalDistributedSystem
> > > >> was created and the member has joined the cluster.
> > > >>
> > > >> We would need to make one of these changes:
> > > >>
> > > >> 1) GemFireCacheImpl creates its own SecurityService which is
> different
> > > >> from
> > > >> the one used by membership. Locators are the only members that
> really
> > > need
> > > >> membership to have a fully configured SecurityService (and I think
> we
> > > >> already have a bug in which 2nd Locators need to be manually
> > configured
> > > >> rather than use Cluster Config).
> > > >>
> > > >> 2) Move Cluster Config request from GemFireCacheImpl to
> > > >> InternalDistributedSystem. This is actually more correct, since
> > Cluster
> > > >> 

Re: Apache Geode + Narayana JTA = Global Transactions with Last-Resource Optimization

[Swapnil Bawaskar]

Great work Christian!
Can you add a page about this on the wiki
 as well, so that
your work is not lost.

On Mon, Jul 3, 2017 at 10:58 AM Dan Smith  wrote:

> Awesome!
>
> -Dan
>
> On Mon, Jul 3, 2017 at 6:08 AM, Christian Tzolov 
> wrote:
>
> > Recent experience of setting up Geode as a last-resource inside WebSphere
> > JTA, has motivated me to find a simpler way to integrate with JTA.
> >
> > The https://github.com/tzolov/narayana-jta-geode-support provides
> > utilities
> > to integrate Geode with Narayana JTA. This enables global transactions
> > using the embeddable, standalone, open-source Narayana JTA and implements
> > the Last-Resource Commit Optimization (LRCO) to preserve the ACID
> > properties.  (e.g. Geode is registered as last-resource in the global
> > transactions).
> >
> > With narayana-geode-core you can use it with the plain Geode API:
> > http://bit.ly/2tByAr5
> > or use narayana-geode-springboot for seamless integration with Spring
> Boot
> > and SpringDataGemfire. See the http://bit.ly/2siBEof example
> >
> > All dependencies are resolved from maven central.
> >
> > So if you build a SpringBoot app you only need to add the maven
> > dependencies and set a single annotation @EnableGeodeNarayanaJta. No need
> > for obsolete, commercial  Application Servers no accidental complexity.
> >
> > I’ve tried to explain the problem and the solution in my blog here:
> > http://blog.tzolov.net/2017/07/apache-geode-gemfire-
> > narayana-jta.html?view=sidebar
> >
> > Because Narayana uses LGPL it is not compatible for distribution with
> ASF,
> > but it can be referred as optional component.
> >
> > Cheers,
> > Christian
> >
> > P.S. the implementation uses code and ideas from the Geode engineering
> team
> > and from John Blum ;)
> > --
> > Christian Tzolov  | Data Engineer,
> EMEA
> > | Pivotal  | ctzo...@pivotal.io |+31610285517
> <+31%206%2010285517>
> >
>

Re: Apache Geode + Narayana JTA = Global Transactions with Last-Resource Optimization

[Swapnil Bawaskar]

Sure, would be glad to help.

On Thu, Jul 13, 2017 at 2:57 AM Christian Tzolov  wrote:

> Hi
>
> As suggested i've added related wiki page here:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71764563#
>
> Feel free to review and edit!
>
> @Swapnil, @John, since this is an external, optional utility it might
> useful to add you as github (
> https://github.com/tzolov/narayana-jta-geode-support) project owners too?
> As an attempt to keep the code in synch with the core Geode/SDG projects?
>
> Cheers,
> Christain
>
> On 4 July 2017 at 22:59, John Blum  wrote:
>
> > 1 additional note...
> >
> > As a follow up to what *Christian* has provided here (which is highly
> > appreciated), I am also adding formal support for using Geode as a Last
> > Resource in a CMT/JTA transaction initiated by a *Spring*
> > managed/demarcated transaction in...
> >
> > DATAGEODE-16 - "*Add support for Geode JCA ResourceAdapter*
> > <https://jira.spring.io/browse/DATAGEODE-16>" [1]
> >
> > The work being done in DATAGEODE-16 is based on POC work we did for a
> > Pivotal customer and will benefit the entire community.
> >
> > Cheers,
> > John
> >
> >
> > [1] https://jira.spring.io/browse/DATAGEODE-16
> >
> >
> > On Tue, Jul 4, 2017 at 1:53 PM, John Blum  wrote:
> >
> > > Well done Christian!  Thank you for the hard work and contributions.
> > >
> > > Cheers,
> > > -j
> > >
> > > On Mon, Jul 3, 2017 at 12:43 PM, Swapnil Bawaskar <
> sbawas...@pivotal.io>
> > > wrote:
> > >
> > >> Great work Christian!
> > >> Can you add a page about this on the wiki
> > >> <https://cwiki.apache.org/confluence/display/GEODE/Index> as well, so
> > >> that
> > >> your work is not lost.
> > >>
> > >> On Mon, Jul 3, 2017 at 10:58 AM Dan Smith  wrote:
> > >>
> > >> > Awesome!
> > >> >
> > >> > -Dan
> > >> >
> > >> > On Mon, Jul 3, 2017 at 6:08 AM, Christian Tzolov <
> ctzo...@pivotal.io>
> > >> > wrote:
> > >> >
> > >> > > Recent experience of setting up Geode as a last-resource inside
> > >> WebSphere
> > >> > > JTA, has motivated me to find a simpler way to integrate with JTA.
> > >> > >
> > >> > > The https://github.com/tzolov/narayana-jta-geode-support provides
> > >> > > utilities
> > >> > > to integrate Geode with Narayana JTA. This enables global
> > transactions
> > >> > > using the embeddable, standalone, open-source Narayana JTA and
> > >> implements
> > >> > > the Last-Resource Commit Optimization (LRCO) to preserve the ACID
> > >> > > properties.  (e.g. Geode is registered as last-resource in the
> > global
> > >> > > transactions).
> > >> > >
> > >> > > With narayana-geode-core you can use it with the plain Geode API:
> > >> > > http://bit.ly/2tByAr5
> > >> > > or use narayana-geode-springboot for seamless integration with
> > Spring
> > >> > Boot
> > >> > > and SpringDataGemfire. See the http://bit.ly/2siBEof example
> > >> > >
> > >> > > All dependencies are resolved from maven central.
> > >> > >
> > >> > > So if you build a SpringBoot app you only need to add the maven
> > >> > > dependencies and set a single annotation @EnableGeodeNarayanaJta.
> No
> > >> need
> > >> > > for obsolete, commercial  Application Servers no accidental
> > >> complexity.
> > >> > >
> > >> > > I’ve tried to explain the problem and the solution in my blog
> here:
> > >> > > http://blog.tzolov.net/2017/07/apache-geode-gemfire-
> > >> > > narayana-jta.html?view=sidebar
> > >> > >
> > >> > > Because Narayana uses LGPL it is not compatible for distribution
> > with
> > >> > ASF,
> > >> > > but it can be referred as optional component.
> > >> > >
> > >> > > Cheers,
> > >> > > Christian
> > >> > >
> > >> > > P.S. the implementation uses code and ideas from the Geode
> > engineering
> > >> > team
> > >> > > and from John Blum ;)
> > >> > > --
> > >> > > Christian Tzolov <http://www.linkedin.com/in/tzolov> | Data
> > Engineer,
> > >> > EMEA
> > >> > > | Pivotal <http://pivotal.io/> | ctzo...@pivotal.io |+31610285517
> <+31%206%2010285517>
> > >> > <+31%206%2010285517>
> > >> > >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > -John
> > > john.blum10101 (skype)
> > >
> >
> >
> >
> > --
> > -John
> > john.blum10101 (skype)
> >
>
>
>
> --
> Christian Tzolov <http://www.linkedin.com/in/tzolov> | Data Engineer, EMEA
> | Pivotal <http://pivotal.io/> | ctzo...@pivotal.io |+31610285517
> <+31%206%2010285517>
>

Re: New Committer: Deepak Dixit

[Swapnil Bawaskar]

Welcome Deepak!

On Sat, Jul 15, 2017 at 8:45 PM Amey Barve  wrote:

> Welcome Deepak and Congratulations!
>
> Regards,
> Amey
>
> On 16 Jul 2017 2:37 am, "Nitin Lamba"  wrote:
>
> Welcome Deepak!
> Nitin
>
> On Sat, Jul 15, 2017 at 1:07 AM, Avinash Dongre 
> wrote:
>
> > Welcome Deepak !
> >
> > Thanks
> > Avinash
> >
> > On Sat, Jul 15, 2017 at 8:08 AM Anthony Baker  wrote:
> >
> > > Welcome Deepak!
> > >
> > > Anthony
> > >
> > > Sent from my iPhone
> > >
> > > > On Jul 14, 2017, at 4:46 PM, Mark Bretl  wrote:
> > > >
> > > > The Apache Geode Project Management Committee has invited Deepak
> Dixit
> > to
> > > > be committer on the project. We are pleased to announce he has
> > accepted.
> > > >
> > > > Please join me in welcoming Deepak!
> > > >
> > > > Thanks,
> > > >
> > > > Mark
> > > > On behalf of the Apache Geode PMC
> > >
> >
>

[DISCUSS] authorizing function execution

[Swapnil Bawaskar]

Discuss fix for GEODE-2817


Currently to execute a function, you will need "data:write" permission, but
it really depends on what the function is doing. For example, if a function
is just reading data, the function author might want users with DATA:READ
permissions to execute the function. The two options mentioned in the
ticket are:

1) externalize SecurityService so that function author can use it in the
function.execute code to check authorization.
2) add a method to function interface to tell the framework what permission
this function needs to execute, so that the framework will check the
permission before executing the function.

I vote for #2 because, I think, a function author will be able to easily
discover a method on the Function interface, rather than trying to look for
SecurityService.

I propose that we add the following new method to Function:

default public List requiredPermissions() {
   // default DATA:WRITE
}

In order to preserve existing behavior, the default required permission
would be DATA:WRITE.

Re: How to export region keys in json file

[Swapnil Bawaskar]

I think you need to wrap the entire json string in quotes. From GEODE-3001
:

gfsh>put --key-class=io.pivotal.gemfire.testing.MyKey
--key="{'identifier': 'KIWI131117+65', 'name':'name'}" --value=foo2
--region=/foo



On Thu, Aug 17, 2017 at 6:15 AM Dinesh Akhand  wrote:

> Hi I tried below,
>
> gfsh>query --query='select e.key from /CUSTOMER_1.entrySet e'
>
> Result : true
> startCount : 0
> endCount   : 20
> Rows   : 7
>
>name| customerID | cycleCode |
> customerGroup | partitionID
> -- | -- | - |
> - | ---
> amdocs.imdg.model.keys.CustomerKey | 1592   | 1 | 30006
>  | 300062
>
>
> but when I ran
>
> gfsh>remove --region=/ CUSTOMER_1
> --key={\"customerID\":1592,\"customerGroup\":30006,\"cycleCode\":1,\"partitionID\":300062}
> --key-class=amdocs.imdg.model.keys.CustomerKey
> Result: false
> Message   : Key is not present in the region
> Key Class : java.lang.String
> Key   :
> {\"customerID\":1592,\"customerGroup\":30006,\"cycleCode\":1,\"partitionID\":300062}
> --key-class=amdocs.imdg.model.keys.CustomerKey
>
> Showing error key not exist.
>
> We have customerID=LONG type
> cycleCode = short
> customerGroup = int
> partitionID =int
>
> Can you please suggest.
>
> Thanks,
> Dinesh Akhand
>
>
> From: Barry Oglesby [mailto:bogle...@pivotal.io]
> Sent: Thursday, August 17, 2017 4:18 AM
> To: dev@geode.apache.org
> Subject: Re: How to export region keys in json file
>
> There isn't anything built in to export keys as JSON. There was some talk
> about exporting values as JSON at one time, but that hasn't been done.
>
> I attached a function to dump the keys as JSON. It just logs them to the
> terminal, but it can easily be changed to write them to a file.
>
> It dumps output like:
>
> Region data contains the following 34 keys:
>
> {"customerID":"28","customerGroup":"group-28","cycleCode":"cycleCode-28","partitionID":28}
>
> {"customerID":"21","customerGroup":"group-21","cycleCode":"cycleCode-21","partitionID":21}
>
> {"customerID":"68","customerGroup":"group-68","cycleCode":"cycleCode-68","partitionID":68}
>
> {"customerID":"81","customerGroup":"group-81","cycleCode":"cycleCode-81","partitionID":81}
>
> {"customerID":"35","customerGroup":"group-35","cycleCode":"cycleCode-35","partitionID":35}
>
> {"customerID":"90","customerGroup":"group-90","cycleCode":"cycleCode-90","partitionID":90}
>
> {"customerID":"15","customerGroup":"group-15","cycleCode":"cycleCode-15","partitionID":15}
>
> {"customerID":"44","customerGroup":"group-44","cycleCode":"cycleCode-44","partitionID":44}
>
>
> Thanks,
> Barry Oglesby
>
>
> On Wed, Aug 16, 2017 at 4:35 AM, Dinesh Akhand  > wrote:
> Hi,
>
> How I can export Region keys in json file.
>
> It will help full for us to use get command or remove command for
> particular key when we have the composite keys defined.
>
> get --region=/CUSTOMER_1
> --key=({'customerID':'1593','customerGroup':'30013','cycleCode':'1','partitionID':'300133'})
> --key-class=amdocs.imdg.model.keys.CustomerKey
> Message : Error in converting JSON Couldn't convert JSON to Object of type
> class amdocs.imdg.model.keys.CustomerKey
>
>
> Thanks,
> Dinesh Akhand
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>

Re: [DISCUSS] authorizing function execution

[Swapnil Bawaskar]

So, it sounds like if we did #1 above: i.e:
1) externalize SecurityService so that function author can use it in the
function.execute code to check authorization.

we could get this to work with lambdas:
ResultCollector rc = getExecution().execute(context ->
  context.getCache().getSecurityService().authorizeRead();
  // perform read
  context.getResultSender().lastResult(result)
);

Udo,
I think there are plenty of use cases where a user would only have
privileges to read data, if this reading of data involves a function, that
user will need write privileges too.

On Thu, Aug 17, 2017 at 11:26 AM Udo Kohlmeyer 
wrote:

> In the case of lambda executions... we have no way to annotate the
> lambda...
>
> So maybe the "outer" service call needs to help us... Maybe the security
> framework should automatically prevent the execution of any function is
> the user does not have "DATA:READ"/"DATA:WRITE" or "FUNCTION:EXEC"
> privileges.
>
> In addition to this, do we need to distinguish between "read" and
> "write" functions?
>
>
> On 8/17/17 10:43, Dan Smith wrote:
> >> if we get to Lambda-based functions
> > No if about it, this works right now and we do this in our tests :)
> >
> > ResultCollector rc = getExecution().execute(context ->
> >context.getResultSender().lastResult("done")
> > );
> >
> > -Dan
> >
> >
> > On Thu, Aug 17, 2017 at 9:59 AM, Udo Kohlmeyer 
> > wrote:
> >
> >> I think there might be more to this than just "Data:READ"/"Data:WRITE".
> >> Why would we not support something like
> /*@Authorize(hasRole("functionExecutor"))*/
> >> or /*@Authorize(requiresPermission("DATA:READ"))*/
> >>
> >> The next question in my mind would be, if we get to Lambda-based
> >> functions, how do we specify authorization credentials then? Annotations
> >> are great to "static" definition on services, not so great for more
> >> "dynamic" execution of stuff...
> >>
> >>
> >>
> >> On 8/17/17 09:29, Dan Smith wrote:
> >>
> >>> I like the annotation idea, especially considering that Jared was
> talking
> >>> about adding a @RegisterableFunction annotation as well. maybe
> something
> >>> like @ResourcePermission("Data:READ") or something like that?
> >>>
> >>> -Dan
> >>>
> >>> On Thu, Aug 17, 2017 at 8:18 AM, Michael William Dodge <
> mdo...@pivotal.io
> >>> wrote:
> >>>
> >>> What about an annotation for read-only functions or a subinterface off
> >>>> org.apache.geode.cache.execute.Function?
> >>>>
> >>>> Sarge
> >>>>
> >>>> On 17 Aug, 2017, at 01:42, Swapnil Bawaskar 
> >>>> wrote:
> >>>>
> >>>>> Discuss fix for GEODE-2817
> >>>>> <https://issues.apache.org/jira/browse/GEODE-2817>
> >>>>>
> >>>>> Currently to execute a function, you will need "data:write"
> permission,
> >>>>>
> >>>> but
> >>>>
> >>>>> it really depends on what the function is doing. For example, if a
> >>>>>
> >>>> function
> >>>>
> >>>>> is just reading data, the function author might want users with
> >>>>> DATA:READ
> >>>>> permissions to execute the function. The two options mentioned in the
> >>>>> ticket are:
> >>>>>
> >>>>> 1) externalize SecurityService so that function author can use it in
> the
> >>>>> function.execute code to check authorization.
> >>>>> 2) add a method to function interface to tell the framework what
> >>>>>
> >>>> permission
> >>>>
> >>>>> this function needs to execute, so that the framework will check the
> >>>>> permission before executing the function.
> >>>>>
> >>>>> I vote for #2 because, I think, a function author will be able to
> easily
> >>>>> discover a method on the Function interface, rather than trying to
> look
> >>>>>
> >>>> for
> >>>>
> >>>>> SecurityService.
> >>>>>
> >>>>> I propose that we add the following new method to Function:
> >>>>>
> >>>>> default public List requiredPermissions() {
> >>>>> // default DATA:WRITE
> >>>>> }
> >>>>>
> >>>>> In order to preserve existing behavior, the default required
> permission
> >>>>> would be DATA:WRITE.
> >>>>>
> >>>>
>
>

Re: [Discuss] Enable region set operations to start JTA transactions

[Swapnil Bawaskar]

How is this going to work for a PartitionedRegion?

Are we going to throw TransactionDataNotColocatedException when iterating
over non-colocated data in the PR?
If so, how useful is this work going to be if the set ops always throw
exceptions?
If not, then that will mean we have to maintain a txState on all the
members of the system. This effectively means supporting distributed
transactions.

At this point, I would like to question the ability to perform set ops from
within a transaction. We already do not allow region operations (clear,
destroy etc) from within a transaction, should we disable set ops
(entries(), keys()) etc as well?




On Mon, Aug 28, 2017 at 9:27 AM Michael Stolz  wrote:

> +1 for revising set ops to bootstrap by default. Document the behavior
> change in release notes along with property to revert to old behavior.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: +1-631-835-4771 <(631)%20835-4771>
>
> On Fri, Aug 25, 2017 at 6:41 PM, Anilkumar Gingade 
> wrote:
>
> > +1 Having a consistent behavior for set operation within the transaction
> > (wherever its invoked). And having a gemfire property to allow old way.
> >
> > On Fri, Aug 25, 2017 at 2:49 PM, Eric Shu  wrote:
> >
> > > Hi Team,
> > >
> > > Currently in GEODE JTA implementation, if GEODE transaction is not yet
> > > bootstrapped (there is no TXState created yet for the transaction),
> > region
> > > set operations will not bootstrap the JTA transaction. In other word,
> if
> > > the first operation of the JTA transaction is region set operation, the
> > set
> > > is not in a transactional view (TXState). However, if the JTA
> > > UserTransaction has been bootstrapped (first operation is a get or put
> > > operation, etc), the following region set operation is transactional
> (in
> > > TXState). To some users, this seems to be a bug. Please see the GEODE
> > > ticket (https://issues.apache.org/jira/browse/GEODE-3521).
> > >
> > > Although we intentionally implemented this way to alleviate the memory
> > > requirements for users (each transaction will have a new copy of region
> > > data put into TXState but not if the set op is the first op), we think
> > the
> > > new user has a valid point as well. We propose to allow region set op
> to
> > > bootstrap the transaction. To allow backward compatibility, we will
> > > introduce a new GemFire property to allow the old behavior.
> > >
> > > The question is what should be a default behavior. Do we allow region
> set
> > > op to bootstrap transaction as default? If so, existing customers need
> to
> > > set the new property to get the old behavior. Or we maintain the old
> > > behavior, and require the new users to set the property.
> > >
> > > Please comment if you have any suggestions.
> > >
> > > Thanks,
> > > Eric
> > >
> >
>

Re: 1.3.0 release

[Swapnil Bawaskar]

Sound good.

I would like to volunteer to be the release manager.

Thanks!


On Tue, Sep 12, 2017 at 2:24 PM Anthony Baker  wrote:

> Hi all,
>
> I think we should begin discussing scope and timeline for the 1.3.0
> release.  I know we’re still finalizing 1.2.1, but we released 1.2.0 almost
> two months ago and we’ve fixed almost 200 issues in that time.  IMO, we
> should complete 1.2.1 and then immediately turn around 1.3.0.
>
> Thoughts?  Any volunteers for release manager?
>
> Anthony
>
>

Re: [DISCUSS] authorizing function execution

[Swapnil Bawaskar]

Indeed, the function author has a higher level of privilege than someone
who is invoking a function, that is why the proposal here is to let the
function author choose what level of permissions are required to invoke a
function.

On Thu, Sep 14, 2017 at 11:20 AM Anilkumar Gingade 
wrote:

> >> from reaching into internal classes
> If thats the case; one could do anything, even with read permission...Isn't
> it...
>
>
>
> On Thu, Sep 14, 2017 at 10:43 AM, Jared Stewart 
> wrote:
>
> > There is nothing to prevent code in a function that's executing on a
> > server from reaching into internal classes and bypassing the public
> region
> > APIs. I think a function's author should ultimately determine the
> > permissions required to execute it.
> >
> > > On Sep 14, 2017, at 10:34 AM, Anilkumar Gingade 
> > wrote:
> > >
> > > When a function is accessing/modifying region; the function will be
> doing
> > > so by region apis, don't we have credential check with region apis; if
> > not
> > > can we add those checks here...instead of having it in the function...
> > >
> > > -Anil.
> > >
> > >
> > >
> > > On Wed, Sep 13, 2017 at 11:22 AM, Jared Stewart 
> > wrote:
> > >
> > >> After some more investigation into the implementation details, here is
> > our
> > >> updated proposal to add to the Function interface:
> > >>
> > >> default Collection getRequiredPermissions(
> > Optional
> > >> onRegion) {
> > >>  return Collections.singletonList(ResourcePermissions.DATA_WRITE);
> > >> }
> > >>
> > >> This method can be overridden by Function authors who want to require
> > >> permissions other than DATA:WRITE.. The onRegion parameter will be
> > present
> > >> only when a Function is executed via FunctionService.onRegion, and is
> > >> intended to allow Function authors to require different permissions
> > >> depending on the Region which Function will be executed on.  We pass
> the
> > >> region name into this method rather than the full FunctionContext
> > because
> > >> the latter would be much more expansive to implement.
> > >>
> > >> Any feedback is appreciated.
> > >>
> > >> Thanks,
> > >> Jared
> > >>
> > >>> On Aug 17, 2017, at 1:42 AM, Swapnil Bawaskar 
> > >> wrote:
> > >>>
> > >>> Discuss fix for GEODE-2817
> > >>> <https://issues.apache.org/jira/browse/GEODE-2817>
> > >>>
> > >>> Currently to execute a function, you will need "data:write"
> permission,
> > >> but
> > >>> it really depends on what the function is doing. For example, if a
> > >> function
> > >>> is just reading data, the function author might want users with
> > DATA:READ
> > >>> permissions to execute the function. The two options mentioned in the
> > >>> ticket are:
> > >>>
> > >>> 1) externalize SecurityService so that function author can use it in
> > the
> > >>> function.execute code to check authorization.
> > >>> 2) add a method to function interface to tell the framework what
> > >> permission
> > >>> this function needs to execute, so that the framework will check the
> > >>> permission before executing the function.
> > >>>
> > >>> I vote for #2 because, I think, a function author will be able to
> > easily
> > >>> discover a method on the Function interface, rather than trying to
> look
> > >> for
> > >>> SecurityService.
> > >>>
> > >>> I propose that we add the following new method to Function:
> > >>>
> > >>> default public List requiredPermissions() {
> > >>>  // default DATA:WRITE
> > >>> }
> > >>>
> > >>> In order to preserve existing behavior, the default required
> permission
> > >>> would be DATA:WRITE.
> > >>
> > >>
> >
> >
>

Re: 1.3.0 release

[Swapnil Bawaskar]

I took preliminary look and tagged some issues for 1.3.0.
Looks like we have 11 issues remaining:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-2788&versions=visible&selectedVersion=12340669

Please take a look at these issues to see which are not critical to fix in
1.3 and also look at issues assigned to you/reported by you to see if they
must be tagged for 1.3.

Thanks!



On Fri, Sep 15, 2017 at 10:36 AM Anthony Baker  wrote:

> Excellent!  Can you review the open issues currently tagged for 1.3.0 (I
> think it’s probably not accurate) and gather consensus on any remaining
> changes needed?
>
> Anthony
>
> > On Sep 12, 2017, at 2:40 PM, Swapnil Bawaskar 
> wrote:
> >
> > Sound good.
> >
> > I would like to volunteer to be the release manager.
> >
> > Thanks!
> >
> >
> > On Tue, Sep 12, 2017 at 2:24 PM Anthony Baker  wrote:
> >
> >> Hi all,
> >>
> >> I think we should begin discussing scope and timeline for the 1.3.0
> >> release.  I know we’re still finalizing 1.2.1, but we released 1.2.0
> almost
> >> two months ago and we’ve fixed almost 200 issues in that time.  IMO, we
> >> should complete 1.2.1 and then immediately turn around 1.3.0.
> >>
> >> Thoughts?  Any volunteers for release manager?
> >>
> >> Anthony
> >>
> >>
>
>

Re: [DISCUSS] GEODE-3617 Replace gemfire prefix with geode

[Swapnil Bawaskar]

In the interest of keeping the re-branding tasks together, I took the
liberty to made GEODE-3617 a subtask of
https://issues.apache.org/jira/browse/GEODE-1466.

On Tue, Sep 19, 2017 at 2:05 PM Dave Barnes  wrote:

> Here's an updated link to the properties spreadsheet that should have wider
> viewability:
>
>
> https://docs.google.com/spreadsheets/d/1C2D7cpcYMy5bd0aXXReJn6wWsBuLZrCegl7_cevvKo0/edit?usp=sharing
>
> On Tue, Sep 19, 2017 at 12:10 PM, Dave Barnes  wrote:
>
> > There's no single list of "all properties". Here's one that Tech Pubs
> last
> > reviewed about 6 months ago...
> >
> > https://docs.google.com/spreadsheets/d/1C2D7cpcYMy5bd0aXXReJn6wWsBuLZ
> > rCegl7_cevvKo0/edit#gid=0
> >
> >
> > On Thu, Sep 14, 2017 at 12:06 PM, Bruce Schuchardt <
> bschucha...@pivotal.io
> > > wrote:
> >
> >> There are plenty of places besides DistributionConfig that use "gemfire"
> >> prefixed system properties and there are now also places that use
> "geode"
> >> prefixed system properties.  I think the whole mess needs to be managed
> and
> >> allow either prefix, or even as someone suggested making it plug-able.
> >> There is also an HTML file that, now out of date, lists most of these
> >> system properties.
> >>
> >>
> >>
> >> On 9/14/17 10:14 AM, Darrel Schneider wrote:
> >>
> >>> +1 to having DistributionConfig look for both the "gemfire." and
> "geode."
> >>> prefixes.
> >>> +1 to having DistributionConfig look for both a "gemfire.properties"
> and
> >>> "geode.properties" file.
> >>> Since the geode flavors are newer it should look for them first and
> only
> >>> look for the old gemfire flavor if a geode one is not found.
> >>>
> >>> On Thu, Sep 14, 2017 at 9:35 AM, Kirk Lund  wrote:
> >>>
> >>> That's a bigger change and I'm not sure how you would handle backwards
>  compatibility for users using gemfire.properties.
> 
>  On Thu, Sep 14, 2017 at 9:15 AM, Jacob Barrett 
>  wrote:
> 
>  Or better yet, we stop using properties files already.
> >
> > On Thu, Sep 14, 2017 at 8:55 AM Dave Barnes 
> > wrote:
> >
> > Is there a possibility that the code might find its way into
> additional
> >> contexts with other names? If so, perhaps we should consider a more
> >>
> > generic
> >
> >> identifier, such as PRODUCT_PREFIX.
> >>
> >> On Thu, Sep 14, 2017 at 4:42 AM, Dinesh Akhand  >
> >> wrote:
> >>
> >> Hi,
> >>>
> >>> Why we are keeping gemfire in current geode 1.2 , Can we replace
> this
> >>>
> >> with
> >>
> >>> GEODE
> >>> File : DistributionConfig.java
> >>>
> >>> Current code:
> >>>String GEMFIRE_PREFIX = "gemfire.";
> >>>
> >>> Suggestion to change:
> >>>   String GEODE_PREFIX = "geode.";
> >>>
> >>> Why do you think ?
> >>> Can we go ahead  and change this ?
> >>> It will impact lots of files & all configuration will be now using
> >>>
> >> with
> 
> > geode.
> >>>
> >>> Thanks,
> >>> Dinesh Akhand
> >>> This message and the information contained herein is proprietary
> and
> >>> confidential and subject to the Amdocs policy statement,
> >>>
> >>> you may review at https://www.amdocs.com/about/email-disclaimer <
> >>> https://www.amdocs.com/about/email-disclaimer>
> >>>
> >>>
> >>
> >
>

Re: [DISCUSS] Addition of isValid API to Index interface

[Swapnil Bawaskar]

Sorry for not reading this thread earlier, but I was wondering what is the
point of just invalidating the index and having it lie around if it is not
going to be used?
Can we just drop the index instead, and log a warning message to that
effect? This will free up the memory used by the index and will proactively
let the admin/user know what happened.

On Wed, Sep 20, 2017 at 9:59 AM Nabarun Nag  wrote:

> The PR #768 has been created for this issue and also GEODE-3520 has been
> changed to reflect this requirement.
>
> Regards
> Nabarun
>
> On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag  wrote:
>
> > Thanks you guys for the review. I will revert the GEODE-3520 ticket to
> > reflect that invalidate should happen for both synchronous and
> asynchronous
> > index maintenance.
> > Also, I will resend the PR which reflects the changes mentioned in the
> > ticket
> >
> > Regards
> > Nabarun Nag
> >
> >
> >
> >
> >
> > On Thu, Sep 14, 2017 at 5:55 PM Anilkumar Gingade 
> > wrote:
> >
> >> Dan, you are right...Thanks to Jason, myself and Jason looked into the
> >> code
> >> to see if index is updated before the event/change is saved/injected
> into
> >> the region...It looks like the index update are happening after the
> region
> >> change/update is saved. Moving the index update before that is not an
> easy
> >> task...
> >>
> >> For time, when there is any problem with index update, we can proceed
> with
> >> invalidating the indexes...But we really need to look at making region
> and
> >> index updates in a transactional way, silently invalidating indexes may
> >> not
> >> be acceptable...
> >>
> >> -Anil.
> >>
> >>
> >>
> >>
> >> On Thu, Sep 14, 2017 at 1:12 PM, Dan Smith  wrote:
> >>
> >> > I'm still going to push that we stick with Naba's original proposal.
> >> >
> >> > The current behavior is clearly broken. If one index update fails, an
> >> > exception gets thrown to the user (nice!) but it leaves the put in a
> >> > partially completed state - some other indexes may not have been
> >> updated,
> >> > WAN/AEQs may not have been notified, etc.
> >> >
> >> > We should never leave the system in this corrupted state. It would be
> >> nice
> >> > to be able to cleanly rollback the put, but we don't have that
> >> capability
> >> > especially considering that cache writers have already been invoked.
> So
> >> the
> >> > next best thing is to invalidate the index that failed to update.
> >> >
> >> > Logging an error an allowing the put to succeed does match what we do
> >> with
> >> > CacheListeners. Exceptions from CacheListeners do not fail the put.
> >> >
> >> > -Dan
> >> >
> >> > On Mon, Sep 11, 2017 at 3:29 PM, Jason Huynh 
> wrote:
> >> >
> >> > > Anil, we actually do have a case where the index is out of sync with
> >> the
> >> > > region currently.  It's just not likely to happen but if there is an
> >> > > exception from an index, the end result is that certain indexes get
> >> > updated
> >> > > and the region has already been updated.
> >> > > However the exception is thrown back to the putter, so it becomes
> very
> >> > > obvious something is wrong but I believe Naba has updated the ticket
> >> to
> >> > > show a test that reproduces the problem...
> >> > >
> >> > >
> >> > > On Mon, Sep 11, 2017 at 2:50 PM Anilkumar Gingade <
> >> aging...@pivotal.io>
> >> > > wrote:
> >> > >
> >> > > > The other way to look at it is; what happens to a cache op; when
> >> there
> >> > is
> >> > > > an exception after Region.Entry is created? can it happen? In that
> >> > case,
> >> > > do
> >> > > > we stick the entry into the Cache or not? If an exception is
> >> handled,
> >> > how
> >> > > > is it done, can we look at using the same for Index...
> >> > > >
> >> > > > Also previously, once the valid index is created (verified during
> >> > create
> >> > > or
> >> > > > first put into the cache); we never had any issue where index is
> >> out of
> >> > > > sync with cache...If that changes with new futures (security?)
> then
> >> we
> >> > > may
> >> > > > have to change the expectation with indexing...
> >> > > >
> >> > > > -Anil.
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Sep 11, 2017 at 2:16 PM, Anthony Baker  >
> >> > > wrote:
> >> > > >
> >> > > > > I’m confused.  Once a cache update has been distributed to other
> >> > > members
> >> > > > > it can’t be undone.  That update could have triggered myriad
> other
> >> > > > > application behaviors.
> >> > > > >
> >> > > > > Anthony
> >> > > > >
> >> > > > > > On Sep 11, 2017, at 2:04 PM, Michael Stolz  >
> >> > > wrote:
> >> > > > > >
> >> > > > > > Great, that's exactly the behavior I would expect.
> >> > > > > >
> >> > > > > > Thanks.
> >> > > > > >
> >> > > > > > --
> >> > > > > > Mike Stolz
> >> > > > > > Principal Engineer, GemFire Product Manager
> >> > > > > > Mobile: +1-631-835-4771 <(631)%20835-4771>
> <(631)%20835-4771> <(631)%20835-4771>
> >> > > > > >
> >> > > > > > On Mon, Sep 11, 2017 at 4:34 PM, Jason Huynh <
> jhu...@pivotal.io
> >> >
> >>

Re: [DISCUSS] Addition of isValid API to Index interface

[Swapnil Bawaskar]

Thanks for the explanation Naba, please find my reply below:
1. Debugging: If we log a warning, that should get noticed immediately, so
I don't think we need to worry about logs rolling.
2. Performance for a single put: We can always schedule an async task to
drop the index.

On Wed, Sep 20, 2017 at 3:49 PM Nabarun Nag  wrote:

> Hi Swapnil,
> There were few factors we considered before going with just invalidating
> the index rather than destroying the index.
> 1. Debugging reasons : If the indexes were destroyed and logs roll over,
> and suddenly we see that indexes have disappeared, it will be tough to
> differentiate between whether the indexes were created improperly in the
> first place [/ restarts] or if a put corrupted it hence it was destroyed.
> In case of marking it invalid, we know for sure that a put has corrupted
> the index and prevent confusion for the user.
>
> 2. Performance perspective  : We were worried that if a put corrupts
> multiple indexes and then that put is also responsible for destroying those
> indexes may slow down the execution as it will have to acquire/release
> locks to destroy indexes [especially in case of PR indexes]. We were also
> worried about race conditions arising in that case.
>
> Regards
> Nabarun
>
> On Wed, Sep 20, 2017 at 2:59 PM Swapnil Bawaskar 
> wrote:
>
> > Sorry for not reading this thread earlier, but I was wondering what is
> the
> > point of just invalidating the index and having it lie around if it is
> not
> > going to be used?
> > Can we just drop the index instead, and log a warning message to that
> > effect? This will free up the memory used by the index and will
> proactively
> > let the admin/user know what happened.
> >
> > On Wed, Sep 20, 2017 at 9:59 AM Nabarun Nag  wrote:
> >
> > > The PR #768 has been created for this issue and also GEODE-3520 has
> been
> > > changed to reflect this requirement.
> > >
> > > Regards
> > > Nabarun
> > >
> > > On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag  wrote:
> > >
> > > > Thanks you guys for the review. I will revert the GEODE-3520 ticket
> to
> > > > reflect that invalidate should happen for both synchronous and
> > > asynchronous
> > > > index maintenance.
> > > > Also, I will resend the PR which reflects the changes mentioned in
> the
> > > > ticket
> > > >
> > > > Regards
> > > > Nabarun Nag
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Sep 14, 2017 at 5:55 PM Anilkumar Gingade <
> aging...@pivotal.io
> > >
> > > > wrote:
> > > >
> > > >> Dan, you are right...Thanks to Jason, myself and Jason looked into
> the
> > > >> code
> > > >> to see if index is updated before the event/change is saved/injected
> > > into
> > > >> the region...It looks like the index update are happening after the
> > > region
> > > >> change/update is saved. Moving the index update before that is not
> an
> > > easy
> > > >> task...
> > > >>
> > > >> For time, when there is any problem with index update, we can
> proceed
> > > with
> > > >> invalidating the indexes...But we really need to look at making
> region
> > > and
> > > >> index updates in a transactional way, silently invalidating indexes
> > may
> > > >> not
> > > >> be acceptable...
> > > >>
> > > >> -Anil.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Thu, Sep 14, 2017 at 1:12 PM, Dan Smith 
> wrote:
> > > >>
> > > >> > I'm still going to push that we stick with Naba's original
> proposal.
> > > >> >
> > > >> > The current behavior is clearly broken. If one index update fails,
> > an
> > > >> > exception gets thrown to the user (nice!) but it leaves the put
> in a
> > > >> > partially completed state - some other indexes may not have been
> > > >> updated,
> > > >> > WAN/AEQs may not have been notified, etc.
> > > >> >
> > > >> > We should never leave the system in this corrupted state. It would
> > be
> > > >> nice
> > > >> > to be able to cleanly rollback the put, but we don't have that
> > > >> capability
> > > >

Re: [DISCUSS] Addition of isValid API to Index interface

[Swapnil Bawaskar]

Can anyone help me understand the race condition?
My understanding was that in addition to setting the isInvalid flag, we
fire off a background task to drop the index. This task can use the
plumbing that gfsh uses to drop the index. Even if we schedule more than
one of these tasks, the later tasks would become no-op.
In my opinion we should do this optimization now, but if others disagree, I
will file an improvement jira.


On Thu, Sep 21, 2017 at 10:45 AM Jason Huynh  wrote:

> I agree with Dan and Naba.  The possible race conditions seem very risky to
> me.  Memory optimization isn't really needed here because the user has
> already budgeted memory for the index.  They probably weren't trying to
> create an index that would be corrupted and when it does get corrupted, it
> won't take any more memory than they had originally expected from an index.
>
>
>
> On Thu, Sep 21, 2017 at 10:25 AM Dan Smith  wrote:
>
> > Scheduling an asynchronous task to drop the index everywhere seems like
> > it's adding a lot of complexity and potential for race conditions. Is it
> > really worth optimizing memory usage in this edge case where we hit an
> > exception updating an index? The objective of these changes is to fix a
> > window where we are leaving the system in a corrupt state, I think we
> > should do that in the least complicated way possible.
> >
> > -Dan
> >
> > On Wed, Sep 20, 2017 at 3:54 PM, Swapnil Bawaskar 
> > wrote:
> >
> > > Thanks for the explanation Naba, please find my reply below:
> > > 1. Debugging: If we log a warning, that should get noticed immediately,
> > so
> > > I don't think we need to worry about logs rolling.
> > > 2. Performance for a single put: We can always schedule an async task
> to
> > > drop the index.
> > >
> > > On Wed, Sep 20, 2017 at 3:49 PM Nabarun Nag  wrote:
> > >
> > > > Hi Swapnil,
> > > > There were few factors we considered before going with just
> > invalidating
> > > > the index rather than destroying the index.
> > > > 1. Debugging reasons : If the indexes were destroyed and logs roll
> > over,
> > > > and suddenly we see that indexes have disappeared, it will be tough
> to
> > > > differentiate between whether the indexes were created improperly in
> > the
> > > > first place [/ restarts] or if a put corrupted it hence it was
> > destroyed.
> > > > In case of marking it invalid, we know for sure that a put has
> > corrupted
> > > > the index and prevent confusion for the user.
> > > >
> > > > 2. Performance perspective  : We were worried that if a put corrupts
> > > > multiple indexes and then that put is also responsible for destroying
> > > those
> > > > indexes may slow down the execution as it will have to
> acquire/release
> > > > locks to destroy indexes [especially in case of PR indexes]. We were
> > also
> > > > worried about race conditions arising in that case.
> > > >
> > > > Regards
> > > > Nabarun
> > > >
> > > > On Wed, Sep 20, 2017 at 2:59 PM Swapnil Bawaskar <
> sbawas...@pivotal.io
> > >
> > > > wrote:
> > > >
> > > > > Sorry for not reading this thread earlier, but I was wondering what
> > is
> > > > the
> > > > > point of just invalidating the index and having it lie around if it
> > is
> > > > not
> > > > > going to be used?
> > > > > Can we just drop the index instead, and log a warning message to
> that
> > > > > effect? This will free up the memory used by the index and will
> > > > proactively
> > > > > let the admin/user know what happened.
> > > > >
> > > > > On Wed, Sep 20, 2017 at 9:59 AM Nabarun Nag 
> wrote:
> > > > >
> > > > > > The PR #768 has been created for this issue and also GEODE-3520
> has
> > > > been
> > > > > > changed to reflect this requirement.
> > > > > >
> > > > > > Regards
> > > > > > Nabarun
> > > > > >
> > > > > > On Thu, Sep 14, 2017 at 5:29 PM Nabarun Nag 
> > wrote:
> > > > > >
> > > > > > > Thanks you guys for the review. I will revert the GEODE-3520
> > ticket
> > > > to
> > > > > > > reflect that invalidate should happen for both synchronous and
> > > > > &

Re: New Geode PMC Member: Galen O'Sullivan

[Swapnil Bawaskar]

Welcome Galen!

On Tue, Sep 26, 2017 at 12:46 PM Alexander Murmann 
wrote:

> Congratulations, Galen!
>
> On Tue, Sep 26, 2017 at 10:48 AM, Mark Bretl  wrote:
>
> > The Apache Geode Project Management Committee has invited Galen
> > O'Sullivan to join the Geode PMC and we are pleased to announce he has
> > accepted.
> >
> > Please join me in welcoming Galen!
> >
> > Best regards,
> >
> > Mark
> > On behalf of the Apache Geode PMC
> >
>

Re: New Geode PMC Member: Galen O'Sullivan

[Swapnil Bawaskar]

We have a list of committers here: http://geode.apache.org/community/ but
we don't have a list of PMC on the geode site.
Mark, do you maintain one?

On Wed, Sep 27, 2017 at 6:09 AM Michael Stolz  wrote:

> Where can I see the list of PMC members?
>
> --
> Mike Stolz
> Principal Engineer - Gemfire Product Manager
> Mobile: 631-835-4771 <(631)%20835-4771>
>
> On Sep 26, 2017 7:49 PM, "Mark Bretl"  wrote:
>
> > The Apache Geode Project Management Committee has invited Galen
> > O'Sullivan to join the Geode PMC and we are pleased to announce he has
> > accepted.
> >
> > Please join me in welcoming Galen!
> >
> > Best regards,
> >
> > Mark
> > On behalf of the Apache Geode PMC
> >
>

Re: 1.3.0 release

[Swapnil Bawaskar]

Hi All,
We actually have gone up from 11 to 15 issues tagged for release with 1.3.
Based on recent activity (or lack there of) and features not related to
Security, I think we should not wait for the following issues for 1.3: (I
will remove 1.3 labels for these if there are no concerns in 72 hours)
GEODE-3563 <https://issues.apache.org/jira/browse/GEODE-3563> SSL socket
handling problems in TCPConduit run
GEODE-3521 <https://issues.apache.org/jira/browse/GEODE-3521> Allow region
set op to bootstrap JTA
GEODE-3622 <https://issues.apache.org/jira/browse/GEODE-3622> The first
HeapLRU evictions on large region can consume high amounts of CPU
GEODE-3705 <https://issues.apache.org/jira/browse/GEODE-3705> New client
protocol: Implement handshake
GEODE-3682 <https://issues.apache.org/jira/browse/GEODE-3682> Trace
displaying incorrect indexes being used
GEODE-3637 <https://issues.apache.org/jira/browse/GEODE-3637>
configureClientSSLSocket
call can block Acceptor thread

Which brings us down to the following 8:
GEODE-2817 <https://issues.apache.org/jira/browse/GEODE-2817> Have the
function author determine what permissions the function execution requires
GEODE-2919 <https://issues.apache.org/jira/browse/GEODE-2919> Provide finer
grained security
GEODE-3190 <https://issues.apache.org/jira/browse/GEODE-3190> CI failure:
org.apache.geode.internal.cache.Bug48182JUnitTest.test48182WithRegionDestroy
GEODE-3495 <https://issues.apache.org/jira/browse/GEODE-3495> Review and
update dependencies for 1.3.0
GEODE-3621 <https://issues.apache.org/jira/browse/GEODE-3621> Revert
breaking changes in SecurityManager
GEODE-3628 <https://issues.apache.org/jira/browse/GEODE-3628> fix required
permission for lucene query
GEODE-3685 <https://issues.apache.org/jira/browse/GEODE-3685> MBean
wrappers are not always applied correctly
GEODE-3723 <https://issues.apache.org/jira/browse/GEODE-3723> Reconsider
using Optional as the parameter for getRequiredPermissions


On Fri, Sep 15, 2017 at 1:11 PM Swapnil Bawaskar 
wrote:

> I took preliminary look and tagged some issues for 1.3.0.
> Looks like we have 11 issues remaining:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-2788&versions=visible&selectedVersion=12340669
>
> Please take a look at these issues to see which are not critical to fix in
> 1.3 and also look at issues assigned to you/reported by you to see if they
> must be tagged for 1.3.
>
> Thanks!
>
>
>
> On Fri, Sep 15, 2017 at 10:36 AM Anthony Baker  wrote:
>
>> Excellent!  Can you review the open issues currently tagged for 1.3.0 (I
>> think it’s probably not accurate) and gather consensus on any remaining
>> changes needed?
>>
>> Anthony
>>
>> > On Sep 12, 2017, at 2:40 PM, Swapnil Bawaskar 
>> wrote:
>> >
>> > Sound good.
>> >
>> > I would like to volunteer to be the release manager.
>> >
>> > Thanks!
>> >
>> >
>> > On Tue, Sep 12, 2017 at 2:24 PM Anthony Baker 
>> wrote:
>> >
>> >> Hi all,
>> >>
>> >> I think we should begin discussing scope and timeline for the 1.3.0
>> >> release.  I know we’re still finalizing 1.2.1, but we released 1.2.0
>> almost
>> >> two months ago and we’ve fixed almost 200 issues in that time.  IMO, we
>> >> should complete 1.2.1 and then immediately turn around 1.3.0.
>> >>
>> >> Thoughts?  Any volunteers for release manager?
>> >>
>> >> Anthony
>> >>
>> >>
>>
>>

Re: New client protocol configuration

[Swapnil Bawaskar]

As a geode admin setting up the cluster with security, I don't want to
worry about what version of protocol the client is going to use.
+1 for the new protocol to just use existing properties.

On Mon, Oct 2, 2017 at 1:24 PM Dan Smith  wrote:

> I realized I've been assuming you were asking about turning on ssl
> authentication. Maybe you are talking about authenticating with the
> security manager. Either way, what Anthony said still applies - the
> new protocol should just use the existing properties (security-manager
> in that case).
>
> -Dan
>
> On Mon, Oct 2, 2017 at 12:57 PM, John Blum  wrote:
> > I don't mean to derail the topic at hand, but...
> >
> > On the same vain as Properties, can we also stop talking about XML?  I
> much
> > prefer Properties over XML any day, especially given YAML.  However, that
> > does not imply Properties should be added at will.  Properties also
> > increase the "surface area" of the public API as well.
> >
> > Also, the API and XML are not on even plane; not even close.
> >
> > IMO, the API should be the primary means to configure a feature; all
> other
> > configuration options are secondary and optional (as needed).
> >
> > Therefore, given an API-first approach, the other configuration formats
> and
> > options become more apparent (providing the API was designed with the
> right
> > abstractions in the first place).
> >
> > $.0.02,
> > -j
> >
> >
> >
> > On Mon, Oct 2, 2017 at 12:18 PM, Dan Smith  wrote:
> >
> >> One thing to think about - if the new protocol doesn't support two-way
> >> authentication maybe we should throw an exception if the user sets
> >> ssl-require-authentication=true? We definitely don't want to lie to
> >> the user and pretend that we are providing some level of security
> >> which we are not.
> >>
> >> I'm assuming the new protocol will also need to read the ssl-ciphers,
> >> ssl-protocols, ssl-keystore and ssl-truststore settings.
> >>
> >> -Dan
> >>
> >> On Mon, Oct 2, 2017 at 12:08 PM, Anthony Baker 
> wrote:
> >> > Is there a need for property yet?
> >> >
> >> > The authentication-enabled question could be answered from the
> existing
> >> security properties.  That ensures consistency and means a user would
> only
> >> need to set a single switch.
> >> >
> >> > If we only support a single authentication mode, we can defer adding
> >> configration until we need it.
> >> >
> >> > Anthony
> >> >
> >> >> On Oct 2, 2017, at 11:56 AM, Galen O'Sullivan  >
> >> wrote:
> >> >>
> >> >> Currently, we have a setting for the new client protocol that
> controls
> >> >> whether authentication is required or not. We expect to expand this
> in
> >> the
> >> >> future, and also that there may be more configuration options for the
> >> >> protocol. We would like to namespace the settings for this protocol
> but
> >> >> don't really have a good name for the protocol.
> >> >>
> >> >> We're expecting to do configuration via gemfire.properties -- I hear
> >> that's
> >> >> the right place to put these things. It looks like the setting would
> >> take a
> >> >> form like `geode.new-client-protocol.authentication-mode`. "New"
> client
> >> >> protocol is not a good name because it will be outdated before long.
> >> It's
> >> >> not the only client protocol, so "client-protocol" would be
> misleading.
> >> Any
> >> >> other ideas?
> >> >>
> >> >> Thanks,
> >> >> Galen
> >> >
> >>
> >
> >
> >
> > --
> > -John
> > john.blum10101 (skype)
>

Re: 1.3.0 release

[Swapnil Bawaskar]

Hi All,
Thanks for all your efforts in resolving a whopping 51 issues in the past
week! Since I did not hear any concerns, I removed 1.3 label from some of
the issues mentioned below, along with some CI failure issues. This now
brings us down to only 2:
GEODE-3247: Improve OQL expression execution
GEODE-3743: Deprecate option for manual restart of Gateway senders

For GEODE-3247 the implementation is complete, documentation pending. I
hope to cut the release branch and have a release candidate early next week.

Thanks!
Swapnil.


On Sun, Oct 1, 2017 at 9:31 AM Swapnil Bawaskar 
wrote:

> Hi All,
> We actually have gone up from 11 to 15 issues tagged for release with 1.3.
> Based on recent activity (or lack there of) and features not related to
> Security, I think we should not wait for the following issues for 1.3: (I
> will remove 1.3 labels for these if there are no concerns in 72 hours)
> GEODE-3563 <https://issues.apache.org/jira/browse/GEODE-3563> SSL socket
> handling problems in TCPConduit run
> GEODE-3521 <https://issues.apache.org/jira/browse/GEODE-3521> Allow
> region set op to bootstrap JTA
> GEODE-3622 <https://issues.apache.org/jira/browse/GEODE-3622> The first
> HeapLRU evictions on large region can consume high amounts of CPU
> GEODE-3705 <https://issues.apache.org/jira/browse/GEODE-3705> New client
> protocol: Implement handshake
> GEODE-3682 <https://issues.apache.org/jira/browse/GEODE-3682> Trace
> displaying incorrect indexes being used
> GEODE-3637 <https://issues.apache.org/jira/browse/GEODE-3637> 
> configureClientSSLSocket
> call can block Acceptor thread
>
> Which brings us down to the following 8:
> GEODE-2817 <https://issues.apache.org/jira/browse/GEODE-2817> Have the
> function author determine what permissions the function execution requires
> GEODE-2919 <https://issues.apache.org/jira/browse/GEODE-2919> Provide
> finer grained security
> GEODE-3190 <https://issues.apache.org/jira/browse/GEODE-3190> CI failure:
> org.apache.geode.internal.cache.Bug48182JUnitTest.test48182WithRegionDestroy
> GEODE-3495 <https://issues.apache.org/jira/browse/GEODE-3495> Review and
> update dependencies for 1.3.0
> GEODE-3621 <https://issues.apache.org/jira/browse/GEODE-3621> Revert
> breaking changes in SecurityManager
> GEODE-3628 <https://issues.apache.org/jira/browse/GEODE-3628> fix
> required permission for lucene query
> GEODE-3685 <https://issues.apache.org/jira/browse/GEODE-3685> MBean
> wrappers are not always applied correctly
> GEODE-3723 <https://issues.apache.org/jira/browse/GEODE-3723> Reconsider
> using Optional as the parameter for getRequiredPermissions
>
>
>
> On Fri, Sep 15, 2017 at 1:11 PM Swapnil Bawaskar 
> wrote:
>
>> I took preliminary look and tagged some issues for 1.3.0.
>> Looks like we have 11 issues remaining:
>>
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-2788&versions=visible&selectedVersion=12340669
>>
>> Please take a look at these issues to see which are not critical to fix
>> in 1.3 and also look at issues assigned to you/reported by you to see if
>> they must be tagged for 1.3.
>>
>> Thanks!
>>
>>
>>
>> On Fri, Sep 15, 2017 at 10:36 AM Anthony Baker  wrote:
>>
>>> Excellent!  Can you review the open issues currently tagged for 1.3.0 (I
>>> think it’s probably not accurate) and gather consensus on any remaining
>>> changes needed?
>>>
>>> Anthony
>>>
>>> > On Sep 12, 2017, at 2:40 PM, Swapnil Bawaskar 
>>> wrote:
>>> >
>>> > Sound good.
>>> >
>>> > I would like to volunteer to be the release manager.
>>> >
>>> > Thanks!
>>> >
>>> >
>>> > On Tue, Sep 12, 2017 at 2:24 PM Anthony Baker 
>>> wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> I think we should begin discussing scope and timeline for the 1.3.0
>>> >> release.  I know we’re still finalizing 1.2.1, but we released 1.2.0
>>> almost
>>> >> two months ago and we’ve fixed almost 200 issues in that time.  IMO,
>>> we
>>> >> should complete 1.2.1 and then immediately turn around 1.3.0.
>>> >>
>>> >> Thoughts?  Any volunteers for release manager?
>>> >>
>>> >> Anthony
>>> >>
>>> >>
>>>
>>>

Re: 1.3.0 release

[Swapnil Bawaskar]

All issues targeted for 1.3.0 have been resolved. I will cut a release
candidate shortly.

On Sat, Oct 7, 2017 at 11:18 AM Swapnil Bawaskar 
wrote:

> Hi All,
> Thanks for all your efforts in resolving a whopping 51 issues in the past
> week! Since I did not hear any concerns, I removed 1.3 label from some of
> the issues mentioned below, along with some CI failure issues. This now
> brings us down to only 2:
> GEODE-3247: Improve OQL expression execution
> GEODE-3743: Deprecate option for manual restart of Gateway senders
>
> For GEODE-3247 the implementation is complete, documentation pending. I
> hope to cut the release branch and have a release candidate early next week.
>
> Thanks!
> Swapnil.
>
>
> On Sun, Oct 1, 2017 at 9:31 AM Swapnil Bawaskar 
> wrote:
>
>> Hi All,
>> We actually have gone up from 11 to 15 issues tagged for release with
>> 1.3. Based on recent activity (or lack there of) and features not related
>> to Security, I think we should not wait for the following issues for 1.3:
>> (I will remove 1.3 labels for these if there are no concerns in 72 hours)
>> GEODE-3563 <https://issues.apache.org/jira/browse/GEODE-3563> SSL socket
>> handling problems in TCPConduit run
>> GEODE-3521 <https://issues.apache.org/jira/browse/GEODE-3521> Allow
>> region set op to bootstrap JTA
>> GEODE-3622 <https://issues.apache.org/jira/browse/GEODE-3622> The first
>> HeapLRU evictions on large region can consume high amounts of CPU
>> GEODE-3705 <https://issues.apache.org/jira/browse/GEODE-3705> New client
>> protocol: Implement handshake
>> GEODE-3682 <https://issues.apache.org/jira/browse/GEODE-3682> Trace
>> displaying incorrect indexes being used
>> GEODE-3637 <https://issues.apache.org/jira/browse/GEODE-3637> 
>> configureClientSSLSocket
>> call can block Acceptor thread
>>
>> Which brings us down to the following 8:
>> GEODE-2817 <https://issues.apache.org/jira/browse/GEODE-2817> Have the
>> function author determine what permissions the function execution requires
>> GEODE-2919 <https://issues.apache.org/jira/browse/GEODE-2919> Provide
>> finer grained security
>> GEODE-3190 <https://issues.apache.org/jira/browse/GEODE-3190> CI
>> failure:
>> org.apache.geode.internal.cache.Bug48182JUnitTest.test48182WithRegionDestroy
>> GEODE-3495 <https://issues.apache.org/jira/browse/GEODE-3495> Review and
>> update dependencies for 1.3.0
>> GEODE-3621 <https://issues.apache.org/jira/browse/GEODE-3621> Revert
>> breaking changes in SecurityManager
>> GEODE-3628 <https://issues.apache.org/jira/browse/GEODE-3628> fix
>> required permission for lucene query
>> GEODE-3685 <https://issues.apache.org/jira/browse/GEODE-3685> MBean
>> wrappers are not always applied correctly
>> GEODE-3723 <https://issues.apache.org/jira/browse/GEODE-3723> Reconsider
>> using Optional as the parameter for getRequiredPermissions
>>
>>
>>
>> On Fri, Sep 15, 2017 at 1:11 PM Swapnil Bawaskar 
>> wrote:
>>
>>> I took preliminary look and tagged some issues for 1.3.0.
>>> Looks like we have 11 issues remaining:
>>>
>>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-2788&versions=visible&selectedVersion=12340669
>>>
>>> Please take a look at these issues to see which are not critical to fix
>>> in 1.3 and also look at issues assigned to you/reported by you to see if
>>> they must be tagged for 1.3.
>>>
>>> Thanks!
>>>
>>>
>>>
>>> On Fri, Sep 15, 2017 at 10:36 AM Anthony Baker 
>>> wrote:
>>>
>>>> Excellent!  Can you review the open issues currently tagged for 1.3.0
>>>> (I think it’s probably not accurate) and gather consensus on any remaining
>>>> changes needed?
>>>>
>>>> Anthony
>>>>
>>>> > On Sep 12, 2017, at 2:40 PM, Swapnil Bawaskar 
>>>> wrote:
>>>> >
>>>> > Sound good.
>>>> >
>>>> > I would like to volunteer to be the release manager.
>>>> >
>>>> > Thanks!
>>>> >
>>>> >
>>>> > On Tue, Sep 12, 2017 at 2:24 PM Anthony Baker 
>>>> wrote:
>>>> >
>>>> >> Hi all,
>>>> >>
>>>> >> I think we should begin discussing scope and timeline for the 1.3.0
>>>> >> release.  I know we’re still finalizing 1.2.1, but we released 1.2.0
>>>> almost
>>>> >> two months ago and we’ve fixed almost 200 issues in that time.  IMO,
>>>> we
>>>> >> should complete 1.2.1 and then immediately turn around 1.3.0.
>>>> >>
>>>> >> Thoughts?  Any volunteers for release manager?
>>>> >>
>>>> >> Anthony
>>>> >>
>>>> >>
>>>>
>>>>

[VOTE] Apache Geode release - 1.3.0 RC1

[Swapnil Bawaskar]

This is the first release candidate for Apache Geode, version 1.3.0. I will
send out a separate vote thread for geode-examples.
Thanks to all the community members for their contributions to this
release!

*** Please download, test and vote by Monday, October 16, 0800 hrs
US Pacific. ***

It fixes 477 issues. release notes can be found at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669

Note that we are voting upon the source tags:  rel/v1.3.0.RC1
https://github.com/apache/geode/tree/rel/v1.3.0.RC1

Commit ID:
95bc7b6eaaff9cc5a200497eb3d0db5019a06bc2 (geode)

Source and binary files:
 https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC1

Maven staging repo:
 https://repository.apache.org/content/repositories/orgapachegeode-1028

Geode's KEYS file containing PGP keys we use to sign the release:
 https://github.com/apache/geode/blob/develop/KEYS

Release Signed with Key: pub 4096R/18F902DB 2016-04-07
Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

Re: [VOTE] Apache Geode release - 1.3.0 RC1

[Swapnil Bawaskar]

I am cancelling this vote because uploading the archives to maven staging
repo failed.
The error was: Missing Signature:
'/org/apache/geode/apache-geode/1.3.0/apache-geode-1.3.0.tar.gz.asc' does
not exist for 'apache-geode-1.3.0.tar.gz' even though that file was created
locally. I will dig into it tomorrow.

On Thu, Oct 12, 2017 at 12:43 AM Swapnil Bawaskar 
wrote:

> This is the first release candidate for Apache Geode, version 1.3.0. I
> will send out a separate vote thread for geode-examples.
> Thanks to all the community members for their contributions to this
> release!
>
> *** Please download, test and vote by Monday, October 16, 0800 hrs
> US Pacific. ***
>
> It fixes 477 issues. release notes can be found at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669
>
> Note that we are voting upon the source tags:  rel/v1.3.0.RC1
> https://github.com/apache/geode/tree/rel/v1.3.0.RC1
>
> Commit ID:
> 95bc7b6eaaff9cc5a200497eb3d0db5019a06bc2 (geode)
>
> Source and binary files:
>  https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC1
>
> Maven staging repo:
>  https://repository.apache.org/content/repositories/orgapachegeode-1028
>
> Geode's KEYS file containing PGP keys we use to sign the release:
>  https://github.com/apache/geode/blob/develop/KEYS
>
> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
>

Re: version for JIRA tickets?

[Swapnil Bawaskar]

Yes, please do Bruce. Thanks!

On Fri, Oct 13, 2017 at 10:07 AM Bruce Schuchardt 
wrote:

> Shall I bump the actual product version to 1.4.0?  Swapnil bumped the
> version that gradle knows about but didn't update Version.java
>
>
> On 10/13/17 8:24 AM, Anthony Baker wrote:
> > Fixed!
> >
> > https://issues.apache.org/jira/projects/GEODE/versions/12341842 <
> https://issues.apache.org/jira/projects/GEODE/versions/12341842>
> >
> > Anthony
> >
> >> On Oct 13, 2017, at 8:16 AM, Bruce Schuchardt 
> wrote:
> >>
> >> I see that the build version has been bumped to 1.4.0 on develop but I
> don't see a 1.4.0 on JIRA & have a ticket to close today.
> >>
> >
>
>

Re: version for JIRA tickets?

[Swapnil Bawaskar]

Kirk, you can checkout release/1.3.0 to verify.

On Fri, Oct 13, 2017 at 10:05 AM Kirk Lund  wrote:

> I have a couple commits from yesterday (and maybe 1 from Wed) that I still
> need to close the corresponding Jira ticket for. How do I determine if the
> commit made it into 1.3 or will instead be in 1.4?
>
> On Fri, Oct 13, 2017 at 8:35 AM, Bruce Schuchardt 
> wrote:
>
> > That was fast!  Thanks Anthony
> >
> >
> >
> > On 10/13/17 8:24 AM, Anthony Baker wrote:
> >
> >> Fixed!
> >>
> >> https://issues.apache.org/jira/projects/GEODE/versions/12341842 <
> >> https://issues.apache.org/jira/projects/GEODE/versions/12341842>
> >>
> >> Anthony
> >>
> >> On Oct 13, 2017, at 8:16 AM, Bruce Schuchardt 
> >>> wrote:
> >>>
> >>> I see that the build version has been bumped to 1.4.0 on develop but I
> >>> don't see a 1.4.0 on JIRA & have a ticket to close today.
> >>>
> >>>
> >>
> >
>

[VOTE] Apache Geode release - 1.3.0 RC2

[Swapnil Bawaskar]

This is the second release candidate for Apache Geode, version 1.3.0. I
will send out a separate vote thread for geode-examples.
Thanks to all the community members for their contributions to this
release!

*** Please download, test and vote by Monday, October 16, 0800 hrs
US Pacific. ***

It fixes 376 issues. release notes can be found at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669

Note that we are voting upon the source tags:  rel/v1.3.0.RC2
https://github.com/apache/geode/tree/rel/v1.3.0.RC2

Commit ID:
9e076738fc2ae40f95bd179b5c1624e664a28d61

Source and binary files:
 https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC2

Maven staging repo:
 https://repository.apache.org/content/repositories/orgapachegeode-1033

Geode's KEYS file containing PGP keys we use to sign the release:
 https://github.com/apache/geode/blob/develop/KEYS

Release Signed with Key: pub 4096R/18F902DB 2016-04-07
Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

Re: [VOTE] Apache Geode release - 1.3.0 RC2

[Swapnil Bawaskar]

I forgot to update the vote deadline. Please vote by Wednesday 10/18 by
8:00am PST.

Thanks!


On Fri, Oct 13, 2017 at 1:39 PM Swapnil Bawaskar 
wrote:

> This is the second release candidate for Apache Geode, version 1.3.0. I
> will send out a separate vote thread for geode-examples.
> Thanks to all the community members for their contributions to this
> release!
>
> *** Please download, test and vote by Monday, October 16, 0800 hrs
> US Pacific. ***
>
> It fixes 376 issues. release notes can be found at:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669
>
> Note that we are voting upon the source tags:  rel/v1.3.0.RC2
> https://github.com/apache/geode/tree/rel/v1.3.0.RC2
>
> Commit ID:
> 9e076738fc2ae40f95bd179b5c1624e664a28d61
>
> Source and binary files:
>  https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC2
>
> Maven staging repo:
>  https://repository.apache.org/content/repositories/orgapachegeode-1033
>
> Geode's KEYS file containing PGP keys we use to sign the release:
>  https://github.com/apache/geode/blob/develop/KEYS
>
> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
>

Re: [VOTE] Apache Geode release - 1.3.0 RC2

[Swapnil Bawaskar]

Since geode-examples is a separate repo, I would think that it enables us
to release independently. Can you elaborate on why not including examples
makes it -1 for you?

Yes, I will update geode-examples to expect a tgz filename before cutting a
1.3.0 release of examples.

On Sun, Oct 15, 2017 at 8:52 PM Anthony Baker  wrote:

> -1
>
> The geode-examples source should be included in this release.
>
> The download task in the geode-examples build also needs to be updated
> since the distribution archive name has changed from .tar.gz to .tgz.  Are
> there any other downstream dependencies affected by this change?
>
> Anthony
>
>
> > On Oct 13, 2017, at 1:39 PM, Swapnil Bawaskar 
> wrote:
> >
> > This is the second release candidate for Apache Geode, version 1.3.0. I
> > will send out a separate vote thread for geode-examples.
> > Thanks to all the community members for their contributions to this
> > release!
> >
> > *** Please download, test and vote by Monday, October 16, 0800 hrs
> > US Pacific. ***
> >
> > It fixes 376 issues. release notes can be found at:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669
> >
> > Note that we are voting upon the source tags:  rel/v1.3.0.RC2
> > https://github.com/apache/geode/tree/rel/v1.3.0.RC2
> >
> > Commit ID:
> > 9e076738fc2ae40f95bd179b5c1624e664a28d61
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC2
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachegeode-1033
> >
> > Geode's KEYS file containing PGP keys we use to sign the release:
> > https://github.com/apache/geode/blob/develop/KEYS
> >
> > Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> > Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
>
>

Re: [VOTE] Apache Geode release - 1.3.0 RC2

[Swapnil Bawaskar]

Ok, I will cut another release candidate. This vote is cancelled. However,
we don't really ship the examples in the geode distribution (the examples
is a different set of files: binary and src distributions). So, voting for
1.3 of geode, followed by voting for 1.3 of examples, I think should have
worked.

On Tue, Oct 17, 2017 at 2:33 PM Jared Stewart  wrote:

> If we do end up cutting a new RC, I would like to add @Experimental to
> GfshRule since it is going to be made public for the first time and I think
> the API is still being hammered out.  Any thoughts?
>
> - Jared
>
> > On Oct 17, 2017, at 2:04 PM, Dan Smith  wrote:
> >
> > -1 without the examples.
> >
> > We *could* theoretically be releasing the examples separately, but that's
> > not our current release process. The release steps include releasing the
> > examples -
> https://cwiki.apache.org/confluence/display/GEODE/Release+Steps.
> >
> > I don't think the examples release needs to wait for the outstanding PRs,
> > unless any of those are critical for 1.3.0?
> >
> > -Dan
> >
> > On Tue, Oct 17, 2017 at 11:20 AM, Swapnil Bawaskar  >
> > wrote:
> >
> >> Since geode-examples is a separate repo, I would think that it enables
> us
> >> to release independently. Can you elaborate on why not including
> examples
> >> makes it -1 for you?
> >>
> >> Yes, I will update geode-examples to expect a tgz filename before
> cutting a
> >> 1.3.0 release of examples.
> >>
> >> On Sun, Oct 15, 2017 at 8:52 PM Anthony Baker 
> wrote:
> >>
> >>> -1
> >>>
> >>> The geode-examples source should be included in this release.
> >>>
> >>> The download task in the geode-examples build also needs to be updated
> >>> since the distribution archive name has changed from .tar.gz to .tgz.
> >> Are
> >>> there any other downstream dependencies affected by this change?
> >>>
> >>> Anthony
> >>>
> >>>
> >>>> On Oct 13, 2017, at 1:39 PM, Swapnil Bawaskar 
> >>> wrote:
> >>>>
> >>>> This is the second release candidate for Apache Geode, version 1.3.0.
> I
> >>>> will send out a separate vote thread for geode-examples.
> >>>> Thanks to all the community members for their contributions to this
> >>>> release!
> >>>>
> >>>> *** Please download, test and vote by Monday, October 16, 0800 hrs
> >>>> US Pacific. ***
> >>>>
> >>>> It fixes 376 issues. release notes can be found at:
> >>>>
> >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> >> projectId=12318420&version=12340669
> >>>>
> >>>> Note that we are voting upon the source tags:  rel/v1.3.0.RC2
> >>>> https://github.com/apache/geode/tree/rel/v1.3.0.RC2
> >>>>
> >>>> Commit ID:
> >>>> 9e076738fc2ae40f95bd179b5c1624e664a28d61
> >>>>
> >>>> Source and binary files:
> >>>> https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC2
> >>>>
> >>>> Maven staging repo:
> >>>>
> https://repository.apache.org/content/repositories/orgapachegeode-1033
> >>>>
> >>>> Geode's KEYS file containing PGP keys we use to sign the release:
> >>>> https://github.com/apache/geode/blob/develop/KEYS
> >>>>
> >>>> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> >>>> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> >>>
> >>>
> >>
>
>

[VOTE] Apache Geode release - 1.3.0 RC3

[Swapnil Bawaskar]

This is the third release candidate for Apache Geode, version 1.3.0.
Thanks to all the community members for their contributions to this
release!

*** Please download, test and vote by Monday, October 23, 0800 hrs
US Pacific. ***

It fixes 376 issues. release notes can be found at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669

Note that we are voting upon the source tags:  rel/v1.3.0.RC3
https://github.com/apache/geode/tree/rel/v1.3.0.RC3
https://github.com/apache/geode-examples/tree/rel/v1.3.0.RC3

Commit ID:
9e076738fc2ae40f95bd179b5c1624e664a28d61 (geode)
4ff8f8eafd0927888e711ee45d283ab07d345000   (geode-examples)

Source and binary files:
 https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC3

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachegeode-1034


Geode's KEYS file containing PGP keys we use to sign the release:
 https://github.com/apache/geode/blob/develop/KEYS

Release Signed with Key: pub 4096R/18F902DB 2016-04-07
Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

Re: [VOTE] Apache Geode release - 1.3.0 RC3

[Swapnil Bawaskar]

The commit sha in the vote request should be: 59f2a73d108101744ed7b2d88e8d1c
948d19781c
Sorry I forgot to update it for the latest release candidate.

Also, I ended up dropping and then re-adding all distributions
https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC3
Can you please give this another try?

Thanks!

On Fri, Oct 20, 2017 at 9:30 PM Anthony Baker  wrote:

> I’m confused about which source tag we are voting on.
>
> ~/code/incubator-geode ((rel/v1.3.0.RC3))$ git tag -v rel/v1.3.0.RC3
> object 59f2a73d108101744ed7b2d88e8d1c948d19781c
> type commit
> tag rel/v1.3.0.RC3
> tagger Swapnil Bawaskar  1508355792 +0530
>
> Release candidate 3 for 1.3.0
> gpg: Signature made Wed Oct 18 12:43:21 2017 PDT
> gpg:using RSA key 8F8F2BCC18F902DB
> gpg: Good signature from "Swapnil Bawaskar " [full]
>
> This is different than the commit sha in the vote request.  Also, the
> source and binary distributions appear to have been generated from
> different sha’s?
>
> ~/working$ ./apache-geode-1.3.0/bin/gfsh version --full | grep Revision
> Source-Revision: 9e076738fc2ae40f95bd179b5c1624e664a28d61
>
> ~/working$
> ./apache-geode-1.3.0-src/geode-assembly/build/install/apache-geode/bin/gfsh
> version --full | grep Revision
> Source-Revision: 59f2a73d108101744ed7b2d88e8d1c948d19781c
>
> Voting -1 based on this discrepancy.
>
>
> Other notes:
> - builds from source
> - binary distribution works
> - geode-examples builds from source
> - no binaries in source distribution
> - we should update the geode-examples distribution filename to ‘tgz’ to
> match the geode distribution in the next release
>
> Anthony
>
> > On Oct 18, 2017, at 1:51 PM, Swapnil Bawaskar 
> wrote:
> >
> > This is the third release candidate for Apache Geode, version 1.3.0.
> > Thanks to all the community members for their contributions to this
> > release!
> >
> > *** Please download, test and vote by Monday, October 23, 0800 hrs
> > US Pacific. ***
> >
> > It fixes 376 issues. release notes can be found at:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669
> >
> > Note that we are voting upon the source tags:  rel/v1.3.0.RC3
> > https://github.com/apache/geode/tree/rel/v1.3.0.RC3
> > https://github.com/apache/geode-examples/tree/rel/v1.3.0.RC3
> >
> > Commit ID:
> > 9e076738fc2ae40f95bd179b5c1624e664a28d61 (geode)
> > 4ff8f8eafd0927888e711ee45d283ab07d345000   (geode-examples)
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC3
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachegeode-1034
> >
> >
> > Geode's KEYS file containing PGP keys we use to sign the release:
> > https://github.com/apache/geode/blob/develop/KEYS
> >
> > Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> > Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
>
>

Re: [VOTE] Apache Geode release - 1.3.0 RC3

[Swapnil Bawaskar]

I noticed that my svn commit had failed. The geode distribution should be
available now.

On Tue, Oct 24, 2017 at 7:32 PM Anthony Baker  wrote:

> The dist location only has the examples.  Where are the geode
> distributions?
>
> Anthony
>
> > On Oct 20, 2017, at 2:11 PM, Swapnil Bawaskar 
> wrote:
> >
> > The commit sha in the vote request should be:
> 59f2a73d108101744ed7b2d88e8d1c
> > 948d19781c
> > Sorry I forgot to update it for the latest release candidate.
> >
> > Also, I ended up dropping and then re-adding all distributions
> > https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC3
> > Can you please give this another try?
> >
> > Thanks!
> >
> > On Fri, Oct 20, 2017 at 9:30 PM Anthony Baker  wrote:
> >
> >> I’m confused about which source tag we are voting on.
> >>
> >> ~/code/incubator-geode ((rel/v1.3.0.RC3))$ git tag -v rel/v1.3.0.RC3
> >> object 59f2a73d108101744ed7b2d88e8d1c948d19781c
> >> type commit
> >> tag rel/v1.3.0.RC3
> >> tagger Swapnil Bawaskar  1508355792 +0530
> >>
> >> Release candidate 3 for 1.3.0
> >> gpg: Signature made Wed Oct 18 12:43:21 2017 PDT
> >> gpg:using RSA key 8F8F2BCC18F902DB
> >> gpg: Good signature from "Swapnil Bawaskar "
> [full]
> >>
> >> This is different than the commit sha in the vote request.  Also, the
> >> source and binary distributions appear to have been generated from
> >> different sha’s?
> >>
> >> ~/working$ ./apache-geode-1.3.0/bin/gfsh version --full | grep Revision
> >> Source-Revision: 9e076738fc2ae40f95bd179b5c1624e664a28d61
> >>
> >> ~/working$
> >>
> ./apache-geode-1.3.0-src/geode-assembly/build/install/apache-geode/bin/gfsh
> >> version --full | grep Revision
> >> Source-Revision: 59f2a73d108101744ed7b2d88e8d1c948d19781c
> >>
> >> Voting -1 based on this discrepancy.
> >>
> >>
> >> Other notes:
> >> - builds from source
> >> - binary distribution works
> >> - geode-examples builds from source
> >> - no binaries in source distribution
> >> - we should update the geode-examples distribution filename to ‘tgz’ to
> >> match the geode distribution in the next release
> >>
> >> Anthony
> >>
> >>> On Oct 18, 2017, at 1:51 PM, Swapnil Bawaskar 
> >> wrote:
> >>>
> >>> This is the third release candidate for Apache Geode, version 1.3.0.
> >>> Thanks to all the community members for their contributions to this
> >>> release!
> >>>
> >>> *** Please download, test and vote by Monday, October 23, 0800 hrs
> >>> US Pacific. ***
> >>>
> >>> It fixes 376 issues. release notes can be found at:
> >>>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669
> >>>
> >>> Note that we are voting upon the source tags:  rel/v1.3.0.RC3
> >>> https://github.com/apache/geode/tree/rel/v1.3.0.RC3
> >>> https://github.com/apache/geode-examples/tree/rel/v1.3.0.RC3
> >>>
> >>> Commit ID:
> >>> 9e076738fc2ae40f95bd179b5c1624e664a28d61 (geode)
> >>> 4ff8f8eafd0927888e711ee45d283ab07d345000   (geode-examples)
> >>>
> >>> Source and binary files:
> >>> https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC3
> >>>
> >>> Maven staging repo:
> >>> https://repository.apache.org/content/repositories/orgapachegeode-1034
> >>>
> >>>
> >>> Geode's KEYS file containing PGP keys we use to sign the release:
> >>> https://github.com/apache/geode/blob/develop/KEYS
> >>>
> >>> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> >>> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> >>
> >>
>
>

Re: [DISCUSS] Storage-class memory ecosystem program

[Swapnil Bawaskar]

Thanks Roman!

I have volunteered to be on that group, if anyone else is interested,
please chime in on that email thread.

On Tue, Oct 24, 2017 at 10:57 PM Roman Shaposhnik 
wrote:

> FYI
>
>
> -- Forwarded message --
> From: Gang(Gary) Wang 
> Date: Tue, Oct 24, 2017 at 10:10 AM
> Subject: Re: [DISCUSS] Storage-class memory ecosystem program
> To: gene...@incubator.apache.org
> Cc: d...@mnemonic.incubator.apache.org
>
>
> Add *Hive/LLAP and RocketMQ*
>
>- *Ignite* represented by Denis Magda
>- *Arrow *represented by Wes McKinney
>- *Hbase *represented by Anoop John
>- *Crail* represented by *Patrick Stuedi*
>- *Hive/LLAP *represented by *Gopal Vijayaraghavan*
>- *RocketMQ  *represented by *Von Gosling*
>- *ORC *represented by* Owen O'Malley*
>- *Mnemonic *represented by* Gary*
>
> With above projects, we could cover Storage-class memory oriented
> *Distributed
> Database, KV Store, Columnar Structured Dataset, Distributed Data Store,
> Columnar Storage, **Live Long And Process, Distributed Messaging and
> Streaming, **Durable Object Model, Durable Computing Model* for new
> generation high-performance applications, e.g. data querying, processing,
> and analytics.
>
> On Mon, Oct 23, 2017 at 9:32 PM, Von Gosling 
> wrote:
>
> > I'm happy to represent Apache RocketMQ on this  storage collaboration
> > effort. This is a very key in low latency messaging.
> >
> > Best Regards,
> > Von Gosling
> >
> >
> >
> > > 在 2017年10月20日，02:55，Gang(Gary) Wang  写道：
> > >
> > > Hi all,
> > >
> > > We can expect more and more projects will take the huge potential
> > > advantages of storage-class memory for data processing and analytics
> > > because silicon companies are able to produce high capacity
> non-volatile
> > > memory on a large scale, this hardware technology will fundamentally
> > change
> > > the way to construct high performance applications similar to what
> > happened
> > > when replacing tape with disk technology since the 1980s. so if
> > possible, I
> > > advocate establishing an Apache working group to enhance the
> > collaboration
> > > and synergies mentioned by Patrick Stuedi for storage-class memory
> > > technology-oriented projects.
> > >
> > > Best.
> > > Gary.
> >
> >
>

Re: [VOTE] Apache Geode release - 1.3.0 RC3

[Swapnil Bawaskar]

This vote is cancelled. I will create one more release candidate.

On Thu, Oct 26, 2017 at 6:39 AM Dan Smith  wrote:

> Looks like this RC is still a no go due to the missing signatures Anthony
> pointed out. Swap, when you fix that can you create a new release candidate
> or at least send out a new VOTE email? It's a bit confusing what the state
> of this VOTE is and what the deadline is.
>
> -Dan
>
> On Wed, Oct 25, 2017 at 1:57 PM, Anthony Baker  wrote:
>
> > The following files are missing signatures:
> >
> > apache-geode-1.3.0-src.tgz
> > apache-geode-1.3.0-src.zip
> > apache-geode-1.3.0.tgz
> > apache-geode-1.3.0.zip
> >
> > -1 until this is fixed.
> >
> > Anthony
> >
> > > On Oct 24, 2017, at 10:16 AM, Swapnil Bawaskar 
> > wrote:
> > >
> > > I noticed that my svn commit had failed. The geode distribution should
> be
> > > available now.
> > >
> > > On Tue, Oct 24, 2017 at 7:32 PM Anthony Baker 
> wrote:
> > >
> > >> The dist location only has the examples.  Where are the geode
> > >> distributions?
> > >>
> > >> Anthony
> > >>
> > >>> On Oct 20, 2017, at 2:11 PM, Swapnil Bawaskar 
> > >> wrote:
> > >>>
> > >>> The commit sha in the vote request should be:
> > >> 59f2a73d108101744ed7b2d88e8d1c
> > >>> 948d19781c
> > >>> Sorry I forgot to update it for the latest release candidate.
> > >>>
> > >>> Also, I ended up dropping and then re-adding all distributions
> > >>> https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC3
> > >>> Can you please give this another try?
> > >>>
> > >>> Thanks!
> > >>>
> > >>> On Fri, Oct 20, 2017 at 9:30 PM Anthony Baker 
> > wrote:
> > >>>
> > >>>> I’m confused about which source tag we are voting on.
> > >>>>
> > >>>> ~/code/incubator-geode ((rel/v1.3.0.RC3))$ git tag -v rel/v1.3.0.RC3
> > >>>> object 59f2a73d108101744ed7b2d88e8d1c948d19781c
> > >>>> type commit
> > >>>> tag rel/v1.3.0.RC3
> > >>>> tagger Swapnil Bawaskar  1508355792 +0530
> > >>>>
> > >>>> Release candidate 3 for 1.3.0
> > >>>> gpg: Signature made Wed Oct 18 12:43:21 2017 PDT
> > >>>> gpg:using RSA key 8F8F2BCC18F902DB
> > >>>> gpg: Good signature from "Swapnil Bawaskar "
> > >> [full]
> > >>>>
> > >>>> This is different than the commit sha in the vote request.  Also,
> the
> > >>>> source and binary distributions appear to have been generated from
> > >>>> different sha’s?
> > >>>>
> > >>>> ~/working$ ./apache-geode-1.3.0/bin/gfsh version --full | grep
> > Revision
> > >>>> Source-Revision: 9e076738fc2ae40f95bd179b5c1624e664a28d61
> > >>>>
> > >>>> ~/working$
> > >>>>
> > >> ./apache-geode-1.3.0-src/geode-assembly/build/install/
> > apache-geode/bin/gfsh
> > >>>> version --full | grep Revision
> > >>>> Source-Revision: 59f2a73d108101744ed7b2d88e8d1c948d19781c
> > >>>>
> > >>>> Voting -1 based on this discrepancy.
> > >>>>
> > >>>>
> > >>>> Other notes:
> > >>>> - builds from source
> > >>>> - binary distribution works
> > >>>> - geode-examples builds from source
> > >>>> - no binaries in source distribution
> > >>>> - we should update the geode-examples distribution filename to ‘tgz’
> > to
> > >>>> match the geode distribution in the next release
> > >>>>
> > >>>> Anthony
> > >>>>
> > >>>>> On Oct 18, 2017, at 1:51 PM, Swapnil Bawaskar <
> sbawas...@pivotal.io>
> > >>>> wrote:
> > >>>>>
> > >>>>> This is the third release candidate for Apache Geode, version
> 1.3.0.
> > >>>>> Thanks to all the community members for their contributions to this
> > >>>>> release!
> > >>>>>
> > >>>>> *** Please download, test and vote by Monday, October 23, 0800 hrs
> > >>>>> US Pacific. ***
> > >>>>>
> > >>>>> It fixes 376 issues. release notes can be found at:
> > >>>>>
> > >>>>
> > >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12318420&version=12340669
> > >>>>>
> > >>>>> Note that we are voting upon the source tags:  rel/v1.3.0.RC3
> > >>>>> https://github.com/apache/geode/tree/rel/v1.3.0.RC3
> > >>>>> https://github.com/apache/geode-examples/tree/rel/v1.3.0.RC3
> > >>>>>
> > >>>>> Commit ID:
> > >>>>> 9e076738fc2ae40f95bd179b5c1624e664a28d61 (geode)
> > >>>>> 4ff8f8eafd0927888e711ee45d283ab07d345000   (geode-examples)
> > >>>>>
> > >>>>> Source and binary files:
> > >>>>> https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC3
> > >>>>>
> > >>>>> Maven staging repo:
> > >>>>> https://repository.apache.org/content/repositories/
> > orgapachegeode-1034
> > >>>>>
> > >>>>>
> > >>>>> Geode's KEYS file containing PGP keys we use to sign the release:
> > >>>>> https://github.com/apache/geode/blob/develop/KEYS
> > >>>>>
> > >>>>> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> > >>>>> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> > >>>>
> > >>>>
> > >>
> > >>
> >
> >
>

[VOTE] Apache Geode release - 1.3.0 RC4

[Swapnil Bawaskar]

This is the fourth release candidate for Apache Geode, version 1.3.0.
Thanks to all the community members for their contributions to this
release!

*** Please download, test and vote by Sunday, October 29, 1200 hrs
US Pacific. ***

It fixes 376 issues. release notes can be found at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12340669

Note that we are voting upon the source tags:  rel/v1.3.0.RC4
https://github.com/apache/geode/tree/rel/v1.3.0.RC4
https://github.com/apache/geode-examples/tree/rel/v1.3.0.RC4

Commit ID:
59f2a73d108101744ed7b2d88e8d1c948d19781c (geode)
4ff8f8eafd0927888e711ee45d283ab07d345000   (geode-examples)

Source and binary files:
 https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC4

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachegeode-1035


Geode's KEYS file containing PGP keys we use to sign the release:
 https://github.com/apache/geode/blob/develop/KEYS

Release Signed with Key: pub 4096R/18F902DB 2016-04-07
Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

[RESULT][VOTE] Apache Geode release - 1.3.0 RC4

[Swapnil Bawaskar]

The vote passes with 4 +1 binding votes.
+1 Anthony Baker
+1 Dan Smith
+1 Bruce Schuchardt
+1Jared Stewart

Vote thread:
http://mail-archives.apache.org/mod_mbox/geode-dev/201710.mbox/%3ccanzq1gd-uj9ko4+8t+qngrorsgbdkw83ohnf9j8wpydgevu...@mail.gmail.com%3E

1.3.0 release in JIRA

[Swapnil Bawaskar]

Can someone please mark 1.3.0 as RELEASED in JIRA? I don't seem to have
permission.

Thanks!
Swapnil.

[ANNOUNCE] Apache Geode 1.3.0

[Swapnil Bawaskar]

The Apache Geode community is pleased to announce the availability
of Apache Geode 1.3.0.

Apache Geode is a data management platform that provides a database-like
consistency model, reliable transaction processing and a shared-nothing
architecture to maintain very low latency performance with high concurrency
processing.

Geode 1.3.0 contains a number of improvements and bug fixes.  This release
includes finer grained security and ability to snapshot multiple regions
simultaneously. Users are encouraged to upgrade to the latest release.

For the full list of changes please review the release notes:

https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.3.0

The release artifacts can be downloaded from the project website:
http://geode.apache.org/releases/

The release documentation is available at:
http://geode.apache.org/docs/guide/13/about_geode.html

We would like to thank all the contributors that made the release possible.

Regards,
Swapnil Bawaskar on behalf of the Apache Geode team

Re: [Discussion] SQL+Streaming/JDBC as one of the unified interfaces

[Swapnil Bawaskar]

Hi Christian and Julian,
This indeed sounds very promising.
Looking at the formidable list of "Powered by Calcite
" projects, I think we
should explore embedding Calcite in Geode.

On Thu, Nov 16, 2017 at 2:52 PM Julian Hyde  wrote:

> I'm Julian Hyde, original developer of Calcite and a current PMC member.
>
> I'd love to see Calcite-Geode integration and so I'm delighted with
> the work Christian has been doing. Whenever someone builds an adapter
> for a particular data store X, the question comes up whether it should
> live in X or become an adapter in Calcite. Generally I prefer the
> former. I think Geode would benefit by having a built-in SQL interface
> and ODBC/JDBC server, and Calcite embeds quite easily to make that
> happen. (You just need a couple of jars from maven, and implement some
> SPIs to provide metadata; no extra data on disk.)
>
> If you don't want that, I think our community would be happy to accept
> Christian's code as a Geode adapter in Calcite. Geode would be able to
> include that adapter later, albeit with a small added complication of
> version differences.
>
> But if there are particular concerns, or reasons why you do not want
> SQL support in Geode, now would be a good time to discuss.
>
> Julian
>
>
> On Thu, Nov 16, 2017 at 12:22 PM, Christian Tzolov 
> wrote:
> > Hi Jason
> > ,
> >
> > Thanks for the comments!
> >
> > Regarding the talk, i'm not completely sure but i believe that as a part
> of
> > the S1P conference the sessions will be recorded.
> >
> >>>
> > Also for question 1. Would you be interested to have the adapter as part
> of
> >
> > Geode's code ecosystem?
> >>
> > Do you mean to create a module in Geode for this adapter?  Would it make
> >
> > sense to add a Geode module to Calcite?  Were you wanting a tighter
> >
> > integration (beyond an adapter) with Calcite within
> >
> > Geode?
> >
> > Currently the adapter is implemented as a pure Geode client using only
> the
> > public Geode API/OQL. There are no dependencies from Geode to the
> adapter!
> >
> > 1. If the adapter became one of Calcite project adapter (
> > https://calcite.apache.org/docs/adapter.html) it will benefit from being
> > up to date with latest Calcite developments. But IMO this might not the
> > most important driving force for evolving the adapter.
> > 2. If it became an "extension" project/module under Geode's project
> > umbrella it will stay closer to it potential users and will evolve with
> > their needs. Hopefully it may attract more contributors if found useful.
> >
> > Personally I would be interested to explore further the approach and
> figure
> > out what are its strengths and weaknesses.
> >
> > - Cheers,
> > Christian
> >
> >
> > On 13 November 2017 at 19:46, Jason Huynh  wrote:
> >
> >> Hi Christian,
> >>
> >> I don't know much about Calcite and haven't had a chance to try out your
> >> adapter yet but it sounds like a neat idea.  Will your talk be recorded
> and
> >> available after the Summit?
> >>
> >> Also for question 1. Would you be interested to have the adapter as
> part of
> >> Geode's code ecosystem?
> >> Do you mean to create a module in Geode for this adapter?  Would it make
> >> sense to add a Geode module to Calcite?  Were you wanting a tighter
> >> integration (beyond an adapter) with Calcite within Geode?
> >>
> >> -Jason
> >>
> >> On Fri, Nov 10, 2017 at 3:49 AM Christian Tzolov 
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I've been working lately on Apache Calcite SQL/JDBC adapter for Apache
> >> > Geode [1].
> >> >
> >> > Adapter's current implementation act as a plain Geode client (using
> the
> >> > public API/OQL interfaces) trying to push down to Geode OQL as many
> >> > relational expressions as it can. Relational expressions not
> supported by
> >> > OQL are executed by the adapter itself.
> >> >
> >> > While this approach has its advantages and disadvantages, which I will
> >> try
> >> > to address at my Geode Summit talk [2] I would like to ask two
> question:
> >> >
> >> > 1. Would you be interested to have the adapter as part of Geode's code
> >> > ecosystem?
> >> >
> >> > 2. I am aware (an experienced it myself) the SQLFire story. But given
> >> that
> >> > OQL features are expanding (aggregations are are already supported)
> and
> >> > that tools like Calcite offer proper logical/physical (cost based)
> planer
> >> > and SQL extensions such as SQL streaming, would it be useful to
> discuss
> >> > what novel approaches for using SQL/JDBC with Geode are possible?
> >> >
> >> > (Julian Hyde - founder of Calcite - is in cc)
> >> >
> >> > Cheers,
> >> > Christian
> >> >
> >> > [1] https://github.com/tzolov/calcite/tree/geode-1.3
> >> > [2]
> >> >
> >> > https://springoneplatform.io/sessions/enable-sql-jdbc-
> >> access-to-apache-geode-gemfire-using-apache-calcite
> >> >
> >> >
> >> > --
> >> > Christian Tzolov  | Principle
> >> Software
> >> > Engineer | Pivotal 

Re: [Discussion] Geode-Native Removing Stats from Public API

[Swapnil Bawaskar]

A lot of statistics we have are exposed over JMX. I think we should make an
effort to expose all the stats over JMX, making them consumable with
existing tooling rather than investing in jVSD.

On Mon, Nov 20, 2017 at 2:32 PM Addison Huddy  wrote:

> Thanks for the clarification Jake. So yes, I'm in agreement that we
> should simplify the C++ API and remove the stats API from the C++ client.
>
> \ah
>
> On Mon, Nov 20, 2017 at 10:23 AM, Jacob Barrett 
> wrote:
>
> > To clarify, the goal here is to remove access from the public API to
> inject
> > custom stats into our stats stream. We would still collect stats for the
> > internals of the client.
> >
> > The rational is multifaceted:
> >
> > 1) The C++ API is would need a non-trivial amount of time to modernize.
> The
> > current API uses pointers of pointers for maintaining collections. There
> is
> > a strange cross relationship between components in the stats classes
> which
> > create unique pointer ownership questions. Rather than solving those now
> > and further dragging out the modernization of the C++ API we would drop
> it
> > and evaluated adding it back in a modern way in the future. Though I
> > suspect adding it back in the future will never happen for the reasons
> > below.
> >
> > 2) The storage format for our stats in proprietary to Geode and lacks
> wide
> > market adoption.
> >
> > 3) There are no modern tools for analyzing the statistics generated.
> Geode
> > lacks a tool for viewing or analyzing the statistics. Unless work is
> > prioritized on completing the jVSD application then users are forced to
> > write custom applications to extract the contents of the stats files.
> >
> > I support the removal from the Java public API for reasons 2 and 3.
> Unless
> > we put a full effort into creating the ecosystem around the stats format
> to
> > make it usable we should remove it from the public API. I strongly
> > encourage that we remove it internally too but that is for another
> > discussion.
> >
> > -Jake
> >
> >
> > On Mon, Nov 20, 2017 at 9:43 AM Michael Stolz  wrote:
> >
> > > I'm not clear on why we are removing stats gathering capability.
> > > Do we know that customers aren't using this?
> > > Is it badly broken?
> > >
> > > What is actually driving this work?
> > >
> > > --
> > > Mike Stolz
> > > Principal Engineer, GemFire Product Lead
> > > Mobile: +1-631-835-4771 <(631)%20835-4771> <(631)%20835-4771>
> > >
> > > On Mon, Nov 20, 2017 at 11:42 AM, Bruce Schuchardt <
> > bschucha...@pivotal.io
> > > >
> > > wrote:
> > >
> > > > Should this be done for the Java caches as well?
> > > >
> > > >
> > > > On 11/17/17 11:48 AM, David Kimura wrote:
> > > >
> > > >> I agree, a statistics interface seems beyond the scope of Geode
> Native
> > > >> client responsibility.  Hiding or removing seems appropriate to me.
> > > >>
> > > >> Thanks,
> > > >> David
> > > >>
> > > >> On Fri, Nov 17, 2017 at 11:29 AM, Ernest Burghardt
> > > >>  wrote:
> > > >>
> > > >>> +1 for removal
> > > >>>
> > > >>> On Thu, Nov 16, 2017 at 1:46 PM, Jacob Barrett <
> jbarr...@pivotal.io>
> > > >>> wrote:
> > > >>>
> > > >>> I want to open a discussion regarding the removal of
> > StatisticsFactory
> > >  and
> > >  related APIs from the public API. I can't see that we would want
> the
> > >  Geode
> > >  Native client to be a first class statistics/metrics gathering
> API.
> > >  There
> > >  are plenty of other first class players in this space. If this
> > isn't a
> > >  feature of the client then I suggest it be moved internally. It’s
> > > highly
> > >  unlikely it’s being used but in the case that it is we can
> consider
> > >  moving
> > >  it back after some serious refactoring as it relies on an over
> > >  abundance of
> > >  raw pointers. Rather than spend time refactoring it now let’s just
> > > hide
> > >  it
> > >  away.
> > > 
> > >  -Jake
> > > 
> > > 
> > > 
> > > 
> > > 
> > > >
> > >
> >
>

Testing your application

[Swapnil Bawaskar]

In 1.3.0, a new JUnit Rule has been introduced which makes writing
integration/acceptance tests for application easier.
https://cwiki.apache.org/confluence/display/GEODE/Test+your+application

Please give it a spin. All feedback welcome.

Re: [ANNOUNCE] Spring Data Geode 2.0.2.RELEASE (Kay SR2) Available!

[Swapnil Bawaskar]

This is awesome! Thanks John.

On Tue, Nov 28, 2017 at 9:36 AM Udo Kohlmeyer  wrote:

> Another great release from the Spring world!!
>
> --Udo
>
> On Mon, Nov 27, 2017 at 11:39 PM, John Blum  wrote:
>
> > Greetings Apache Geode Community-
> >
> > It is my pleasure to announce the availability of *Spring Data* *Kay
> > SR2* (*Service
> > Release* 2) which includes *Spring Data Geode* *2.0.2.RELEASE*.  The
> > official release announcement is here...
> >
> > https://spring.io/blog/2017/11/27/spring-data-ingalls-sr9-an
> > d-kay-sr2-released
> >
> > Unfortunately, we missed the announce for the first service release of
> > *Kay*,
> > which occurred back on October 27th; my apologies.
> >
> > So, in both *Kay SR1 + SR2*, you will find the follow improvements...
> >
> >
> > * Better, more reliable integration between *Spring's Cache Abstraction*
> > [1]
> > and *Spring* container initialization (particularly in the context of
> > proxies) with Geode CQ [2].  I have even have a complete example here [3]
> > (README in-progress).
> >
> > * In both *Entity-defined* as well as *Caching-defined* *Regions*, I have
> > enabled the ability to configure the ClientRegionShortcut for client-side
> > generated Regions and RegionShortcut for server-side generated Regions.
> > You can learn more about *Entity-defined Regions* [4] and
> *Caching-defined
> > Regions* [5] by following the links.
> >
> > * In addition to *Spring's Caching Annotations* [6] I have also added
> > support for JSR-107 *JCache* API Annotations in SDG's *Caching-defined
> > Regions*.  That means, if you used any of the JSR-107 annotations in your
> > application service components and declared the SDG
> > @EnableCachingDefinedRegions annotation on your *Spring Boot*
> > @Configuration
> > class, then you can define cache client Regions from the "caches"
> > identified in your use of JSR-107 annotations.
> >
> > * I added Annotation support for configuring Apache Geode Region
> > Compression [7] using the @EnableCompression annotation; see docs [8] for
> > more details.  By default, it just enables the Google Snappy Compressor,
> > provided by Apache Geode OOTB.
> >
> >
> > Several other little enhancements were made and help to improve the
> overall
> > development experience when using Apache Geode, especially in a *Spring*
> > context.  See the *changelog* [9] for a complete list of changes in *Kay
> > SR1 + SR2*.
> >
> >
> > Finally, I leave you with a...
> >
> >
> > *SPRING-GEODE TIP:*
> >
> > Let's say you want to create a simple *Spring Boot*, ClientCache
> > application to store and manage customer data.  It is as simple as
> this...
> >
> >
> > // Define Customer...
> >
> > @Region("Customers")
> > public class Customer {
> >
> >private Long id;
> >
> >@Indexed(from = "/Customers")
> >private String name;
> >
> >...
> > }
> >
> >
> > // Define a Repository to persist and query Customers
> >
> > public interface CustomerRepository extends CrudRepository Long>
> > {
> >...
> > }
> >
> >
> > // Finally, create your *Spring Boot* application & run everything...
> >
> > @SpringBootApplication
> > @ClientCacheApplication
> > @EnableEntityDefinedRegions(basePackageClasses = Customer.class,
> > clientRegionShortcut = ClientRegionShortcut.LOCAL)
> > @EnableGemfireRepositories(basePackageClasses = CustomerRepository.class)
> > @EnableIndexing
> > @EnablePdx
> > //@EnableClusterConfiguration(useHttp = true)
> > public class SpringBootApacheGeodeClientApplication {
> >
> >public static void main(String[] args) {
> >   SpringApplication.run(SpringBootApacheGeodeClientApplication.class,
> > args);
> >}
> >
> >...
> > }
> >
> >
> > I challenge anyone to find any boilerplate code in this example
> application
> > (other than the basic, yet essential application/feature bootstrapping).
> >
> > No longer do you need boilerplate code to configure and create a
> > *ClientCache*, *Pool*, *Regions*, *Indexes*, PDX, etc or to perform basic
> > CRUD and simple (OQL) Queries.
> >
> > As am application developer, you can remain completely *focused* and
> > *committed* to your application business requirements and logic, and let
> > *SDG* worry about the plumbing.
> >
> > Because, I am "initially" using a ClientRegionShortcut.LOCAL, this "
> > *ClientCache*" application runs properly, all by itself, without the need
> > to start a server.  This is ideal when you are rapidly prototyping your
> > application and want to iterate (DEV-TEST) quickly!
> >
> > However, it is a simple matter to also convert this application to use
> > Apache Geode's client/server topology as well... simply remove the
> > configuration for the clientRegionShortcut attribute in the
> > @EnableEntityDefinedRegions annotation.
> >
> > By default, a client Region is set to PROXY and all Region ops will be
> sent
> > to a server using the *ClientCache* "DEFAULT" *Pool*, which is configured
> > to connect to a server running on localhost, listening on port 40404, by

Re: Next release: 1.4.0

[Swapnil Bawaskar]

I would like to volunteer as a release manager.
Currently there are 14 issues that are marked for 1.4.0. If you are working
on any of these, can you please update the JIRA?
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-3688&versions=visible&selectedVersion=12341842

Thanks!

On Tue, Nov 28, 2017 at 9:42 AM Anthony Baker  wrote:

> Bump.  Any volunteers?  If not, I’ll do this.
>
> Anthony
>
>
> > On Nov 22, 2017, at 1:48 PM, Anthony Baker  wrote:
> >
> > We released Geode 1.3.0 at the end of October.  Our next release will be
> 1.4.0.  Questions:
> >
> > 1) Who wants to volunteer as a release manager?
> > 2) What do we want to include in the release?
> > 3) When do we want to do this?
> >
> > IMO, let's should shoot for an early Dec release.
> >
> > Anthony
> >
>
>

Re: Next release: 1.4.0

[Swapnil Bawaskar]

Well, making sure that the JIRA's status is up-to-date and removing the
1.4.0 version tag if the fix can wait for a later release.

On Tue, Nov 28, 2017 at 12:22 PM Michael William Dodge 
wrote:

> What sort of update? I know that GEODE-4010 has a PR that's awaiting
> review and merge.
>
> Sarge
>
> > On 28 Nov, 2017, at 10:03, Swapnil Bawaskar 
> wrote:
> >
> > I would like to volunteer as a release manager.
> > Currently there are 14 issues that are marked for 1.4.0. If you are
> working
> > on any of these, can you please update the JIRA?
> >
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-3688&versions=visible&selectedVersion=12341842
> >
> > Thanks!
> >
> > On Tue, Nov 28, 2017 at 9:42 AM Anthony Baker  wrote:
> >
> >> Bump.  Any volunteers?  If not, I’ll do this.
> >>
> >> Anthony
> >>
> >>
> >>> On Nov 22, 2017, at 1:48 PM, Anthony Baker  wrote:
> >>>
> >>> We released Geode 1.3.0 at the end of October.  Our next release will
> be
> >> 1.4.0.  Questions:
> >>>
> >>> 1) Who wants to volunteer as a release manager?
> >>> 2) What do we want to include in the release?
> >>> 3) When do we want to do this?
> >>>
> >>> IMO, let's should shoot for an early Dec release.
> >>>
> >>> Anthony
> >>>
> >>
> >>
>
>

Re: Geode listed on the top 10 most active mailing lists @ Apache in 2017

[Swapnil Bawaskar]

Wohoo!

Also, in 2017 our downloads went up from ~5k in Jan to ~30k in Oct.

graph below:
https://imgur.com/QOk7nu4



On Mon, Jan 1, 2018 at 10:26 PM William Markito Oliveira <
william.mark...@gmail.com> wrote:

> Congrats everyone, Happy new year!
>
> https://blogs.apache.org/foundation/entry/apache-in-2017-by-the
>
>

Re: Using partition resolver via gfsh command vs XML config

[Swapnil Bawaskar]

The release branch for 1.3 was cut before the date mentioned by Jinmei, so
the fix has not been released yet. Can you please try again against a build
on develop? We hope to have a RC of 1.4 out soon.

On Wed, Jan 3, 2018 at 1:23 PM Kirk Lund  wrote:

> According to the context of where that message is used, the specified
> className is blank.
>
>   private static Class forName(String className, String
> neededFor) {
> if (StringUtils.isBlank(className)) {
>   throw new IllegalArgumentException(CliStrings
>
> .format(CliStrings.CREATE_REGION__MSG__INVALID_PARTITION_RESOLVER,
> className, neededFor));
> }
>
>   public static final String CREATE_REGION__MSG__INVALID_PARTITION_RESOLVER
> =
>   "{0} is an invalid Partition Resolver.";
>
> That StringUtils.isBlank call is using Apache Commons:
>
> public static boolean isBlank(String str) {
> int strLen;
> if (str == null || (strLen = str.length()) == 0) {
> return true;
> }
> for (int i = 0; i < strLen; i++) {
> if ((Character.isWhitespace(str.charAt(i)) == false)) {
> return false;
> }
> }
> return true;
> }
>
> Since your output appears to have a className:
> "io.pivotal.bookshop.partitions.CustomerPartitionResolverPartitionResolver
> is an invalid Partition Resolver", I'm not sure what would cause this. If
> this is repeatable, I recommend filing a Jira ticket and attaching your
> class
> io.pivotal.bookshop.partitions.CustomerPartitionResolverPartitionResolver
> so we can try it out.
>
> On Tue, Jan 2, 2018 at 8:57 AM, Mark Secrist  wrote:
>
> > Hi all,
> >  I'm curious why I'd be running into issues using the gfsh command to
> > create a region using a partition resolver, yet can do so using XML
> > configuration. I've tried a couple of different combinations.
> >
> >  I have a partition resolver class that is packaged in a JAR file. I
> tried
> > the following:
> >
> >1. Deployed jar file using 'deploy --j' [success], created region
> using
> >command 'create region --name=BookOrder --type=PARTITION
> >--partition-resolver=' [Failed]
> >- io.pivotal.bookshop.partitions.CustomerPartitionResolverParti
> > tionResolver
> >is an invalid Partition Resolver
> >2. Configured region from gfsh using StringPrefixPartitionResolver for
> >partition resolver [Success]
> >3. Un-deployed jar file. Used traditional classpath and XML
> >configuration [Success]
> >4. Deployed jar file using 'deploy' [Success] Configure region with
> >partition resolver in XML file [Success]
> >
> > If it wasn't for the fact that attempt #2 was successful, I'd believe
> that
> > maybe it wasn't possible to use the gfsh command to configure a region
> with
> > partition resolver. I'm frankly at a loss to understand why this wouldn't
> > work.
> >
> > This is using GemFire 9.2.1 bits by the way...
> >
> > Thoughts?
> > --
> >
> > *Mark Secrist | Director, **Global Education Delivery*
> >
> > msecr...@pivotal.io
> >
> > 970.214.4567 <(970)%20214-4567> Mobile
> >
> >   *pivotal.io *
> >
> > Follow Us: Twitter  | LinkedIn
> >  | Facebook
> >  | YouTube
> >  | Google+
> > 
> >
>

Re: Next release: 1.4.0

[Swapnil Bawaskar]

Since all the issues tagged for 1.4.0 release
<https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-3688&versions=visible&selectedVersion=12341842>
have been addressed, I went ahead and created a release branch for 1.4.0.

Can someone please update the concourse pipelines to pick up this release
branch?

Thanks!


On Tue, Nov 28, 2017 at 1:58 PM Swapnil Bawaskar 
wrote:

> Well, making sure that the JIRA's status is up-to-date and removing the
> 1.4.0 version tag if the fix can wait for a later release.
>
> On Tue, Nov 28, 2017 at 12:22 PM Michael William Dodge 
> wrote:
>
>> What sort of update? I know that GEODE-4010 has a PR that's awaiting
>> review and merge.
>>
>> Sarge
>>
>> > On 28 Nov, 2017, at 10:03, Swapnil Bawaskar 
>> wrote:
>> >
>> > I would like to volunteer as a release manager.
>> > Currently there are 14 issues that are marked for 1.4.0. If you are
>> working
>> > on any of these, can you please update the JIRA?
>> >
>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92&projectKey=GEODE&view=planning&selectedIssue=GEODE-3688&versions=visible&selectedVersion=12341842
>> >
>> > Thanks!
>> >
>> > On Tue, Nov 28, 2017 at 9:42 AM Anthony Baker 
>> wrote:
>> >
>> >> Bump.  Any volunteers?  If not, I’ll do this.
>> >>
>> >> Anthony
>> >>
>> >>
>> >>> On Nov 22, 2017, at 1:48 PM, Anthony Baker  wrote:
>> >>>
>> >>> We released Geode 1.3.0 at the end of October.  Our next release will
>> be
>> >> 1.4.0.  Questions:
>> >>>
>> >>> 1) Who wants to volunteer as a release manager?
>> >>> 2) What do we want to include in the release?
>> >>> 3) When do we want to do this?
>> >>>
>> >>> IMO, let's should shoot for an early Dec release.
>> >>>
>> >>> Anthony
>> >>>
>> >>
>> >>
>>
>>

Concourse ci access

[Swapnil Bawaskar]

Hi,
Can I please get access to concourse so as to kick off builds?

Thanks!

[VOTE] Apache Geode release - 1.4.0 RC1

[Swapnil Bawaskar]

This is the first release candidate for Apache Geode, version 1.4.0.
Thanks to all the community members for their contributions to this
release!

*** Please download, test and vote by Friday, January 26, 1200 hrs
US Pacific. ***

It fixes 277 issues. release notes can be found at:
https://issues.apache.org/jira/projects/GEODE/versions/12341842

Note that we are voting upon the source tags: rel/v1.4.0.RC1
https://github.com/apache/geode/tree/rel/v1.4.0.RC1
https://github.com/apache/geode-examples/tree/rel/v1.4.0.RC1

Commit ID:
7dbb280fbff0320efd427c5553a6e666328bc50c (geode)
ced35f88173b04ac8e104b9cae10cc38289675fa (geode-examples)

Source and binary files:
https://dist.apache.org/repos/dist/dev/geode/1.4.0.RC1

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachegeode-1036


Geode's KEYS file containing PGP keys we use to sign the release:
https://github.com/apache/geode/blob/develop/KEYS

Release Signed with Key: pub 4096R/18F902DB 2016-04-07
Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

Re: [VOTE] Apache Geode release - 1.4.0 RC1

[Swapnil Bawaskar]

This vote is now cancelled. I will send a vote thread for RC2 when this
issue is resolved.

On Thu, Jan 25, 2018 at 7:40 AM Bruce Schuchardt 
wrote:

> -1 based on Dan's findings
>
>
> On 1/24/18 8:08 AM, Anthony Baker wrote:
> > Here’s a better URL for release notes that doesn’t require logging in:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12341842
> <
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12341842
> >
> >
> > Let’s start crowd-sourcing a condensed version here:
> >
> https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.4.0
> <
> https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.4.0
> >
> >
> >
> > Thanks,
> > Anthony
> >
> >
> >> On Jan 23, 2018, at 2:32 PM, Swapnil Bawaskar 
> wrote:
> >>
> >> This is the first release candidate for Apache Geode, version 1.4.0.
> >> Thanks to all the community members for their contributions to this
> >> release!
> >>
> >> *** Please download, test and vote by Friday, January 26, 1200 hrs
> >> US Pacific. ***
> >>
> >> It fixes 277 issues. release notes can be found at:
> >> https://issues.apache.org/jira/projects/GEODE/versions/12341842
> >>
> >> Note that we are voting upon the source tags: rel/v1.4.0.RC1
> >> https://github.com/apache/geode/tree/rel/v1.4.0.RC1
> >> https://github.com/apache/geode-examples/tree/rel/v1.4.0.RC1
> >>
> >> Commit ID:
> >> 7dbb280fbff0320efd427c5553a6e666328bc50c (geode)
> >> ced35f88173b04ac8e104b9cae10cc38289675fa (geode-examples)
> >>
> >> Source and binary files:
> >> https://dist.apache.org/repos/dist/dev/geode/1.4.0.RC1
> >>
> >> Maven staging repo:
> >> https://repository.apache.org/content/repositories/orgapachegeode-1036
> >>
> >>
> >> Geode's KEYS file containing PGP keys we use to sign the release:
> >> https://github.com/apache/geode/blob/develop/KEYS
> >>
> >> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> >> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> >
>
>

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Swapnil Bawaskar]

+1 to making PDX being more usable by adding an API rather than the current
flag.

I also think we should make the client-side usage of PDX simpler by always
using ReflectionBasedAutoSerializer when no serialization mechanism has
been specified see GEODE-2722
.

On Thu, Jan 25, 2018 at 11:07 AM John Blum  wrote:

> I have always thought/wondered, why not just store the data in serialized
> form always.  There are several reasons to do so...
>
> 1. Whenever data is transferred between client & server, between peers,
> over the WAN, overflowed to disk or persisted to disk, it must be
> serialized.
> 2. Naturally it follows that if the data is always stored in serialized
> form, it cuts down on de/serialization overhead.
> 3. Additionally, there is no need for or reduces the flags and other
> configuration settings to configure serialization making it simpler to
> understand, simpler to use.
> 4. When using PDX, Apache Geode is immediately interoperable between
> multiple language clients, primarily Java and .NET/C++, but even other
> language clients, e.g. JavaScript, Ruby, etc, where JSON is serialized to
> PDX.
> 5. PDX is queryable without deserialization. This is HUGE and maybe the
> most important reason!
>
>
>
> The last 2 points suggest that the default serialization format should be
> PDX, and truthfully, I am not really opposed to that.  Although, there are
> some problems with this.
>
> A. PDX does not handle cyclic dependencies unlike Java Serialization.
> However, Java Serialization has massive overhead and is not interoperable
> with native language and other language clients (e.g. JavaScript).
>
> B. PDX does not handle Deltas unlike DataSerialization.  However, even when
> using Deltas with DataSerialization, you must deserialize the data to apply
> the delta.  Quite frankly and ironically, PDX seems better suited to
> handling Deltas than DataSerialization, and without deserializing.
>
> So, I would double down on PDX and forget DataSerialization and Java
> Serialization.  And by "forget", I mean that Apache Geode never "stores"
> DataSerialized or Java Serialized bytes; only PDX!
>
> Therefore, solve the cyclic dependency problem and introduce proper Delta
> handling without deserialization.  Then, optimize it!  Make PDX the best
> serialization option for Java, and specifically for Apache Geode.  With 1
> serialization format to worry about there is less to maintain, less data to
> convert if the user needs to switch.  Flexibility is not always a good
> thing.  It is easier to build up than to build down if you know what I
> mean.
>
> I have made PDX a first class citizen in *Spring Data for Apache Geode*, in
> multiple key functional areas of the framework (e.g. Repositories) and dead
> simple to use/enable (i.e. @EnablePdx).
>
>
>
> Regarding .NET/C++.  Truthfully, I don't really buy the argument that
> .NET/C++ users shouldn't have to write Java types.  If the data is always
> kept serialized, then technically they shouldn't have to, but they are
> already writing their Functions in Java.  Besides, it is not like every
> type needs a Java type, only types that need to be deserialized, if at all.
>
> If the application consists of both Java and .NET/C++ clients, and the Java
> devs want to work with high-level Java types, then they don't really have a
> choice.  However, we can keep the de/serialization overhead at the point of
> access (e.g. in the Function, executed on a particular node, at the time of
> access), to a minimum.
>
> A simple API like...
>
> JavaType object = pdxInstance.getObject(Class type);
>
> ... would do the trick.
>
> The type argument does not need to be the original type that the PDX type
> meta-data was created from, either.  It could be a "projection".  The only
> concern Apache Geode has is mapping PDX fields to an instance of
> "JavaType", where PDX fields are mapped to writable "JavaType" properties
> (perhaps using Reflection here).
>
> If the JavaType does not contain a property matching a PDX field, no big
> deal.  This is the basis for our versioned type handling anyhow
> (adding/removing a field/property).  However, the inverse is a bit more an
> interesting problem, the JavaType has a field/property that is not
> currently stored in PDX.  Perhaps throw an error, or provide a default
> value, or whatever.  That could be configurable.
>
> Maybe, just maybe, a user has the ability to provide their own Converter,
> with it's own custom behavior...
>
> interface Converter {
>
>   T convert(PdxInstance pdxInstance);
>
> }
>
> class JavaTypeConverter extends Converter {
>
>   JavaType convert(PdxInstance pdxInstance) { ... }
>
> }
>
> Then...
>
> Converter javaTypeConverter = new JavaTypeConverter();
> ...
> JavaType object = pdxInstance.getObject(javaTypeConverter);
>
>
> *One final thought...*
>
> Ultimately, I'd like to see Apache Geode introduce a common
> framework/interface for serialization, so that diff

Re: [PROPOSAL]: Adding declarable support in Gfsh command

[Swapnil Bawaskar]

+1 for ability to pass in simple key-value pairs.
Would it be possible to loose the "?" between the class name and the json
though?

Also, since we are essentially trying to pass a java.util.Properties object
to the Declarable, would it be better if we ask the users to pass-in a
string representation of the properties object? something like:
--cache-loader=my.CacheLoader{key2=value2, key1=value1}
A parenthesis around the values may also help in conveying the intention:
--cache-loader=my.CacheLoader({key2=value2, key1=value1})

On Fri, Jan 26, 2018 at 10:24 AM Nick Reich  wrote:

> This would solve the problem of specifying the parameters for a Declarable,
> but if you provided support for any valid json, you could cover other
> situations as well, including those with more complicated and possibly
> nested configuration. If we would ever support such scenarios in the
> future, I assume that we would want a generic solution that would cover all
> configuration. Is this something that we anticipate needing and if so, how
> would the current proposal cover such situations?
>
> On Fri, Jan 26, 2018 at 8:43 AM, Jens Deppe  wrote:
>
> > This also avoids the other option of implementing this by having
> associated
> > 'params' options for each option which can take a Declarable, thus
> reducing
> > the proliferation of options - in particular for 'create region'.
> >
> > i.e. --cache-listener AND --cache-listener-params.
> >
> > Further, this json parameter would not allow for arbitrary json but would
> > be restricted to a simple key/value mapping so that there would be a
> direct
> > translation to Declarable parameters in the cache.xml.
> >
> > --Jens
> >
> > On Fri, Jan 26, 2018 at 8:07 AM, Jinmei Liao  wrote:
> >
> > > Currently, when you want to specify a call-back in gfsh command option,
> > you
> > > can only pass in the class name, e.g.:
> > >
> > > create region --name=regionA --type=PARTITION
> > --cache-loader=my.CacheLoader
> > >
> > > But these callbacks all implements Declarable (CacheLoader,
> CacheWriter,
> > > CacheListener, CustomExpiry etc.), i.e they can initialized with extra
> > > properties that can be set when they are declared in cache.xml, but
> > > currently, our gfsh command doesn't support that.
> > >
> > > We are proposing to add the support to configure Declarables in gfsh
> > > commands by adding json strings at the end of the class name. like
> this:
> > >
> > > create region --name=regionA --type=PARTITION
> > > --cache-loader=my.CacheLoader?{"key":"value,"key2":"value2"}
> > >
> > > (of course, if you don't need to configure your Declarable, you can
> still
> > > only specify a className as before).
> > >
> > > Comments/thoughts?
> > >
> > > --
> > > Cheers
> > >
> > > Jinmei
> > >
> >
>

[VOTE] Apache Geode release - 1.4.0 RC2

[Swapnil Bawaskar]

After fixing the security concerns in the first release candidate, this is
the second release candidate for Apache Geode, version 1.4.0.
Thanks to all the community members for their contributions to this
release!

*** Please download, test and vote by Thursday, Feb 1, 1400 hrs
US Pacific. ***

It fixes 277 issues. release notes can be found at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12341842

Note that we are voting upon the source tags: rel/v1.4.0.RC2
https://github.com/apache/geode/tree/rel/v1.4.0.RC2
https://github.com/apache/geode-examples/tree/rel/v1.4.0.RC2

Commit ID:
2a70679608120042fa7cbee67f4dd21a085d9588 (geode)
ced35f88173b04ac8e104b9cae10cc38289675fa (geode-examples)

Source and binary files:
https://dist.apache.org/repos/dist/dev/geode/1.4.0.RC2

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachegeode-1037


Geode's KEYS file containing PGP keys we use to sign the release:
https://github.com/apache/geode/blob/develop/KEYS

Release Signed with Key: pub 4096R/18F902DB 2016-04-07
Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

[RESULT][VOTE] Apache Geode release - 1.4.0 RC2

[Swapnil Bawaskar]

This vote passes with five +1 votes, no 0 or -1 votes.

Summary:
Dan Smith   +1
Anthony Baker+1
Sai Boorlagadda +1
Jinmei Liao +1
Dick Cavendar+1

vote thread: http://markmail.org/thread/vomlng4jqcy5hfm3


On Wed, Jan 31, 2018 at 8:49 AM Dick Cavender  wrote:

> +1
>
> - Download src and binary dists
> - Confirmed all dists extract
> - Confirmed src builds
> - Confirmed version and gfsh commands
> - Reviewed LICENSE and NOTICE contents
> - Reviewed maven distribution but didn't consume
>
> -Dick
>
>
> On Mon, Jan 29, 2018 at 4:08 PM, Swapnil Bawaskar 
> wrote:
>
> > After fixing the security concerns in the first release candidate, this
> is
> > the second release candidate for Apache Geode, version 1.4.0.
> > Thanks to all the community members for their contributions to this
> > release!
> >
> > *** Please download, test and vote by Thursday, Feb 1, 1400 hrs
> > US Pacific. ***
> >
> > It fixes 277 issues. release notes can be found at:
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12318420&version=12341842
> >
> > Note that we are voting upon the source tags: rel/v1.4.0.RC2
> > https://github.com/apache/geode/tree/rel/v1.4.0.RC2
> > https://github.com/apache/geode-examples/tree/rel/v1.4.0.RC2
> >
> > Commit ID:
> > 2a70679608120042fa7cbee67f4dd21a085d9588 (geode)
> > ced35f88173b04ac8e104b9cae10cc38289675fa (geode-examples)
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/geode/1.4.0.RC2
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachegeode-1037
> >
> >
> > Geode's KEYS file containing PGP keys we use to sign the release:
> > https://github.com/apache/geode/blob/develop/KEYS
> >
> > Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> > Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> >
>

Re: [RESULT][VOTE] Apache Geode release - 1.4.0 RC2

[Swapnil Bawaskar]

Can someone please close the 1.4.0 release in JIRA? I don't have enough
privilege.

On Thu, Feb 1, 2018 at 3:27 PM Xiaojian Zhou  wrote:

> +1
>
> On Thu, Feb 1, 2018 at 2:19 PM, Swapnil Bawaskar 
> wrote:
>
> > This vote passes with five +1 votes, no 0 or -1 votes.
> >
> > Summary:
> > Dan Smith   +1
> > Anthony Baker+1
> > Sai Boorlagadda +1
> > Jinmei Liao +1
> > Dick Cavendar+1
> >
> > vote thread: http://markmail.org/thread/vomlng4jqcy5hfm3
> >
> >
> > On Wed, Jan 31, 2018 at 8:49 AM Dick Cavender 
> > wrote:
> >
> > > +1
> > >
> > > - Download src and binary dists
> > > - Confirmed all dists extract
> > > - Confirmed src builds
> > > - Confirmed version and gfsh commands
> > > - Reviewed LICENSE and NOTICE contents
> > > - Reviewed maven distribution but didn't consume
> > >
> > > -Dick
> > >
> > >
> > > On Mon, Jan 29, 2018 at 4:08 PM, Swapnil Bawaskar <
> sbawas...@pivotal.io>
> > > wrote:
> > >
> > > > After fixing the security concerns in the first release candidate,
> this
> > > is
> > > > the second release candidate for Apache Geode, version 1.4.0.
> > > > Thanks to all the community members for their contributions to this
> > > > release!
> > > >
> > > > *** Please download, test and vote by Thursday, Feb 1, 1400 hrs
> > > > US Pacific. ***
> > > >
> > > > It fixes 277 issues. release notes can be found at:
> > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > > > projectId=12318420&version=12341842
> > > >
> > > > Note that we are voting upon the source tags: rel/v1.4.0.RC2
> > > > https://github.com/apache/geode/tree/rel/v1.4.0.RC2
> > > > https://github.com/apache/geode-examples/tree/rel/v1.4.0.RC2
> > > >
> > > > Commit ID:
> > > > 2a70679608120042fa7cbee67f4dd21a085d9588 (geode)
> > > > ced35f88173b04ac8e104b9cae10cc38289675fa (geode-examples)
> > > >
> > > > Source and binary files:
> > > > https://dist.apache.org/repos/dist/dev/geode/1.4.0.RC2
> > > >
> > > > Maven staging repo:
> > > >
> https://repository.apache.org/content/repositories/orgapachegeode-1037
> > > >
> > > >
> > > > Geode's KEYS file containing PGP keys we use to sign the release:
> > > > https://github.com/apache/geode/blob/develop/KEYS
> > > >
> > > > Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> > > > Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> > > >
> > >
> >
>

[ANNOUNCE] Apache Geode 1.4.0

[Swapnil Bawaskar]

The Apache Geode community is pleased to announce the availability of
Apache Geode 1.4.0.

Apache Geode is a data management platform that provides a database-like
consistency model, reliable transaction processing and a shared-nothing
architecture to maintain very low latency performance with high concurrency
processing.

Geode 1.4.0 contains a number of improvements and bug fixes. This release
includes a simple JDBC connector which enables read-through and write
behind to any RDBMS without any custom code, enables indexing nested fields
for lucene indexes, optimizations to reduce memory footprint of compact
range indexes and optimizations to reduce CPU consumption by eviction
algorithm. Users are encouraged to upgrade to the latest release.

For the full list of changes please review the release notes:
https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.4.0

The release artifacts can be downloaded from the project website:
http://geode.apache.org/releases/

The release documentation is available at:
http://geode.apache.org/docs/guide/14/about_geode.html

We would like to thank all the contributors that made the release possible.

Regards,
Swapnil Bawaskar on behalf of the Apache Geode team

[ANNOUNCE] Apache Geode 1.4.0

[Swapnil Bawaskar]

The Apache Geode community is pleased to announce the availability of
Apache Geode 1.4.0.

Apache Geode is a data management platform that provides a database-like
consistency model, reliable transaction processing and a shared-nothing
architecture to maintain very low latency performance with high concurrency
processing.

Geode 1.4.0 contains a number of improvements and bug fixes. This release
includes a simple JDBC connector which enables read-through and write
behind to any RDBMS without any custom code, enables indexing nested fields
for lucene indexes, optimizations to reduce memory footprint of compact
range indexes and optimizations to reduce CPU consumption by eviction
algorithm. Users are encouraged to upgrade to the latest release.

For the full list of changes please review the release notes:
https://cwiki.apache.org/confluence/display/GEODE/
Release+Notes#ReleaseNotes-1.4.0

The release artifacts can be downloaded from the project website:
http://geode.apache.org/releases/

The release documentation is available at:
http://geode.apache.org/docs/guide/14/about_geode.html

We would like to thank all the contributors that made the release possible.

Regards,
Swapnil Bawaskar on behalf of the Apache Geode team

Re: Next release: 1.5.0

[Swapnil Bawaskar]

I will take up the release management task for 1.5.0

On Mon, Feb 26, 2018 at 5:03 PM Dave Barnes  wrote:

> Status on the 3 doc issues:
> GEODE-4737 / GEODE-3915: JSON args in gfsh - Karen's got it covered
> GEODE-4101:  redirect-output  - Dave, in process, on track
> GEODE-3948: client timeout - Dave, in process. Probably on track - will
> keep you posted
>
> On Mon, Feb 26, 2018 at 11:07 AM, Anthony Baker  wrote:
>
> > Just checking in as we’re approaching the end of February.  We’ve
> finished
> > around 200 issues and enhancements with 3 documentation updates open [1].
> > Is this a good time for another release?
> >
> > Any takers to do the release management tasks for 1.5.0?
> >
> > Anthony
> >
> > [1] https://issues.apache.org/jira/projects/GEODE/versions/12342395 <
> > https://issues.apache.org/jira/projects/GEODE/versions/12342395>
> >
> >
> > > On Feb 7, 2018, at 1:56 PM, Anthony Baker  wrote:
> > >
> > > Hi all,
> > >
> > > Nice work on getting the 1.4.0 release out the door!  Next up is 1.5.0.
> > Any one want to volunteer for release manager?  If you haven’t done this
> > before and would like to try, please review [1].
> > >
> > > I’ve been advocating for more frequent releases.  I’d love see a March
> > release—which means we would need to be ready to cut the release branch
> in
> > early March.
> > >
> > > Thoughts?
> > >
> > > Anthony
> > >
> > > [1]
> https://cwiki.apache.org/confluence/display/GEODE/Release+Steps?src=
> > contextnavpagetreemode <
> https://cwiki.apache.org/confluence/display/GEODE/
> > Release+Steps?src=contextnavpagetreemode>
> > >
> >
> >
>

Re: Next release: 1.5.0

[Swapnil Bawaskar]

Thanks Dave!

All, I have created a release branch (
https://github.com/apache/geode/tree/release/1.5.0) Please review.

On Fri, Mar 2, 2018 at 9:56 AM Dave Barnes  wrote:

> Status on the 3 doc issues:
> GEODE-4737 / GEODE-3915: JSON args in gfsh - Done
> GEODE-4101:  redirect-output - Done
> GEODE-3948: client timeout - Done
>
>
> On Thu, Mar 1, 2018 at 11:22 AM, Swapnil Bawaskar 
> wrote:
>
> > I will take up the release management task for 1.5.0
> >
> > On Mon, Feb 26, 2018 at 5:03 PM Dave Barnes  wrote:
> >
> > > Status on the 3 doc issues:
> > > GEODE-4737 / GEODE-3915: JSON args in gfsh - Karen's got it covered
> > > GEODE-4101:  redirect-output  - Dave, in process, on track
> > > GEODE-3948: client timeout - Dave, in process. Probably on track - will
> > > keep you posted
> > >
> > > On Mon, Feb 26, 2018 at 11:07 AM, Anthony Baker 
> > wrote:
> > >
> > > > Just checking in as we’re approaching the end of February.  We’ve
> > > finished
> > > > around 200 issues and enhancements with 3 documentation updates open
> > [1].
> > > > Is this a good time for another release?
> > > >
> > > > Any takers to do the release management tasks for 1.5.0?
> > > >
> > > > Anthony
> > > >
> > > > [1] https://issues.apache.org/jira/projects/GEODE/versions/12342395
> <
> > > > https://issues.apache.org/jira/projects/GEODE/versions/12342395>
> > > >
> > > >
> > > > > On Feb 7, 2018, at 1:56 PM, Anthony Baker 
> wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Nice work on getting the 1.4.0 release out the door!  Next up is
> > 1.5.0.
> > > > Any one want to volunteer for release manager?  If you haven’t done
> > this
> > > > before and would like to try, please review [1].
> > > > >
> > > > > I’ve been advocating for more frequent releases.  I’d love see a
> > March
> > > > release—which means we would need to be ready to cut the release
> branch
> > > in
> > > > early March.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Anthony
> > > > >
> > > > > [1]
> > > https://cwiki.apache.org/confluence/display/GEODE/Release+Steps?src=
> > > > contextnavpagetreemode <
> > > https://cwiki.apache.org/confluence/display/GEODE/
> > > > Release+Steps?src=contextnavpagetreemode>
> > > > >
> > > >
> > > >
> > >
> >
>

Re: [PROPOSAL]: concurrent bucket moves during rebalance

[Swapnil Bawaskar]

Given that there is already support for parallel rebalancing among regions,
I do not see the value in supporting parallel rebalancing of buckets.

If we end up doing this anyway, I would suggest to not rely on "parameters"
for throttling, as these parameters would have to be configured in advance
without knowing what the actual load looks like when rebalance is in
progress and hence could be difficult to get right. It would be ideal if we
can handle this using back pressure.

On Thu, Mar 8, 2018 at 12:05 PM Nick Reich  wrote:

> Mike,
>
> I think having a good default value for maximum parallel operations will
> play a role in not consuming too many resources. Perhaps defaulting to only
> a single (or other small number based on testing) parallel action(s) per
> member at a time and allowing users that want better performance to
> increase that number would be a good start. That should result in
> performance improvements, but not place increased burden on any specific
> member. Especially when bootstrapping new members, relance speed may be
> more valuable than usual, so making it possible to configure on a per
> rebalance action level would be prefered.
>
> One clarification from my original proposal: regions can already be
> rebalanced in parallel, depending on the value of resource.manager.threads
> (which defaults to 1, so no parallelization or regions in the default
> case).
>
> On Thu, Mar 8, 2018 at 11:46 AM, Michael Stolz  wrote:
>
> > We should be very careful about how much resource we dedicate to
> > rebalancing.
> >
> > One of our competitors rebalances *much* faster than we do, but in doing
> so
> > they consume all available resources.
> >
> > At one bank that caused significant loss of incoming market data that was
> > coming in on a multicast feed, which had a severe adverse effect on the
> > pricing and risk management functions for a period of time. That bank
> > removed the competitor's product and for several years no distributed
> > caching was allowed by the chief architect at that bank. Until he left
> and
> > a new chief architect was named they didn't use any distributed caching
> > products. When they DID go back to using them, it pre-dated Geode, so
> they
> > used GemFire largely because GemFire does not consume all available
> > resources while rebalancing.
> >
> > I do think we need to improve our rebalancing such that it iterates until
> > it achieves balance, but not in a way that will consume all available
> > resources.
> >
> > --
> > Mike Stolz
> >
> >
> > On Thu, Mar 8, 2018 at 2:25 PM, Nick Reich  wrote:
> >
> > > Team,
> > >
> > > The time required to undertake a rebalance of a geode cluster has often
> > > been an area for improvement noted by users. Currently, buckets are
> moved
> > > one at a time and we propose that creating a system that moved buckets
> in
> > > parallel could greatly improve performance for this feature.
> > >
> > > Previously, parallelization was implemented for adding redundant copies
> > of
> > > buckets to restore redundancy. However, moving buckets is a more
> > > complicated matter and requires a different approach than restoration
> of
> > > redundancy. The reason for this is that members could be potentially
> both
> > > be gaining buckets and giving away buckets at the same time. While
> giving
> > > away a bucket, that member still has all of the data for the bucket,
> > until
> > > the receiving member has fully received the bucket and it can safely be
> > > removed from the original owner. This means that unless the member has
> > the
> > > memory overhead to store all of the buckets it will receive and all the
> > > buckets it started with, there is potential that parallel moving of
> > buckets
> > > could cause the member to run out of memory.
> > >
> > > For this reason, we propose a system that does (potentially) several
> > rounds
> > > of concurrent bucket moves:
> > > 1) A set of moves is calculated to improve balance that meet a
> > requirement
> > > that no member both receives and gives away a bucket (no member will
> have
> > > memory overhead of an existing bucket it is ultimately removing and a
> new
> > > bucket).
> > > 2) Conduct all calculated bucket moves in parallel. Parameters to
> > throttle
> > > this process (to prevent taking too many cluster resources, impacting
> > > performance) should be added, such as only allowing each member to
> either
> > > receive or send a maximum number of buckets concurrently.
> > > 3) If cluster is not yet balanced, perform additional iterations of
> > > calculating and conducting bucket moves, until balance is achieved or a
> > > possible maximum iterations is reached.
> > > Note: in both the existing and proposed system, regions are rebalanced
> > one
> > > at a time.
> > >
> > > Please let us know if you have feedback on this approach or additional
> > > ideas that should be considered.
> > >
> >
>

[VOTE] Apache Geode 1.5.0 RC1

[Swapnil Bawaskar]

This is the first release candidate for Apache Geode, version 1.5.0.
Thanks to all the community members for their contributions to this
release!

*** Please download, test and vote by Friday, March 23, 1500 hrs
US Pacific. ***

It fixes 234 issues. release notes can be found at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12342395

Note that we are voting upon the source tags: rel/v1.5.0.RC1
https://github.com/apache/geode/tree/rel/v1.5.0.RC1
https://github.com/apache/geode-examples/tree/rel/v1.5.0.RC1

Commit ID:
4ef51dacd79ff69336fb024f3d4b07271e90 (geode)
4941f05c86d928949fbcdb3fb12295ccecc219eb (geode-examples)

Source and binary files:
https://dist.apache.org/repos/dist/dev/geode/1.5.0.RC1

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachegeode-1038


Geode's KEYS file containing PGP keys we use to sign the release:
https://github.com/apache/geode/blob/develop/KEYS

Release Signed with Key: pub 4096R/18F902DB 2016-04-07
Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

Re: [DISCUSS] New List for Commit and CI Emails

[Swapnil Bawaskar]

I don't think we should send ci to a different email list for it to be
ignored. We ideally should be in a place where emails from ci are not the
norm but an exception which prompts immediate action.

On Tue, Mar 20, 2018 at 3:05 PM Dan Smith  wrote:

> +1 to removing all automated messages from dev@geode
>
> I think we've already shunted commit messages to commits@geode and jira
> issue updates to issues@geode. What I see now is mostly messages related
> to
> ci. Should we create a new ci@geode mailing list, or send those messages
> to
> some other mailing list?
>
> If we want to create a new list, I think the next step is to file a ticket
> with infra after we come to a consensus. Unless our illustrious leader Mark
> has access to do that :)
>
> To move the CI emails to the the new list, we'll have to update concourse,
> jenkins, travis, and spring CI it looks like.
>
> -Dan
>
> On Tue, Mar 20, 2018 at 2:27 PM, Galen O'Sullivan 
> wrote:
>
> > Hi all,
> >
> > I'm wondering if it would be possible to have a separate list for commit
> > announcements, CI emails and similar automated messages, and keep this
> > mailing list for discussion. It would make it easier to filter email,
> and I
> > think it might help improve the discussion. I know I've missed messages
> > sent to this list in the flood of email. What do you think? Who's in
> charge
> > of lists and gets to make those decisions?
> >
> > Thanks,
> > Galen
> >
>

Re: [VOTE] Apache Geode 1.5.0 RC1

[Swapnil Bawaskar]

Removed!

I also filed a JIRA to not generate the md5 checksum file going forward:
https://issues.apache.org/jira/browse/GEODE-4903

On Tue, Mar 20, 2018 at 3:18 PM Dan Smith  wrote:

> It looks like we are still generating .md5 files. That doesn't necessarily
> need to hold up this release (you could just delete them from SVN), but
> we're not supposed to be distributing .md5 files anymore.
>
> http://www.apache.org/dev/release-distribution#sigs-and-sums
>
> -Dan
>
> On Tue, Mar 20, 2018 at 3:09 PM, Swapnil Bawaskar 
> wrote:
>
> > This is the first release candidate for Apache Geode, version 1.5.0.
> > Thanks to all the community members for their contributions to this
> > release!
> >
> > *** Please download, test and vote by Friday, March 23, 1500 hrs
> > US Pacific. ***
> >
> > It fixes 234 issues. release notes can be found at:
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12318420&version=12342395
> >
> > Note that we are voting upon the source tags: rel/v1.5.0.RC1
> > https://github.com/apache/geode/tree/rel/v1.5.0.RC1
> > https://github.com/apache/geode-examples/tree/rel/v1.5.0.RC1
> >
> > Commit ID:
> > 4ef51dacd79ff69336fb024f3d4b07271e90 (geode)
> > 4941f05c86d928949fbcdb3fb12295ccecc219eb (geode-examples)
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/geode/1.5.0.RC1
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachegeode-1038
> >
> >
> > Geode's KEYS file containing PGP keys we use to sign the release:
> > https://github.com/apache/geode/blob/develop/KEYS
> >
> > Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> > Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> >
>

Re: [DISCUSS] New List for Commit and CI Emails

[Swapnil Bawaskar]

I know travis is already configured to send emails only when the build
breaks and then when it is fixed. Is concourse configured the same?

On Wed, Mar 21, 2018 at 9:38 AM Patrick Rhomberg 
wrote:

> I'm with Swapnil on this one.  I think the way we make it less noisy is to
> take the time to fix the failing tests.
>
> I suppose we could split the difference and give the CI emails a, say,
> daily cadence.  No news is good news, or else it gives you all the failures
> in the last 24 hours.  Don't know how easy that would be to cache and
> report under the existing framework, though.
>
> On Wed, Mar 21, 2018 at 12:05 AM, Jacob Barrett 
> wrote:
>
> > It’s sad that the most frequent spammer... e... I mean mailer is the
> > new CI process. If we aren’t going to send it elsewhere how can we make
> it
> > less noisy?
> >
> > -Jake
> >
> >
> > > On Mar 20, 2018, at 8:37 PM, Dan Smith  wrote:
> > >
> > > I was curious about the stats for bot vs. humans on the dev list. Out
> of
> > > 915 messages, looks like we're about 50% robot.
> > >
> > > I'm still be in favor of not sending these messages to dev@geode. Long
> > time
> > > members have probably already created a mail filter by now (I know I
> > have)
> > > so we're only hurting newbies by sending a bunch of messages.
> > >
> > > 1) apac...@gmail.com 241
> > > 2) Spring CI 109
> > > 3) Kirk Lund 63
> > > 4) Apache Jenkins Server 51
> > > 5) Anthony Baker 41
> > > 6) Dan Smith 40
> > > 7) Travis CI 38
> >
>

Re: Recreate Cache -- is it possible?

[Swapnil Bawaskar]

Does this mean auto-reconnect is broken as well?

On Wed, Mar 21, 2018 at 10:28 AM Jinmei Liao  wrote:

> Bruce: this sounds like the root cause of the differences between the dunit
> test and reall app test.
>
> On Wed, Mar 21, 2018 at 10:22 AM, Bruce Schuchardt  >
> wrote:
>
> > It's likely that the JVM is exiting because the AcceptorImpl thread is
> the
> > only non-daemon thread and it is stopped when the cache is closed.  DUnit
> > JVMs have a non-daemon main() thread that keeps them alive.
> >
> >
> >
> > On 3/21/18 9:48 AM, Jinmei Liao wrote:
> >
> >> We would like to allow users to import a new set of cluster
> configuration
> >> with running servers as long as we make sure these servers are vanilla
> >> servers (servers that are just started with nothing in it). Now since
> the
> >> servers are already up, caches are already created, we will need to
> >> re-create the cache with the new xml received from the locator.
> Originally
> >> our implementation on the servers boils down to:
> >>
> >> cache.close("Re-create Cache", true, true);
> >>
> >> GemFireCacheImpl.create(oldDs, cacheConfig);
> >>
> >>
> >> but the cache.close call eventually leads to a VM exit (somehow in the
> >> DUunit VM, it doesn not), so this does not work with real application
> >> environment. Now we are wondering is there a safe to recreate the cache
> >> instance with a new set of properties/cacheXml without triggering the
> >> entire shutdown sequence?
> >>
> >>
> >>
> >
>
>
> --
> Cheers
>
> Jinmei
>

Re: [Proposal] Gfsh Command Feature Flag Annotation

[Swapnil Bawaskar]

I like @Disabled too.

On Mon, Mar 19, 2018 at 12:02 PM Michael William Dodge 
wrote:

> I kind of like @Disabled instead.
>
> Sarge
>
> > On 19 Mar, 2018, at 11:58, Udo Kohlmeyer  wrote:
> >
> > I wonder if this proposal could not be extended to the greater GEODE
> product. As this feature flagging is also relevant to other parts of the
> system and should maybe be consistently applied to all areas.
> >
> > Thoughts?
> >
> >
> > On 3/19/18 11:46, Patrick Rhomberg wrote:
> >> Hello, All
> >>
> >>   I am interested in extending annotation functionality on our gfsh
> >> commands, particularly with respect to feature-flagging commands that
> are
> >> mutually-reliant or not yet feature complete.
> >>   Please review the proposal [1] at your convenience.
> >>
> >> Imagination is Change.
> >> ~Patrick Rhomberg
> >>
> >> [1]
> >>
> https://cwiki.apache.org/confluence/display/GEODE/Proposal+for+Gfsh+Feature+Flag
> >>
> >
>
>

Re: [Proposal] Gfsh Command Feature Flag Annotation

[Swapnil Bawaskar]

This is a great start, however, there may be features that only add options
to gfsh commands rather than adding gfsh commands themselves, we should
accommodate those as and when we encounter them.

Udo, I like the idea of having a more generic solution for feature
flagging, however, if a feature is only introducing public API, I don't see
how we could hide it using an annotation.

On Wed, Mar 21, 2018 at 4:46 PM Swapnil Bawaskar 
wrote:

> I like @Disabled too.
>
> On Mon, Mar 19, 2018 at 12:02 PM Michael William Dodge 
> wrote:
>
>> I kind of like @Disabled instead.
>>
>> Sarge
>>
>> > On 19 Mar, 2018, at 11:58, Udo Kohlmeyer  wrote:
>> >
>> > I wonder if this proposal could not be extended to the greater GEODE
>> product. As this feature flagging is also relevant to other parts of the
>> system and should maybe be consistently applied to all areas.
>> >
>> > Thoughts?
>> >
>> >
>> > On 3/19/18 11:46, Patrick Rhomberg wrote:
>> >> Hello, All
>> >>
>> >>   I am interested in extending annotation functionality on our gfsh
>> >> commands, particularly with respect to feature-flagging commands that
>> are
>> >> mutually-reliant or not yet feature complete.
>> >>   Please review the proposal [1] at your convenience.
>> >>
>> >> Imagination is Change.
>> >> ~Patrick Rhomberg
>> >>
>> >> [1]
>> >>
>> https://cwiki.apache.org/confluence/display/GEODE/Proposal+for+Gfsh+Feature+Flag
>> >>
>> >
>>
>>

Re: [VOTE] Apache Geode 1.5.0 RC1

[Swapnil Bawaskar]

This vote is now cancelled.

Based on the comments, I will cut a new RC when we have fixes for GEODE-4913
and GEODE-4787.


On Fri, Mar 23, 2018 at 3:11 PM Udo Kohlmeyer  wrote:

> -1 Due to the regression in functionality. GEODE-4913
>
> @John, I agree, GEODE-4787 should be addressed with priority.
>
> --Udo
> On 3/23/18 15:04, John Blum wrote:
> > The same could be said of GEODE-4787.
> >
> > On Fri, Mar 23, 2018 at 2:58 PM, Anthony Baker 
> wrote:
> >
> >> Thanks for catching this Diane.  I agree; we shouldn't break something
> that
> >> used to work.
> >>
> >> Anthony
> >>
> >>
> >> On Mar 23, 2018, at 11:27 AM, Diane Hardman 
> wrote:
> >>
> >> -1
> >> I filed GEODE-4913 based on scripts I have that create 2 clusters
> connected
> >> by a WAN gateway on my local machine.
> >> *Problem*: we are opening a cache-server on default port (40404)
> >> irrespective of the fact that one is explicitly specified in cache.xml.
> >> *symptoms*: users who start more than one Geode server on the same “box”
> >> and use cache.xml to specify port would get bind exceptions when they
> start
> >> the second server.
> >> *workaround*:
> >>1. specify `--disable-default-server` on the `start server` command
> >> while still specifying port in cache.xml.
> >>2. specify port on the `start server` command itself and remove any
> >> references to port from cache.xml.
> >>
> >> The problem will break a lot of existing scripts and, though we have a
> >> workaround, it will cause headaches for many when they encounter it.
> >>
> >> On Wed, Mar 21, 2018 at 2:52 PM, Dan Smith  wrote:
> >>
> >> +1
> >>
> >> Run geode-release-check. Verified no MD5 files in the distribution.
> >>
> >> -Dan
> >>
> >> On Wed, Mar 21, 2018 at 11:57 AM, Anthony Baker 
> wrote:
> >>
> >> +1
> >>
> >> - verified signatures and checksums
> >> - checked source release for binaries
> >> - basic gfsh testing
> >> - ran all the examples
> >>
> >> Anthony
> >>
> >> On Mar 20, 2018, at 3:09 PM, Swapnil Bawaskar 
> >>
> >> wrote:
> >>
> >>
> >> This is the first release candidate for Apache Geode, version 1.5.0.
> >> Thanks to all the community members for their contributions to this
> >> release!
> >>
> >> *** Please download, test and vote by Friday, March 23, 1500 hrs
> >> US Pacific. ***
> >>
> >> It fixes 234 issues. release notes can be found at:
> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> >>
> >> projectId=12318420&version=12342395
> >>
> >>
> >> Note that we are voting upon the source tags: rel/v1.5.0.RC1
> >> https://github.com/apache/geode/tree/rel/v1.5.0.RC1
> >> https://github.com/apache/geode-examples/tree/rel/v1.5.0.RC1
> >>
> >> Commit ID:
> >> 4ef51dacd79ff69336fb024f3d4b07271e90 (geode)
> >> 4941f05c86d928949fbcdb3fb12295ccecc219eb (geode-examples)
> >>
> >> Source and binary files:
> >> https://dist.apache.org/repos/dist/dev/geode/1.5.0.RC1
> >>
> >> Maven staging repo:
> >> https://repository.apache.org/content/repositories/orgapachegeode-1038
> >>
> >>
> >> Geode's KEYS file containing PGP keys we use to sign the release:
> >> https://github.com/apache/geode/blob/develop/KEYS
> >>
> >> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> >> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> >>
> >
> >
>
>

Re: Logging in to concourse.apachegeode-ci.info

[Swapnil Bawaskar]

Can I please get access to start jobs in concourse?
my github username is sbawaska

On Thu, Feb 22, 2018 at 10:24 AM Sean Goller  wrote:

> Yeah, the geode pipelines are completely public. You don't need to log into
> concourse to see anything, only to pause, kill or start jobs. Any changes
> made to the pipeline is done through the git repo, and those changes will
> be automatically pushed to concourse when merged.
>
> -S.
>
> On Wed, Feb 21, 2018 at 10:28 AM, Kirk Lund  wrote:
>
> > I just assumed that I was *supposed* to login to that pipeline. Sounds
> like
> > I probably don't need to login(?), but that's really not very obvious
> when
> > looking at the pipeline gui. Nevermind... I don't need to kill or modify
> > anything. I'm just used to logging into any concourse pipeline in order
> to
> > view results.
> >
> > On Wed, Feb 21, 2018 at 9:40 AM, Anthony Baker 
> wrote:
> >
> > > Any committer can ask for login privileges (just like on Jenkins).
> That
> > > only gives you the ability to retrigger a job or kill a job.
> > >
> > > Is that what you want?  If you want to modify job definitions you can
> do
> > > that by committing changes to [1].
> > >
> > > Anthony
> > >
> > > [1] https://github.com/apache/geode/tree/develop/ci <
> > > https://github.com/apache/geode/tree/develop/ci>
> > >
> > >
> > > > On Feb 21, 2018, at 9:24 AM, Dan Smith  wrote:
> > > >
> > > > I don't think you can log in to that pipeline. What are you wanting
> to
> > do
> > > > to it?
> > > >
> > > > -Dan
> > > >
> > > > On Wed, Feb 21, 2018 at 9:15 AM, Kirk Lund  wrote:
> > > >
> > > >> How are we supposed to login to
> > > >> https://concourse.apachegeode-ci.info/teams/main/login?
> > > >>
> > > >> I'm trying to use the "login with GitHub" option (I think this
> worked
> > > for
> > > >> me before) but it keeps failing in a couple different ways... latest
> > > >> failure was "failed to verify token", previous one was "verification
> > > >> failed"
> > > >>
> > > >> Thanks,
> > > >> Kirk
> > > >>
> > >
> > >
> >
>

[VOTE] Apache Geode 1.5.0.RC2

[Swapnil Bawaskar]

Both issues mentioned in the RC1 vote thread

have been resolved.

This is the second release candidate for Apache Geode, version 1.5.0.
Thanks to all the community members for their contributions to this
release!

*** Please download, test and vote by Wednesday, April 4, 1200 hrs
US Pacific. ***

It fixes 235 issues. release notes can be found at:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420&version=12342395

Note that we are voting upon the source tags: rel/v1.5.0.RC1
https://github.com/apache/geode/tree/rel/v1.5.0.RC2
https://github.com/apache/geode-examples/tree/rel/v1.5.0.RC2

Commit ID:
1be57f3d18e5a97dbcc784d6df7724f5f6ced39b (geode)
4941f05c86d928949fbcdb3fb12295ccecc219eb (geode-examples)

Source and binary files:
https://dist.apache.org/repos/dist/dev/geode/1.5.0.RC2

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachegeode-1039


Geode's KEYS file containing PGP keys we use to sign the release:
https://github.com/apache/geode/blob/develop/KEYS

Release Signed with Key: pub 4096R/18F902DB 2016-04-07
Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

[RESULT][VOTE] Apache Geode 1.5.0.RC2

[Swapnil Bawaskar]

This vote passes with six +1 votes, one 0 and zero -1 votes.

Summary:
Sai Boorlagadda +1
Michael Stolz +1
Dan Smith +1
Jinmei Liao +1
Bruce Schuchardt +1
John Blum +0
Anthony Baker +1

On Thu, Apr 5, 2018 at 10:15 AM Anilkumar Gingade 
wrote:

> Sounds like its suggestion to improve the process...I feel that should be
> discussed in different thread; not part of release voting.
>
> -Anil.
>
>
> On Thu, Apr 5, 2018 at 10:10 AM, Jared Stewart 
> wrote:
>
> > Out of curiosity, what was Pulkit's suggestion? I don't see it in this
> > thread.
> >
> > On Thu, Apr 5, 2018, 8:32 AM Alexander Murmann 
> > wrote:
> >
> > > I am very much in favor of Pulkit's suggestion. We've previously
> > discussed
> > > using something like
> > > https://github.com/nebula-plugins/gradle-dependency-lock-plugin. This
> > > would
> > > make a process like Pulkit describes very easy. We could easily be on
> the
> > > latest versions that are known to work and at the same time capture
> which
> > > dependencies cannot easily be upgraded. This would safe lots of manual
> > work
> > > and also provide greater transparency to us into where actual human
> > effort
> > > is required to get back catch up with latest.
> > >
> > > On Thu, Apr 5, 2018 at 8:12 AM, Anthony Baker 
> wrote:
> > >
> > > > I created https://issues.apache.org/jira/browse/GEODE-5001 for this.
> > > >
> > > > Anthony
> > > >
> > > >
> > > > > On Apr 4, 2018, at 5:39 PM, John Blum  wrote:
> > > > >
> > > > > +0
> > > > >
> > > > >
> > > > > The Apache Geode *Log4j* dependency version *2.8.2* is or will
> cause
> > > > > significant issues for apps, and in particular *Spring Boot* 2.0
> > apps.
> > > > >
> > > > > This Geode Log4j version is already quite dated as *Log4j 2.11.0*
> is
> > > now
> > > > > already available [1] and *Spring Boot* 2.0 pulls in *Log4j 2.10.0*
> > > [2].
> > > >
> > > >
> > >
> >
>

[ANNOUNCE] Apache Geode 1.5.0

[Swapnil Bawaskar]

The Apache Geode community is pleased to announce the availability of
Apache Geode 1.5.0.

Apache Geode is a data management platform that provides a database-like
consistency model, reliable transaction processing and a shared-nothing
architecture to maintain very low latency performance with high concurrency
processing.

Geode 1.5.0 contains a number of improvements and bug fixes. This release
adds support for arithmetic operators in the WHERE clause of OQL queries, new
API to destroy a gateway receiver, allows generics to be used with
registerInterest
APIs etc. Users are encouraged to upgrade to the latest release.

For the full list of changes please review the release notes:
https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.5.0

The release artifacts can be downloaded from the project website:
http://geode.apache.org/releases/

The release documentation is available at:
http://geode.apache.org/docs/guide/15/about_geode.html

We would like to thank all the contributors that made the release possible.

Regards,
Swapnil Bawaskar on behalf of the Apache Geode team

Re: [DISCUSS] Release 1.6.0

[Swapnil Bawaskar]

Mike,
The release steps are fairly well documented at:
https://cwiki.apache.org/confluence/display/GEODE/Release+Steps

I will be happy to help if you have any questions.

On Fri, Apr 6, 2018 at 4:03 PM Michael Stolz  wrote:

> I would really appreciate the opportunity to learn how to do the release
> Manager role.
>
> Would someone mentor me?
>
> --
> Mike Stolz
>
> On Apr 6, 2018 6:57 PM, "Alexander Murmann"  wrote:
>
> > Hi everyone!
> >
> > It's great that 1.5.0 just got released and we already have 108 changes
> > ready for 1.6.0. I think we should get all those awesome changes to our
> > users.
> >
> > What does everyone think about starting the release process for 1.6.0?
> >
> > Is anyone interested in taking on the release manager job for 1.6.0?
> >
>

Re: [gemfire-pm] Geode 1.6.0 RC1

[Swapnil Bawaskar]

+1

On Mon, Apr 23, 2018 at 9:30 AM Michael Stolz  wrote:

> The concourse pipeline is green.
> Can I go ahead and start the build for RC1?
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771 <(631)%20835-4771>
> Download the GemFire book here.
> 
>

Re: [VOTE] Apache Geode 1.6.0 RC1

[Swapnil Bawaskar]

+1

On Tue, May 1, 2018 at 2:19 PM Dan Smith  wrote:

> +1
>
> Ran geode-release-check, looks good to me.
>
> -Dan
>
> On Tue, May 1, 2018 at 11:55 AM, Anthony Baker  wrote:
>
> > Ok, thanks Galen.  AFAICT, the KEYS file being referred to is this one:
> > https://dist.apache.org/repos/dist/release/geode/KEYS <
> > https://dist.apache.org/repos/dist/release/geode/KEYS>.  Other Apache
> > projects like Flink, Beam, Impala, or Kafka don’t version control their
> > KEYS file.
> >
> > @PMC - we need more reviews and votes to complete this release in a
> timely
> > manner.  Please check it out.
> >
> > Anthony
> >
> >
> > > On May 1, 2018, at 11:42 AM, Galen O'Sullivan 
> > wrote:
> > >
> > > Thanks for the clarification, Anthony. The release signing page you
> > linked does say this:
> > >
> > > > Since the KEYS may be needed to check signatures for archived
> > releases, it is important that all keys that have ever been used to sign
> > releases are retained in the file. Entries should only be added (as
> > described above), not removed.
> > >
> > > > Your public key should be exported and the result appended to the
> > appropriate KEYS file(s).
> > >
> > > I think we should get Mike's key added to both the develop and release
> > branches. I would prefer if it was present in the release tag (it could
> be
> > confusing for someone checking release history).
> > >
> > > But I guess it shouldn't be too much of a problem if the key isn't in
> > KEYS on the release. It won't affect the binary.
> > >
> > > I'll change to a +0.
> > >
> > > Galen
> > >
> > > On 5/1/18 10:15 AM, Anthony Baker wrote:
> > >> Galen,
> > >>
> > >> Given the above information what are your thoughts?
> > >>
> > >> Anthony
> > >>
> > >>
> > >>> On Apr 30, 2018, at 3:01 PM, Anthony Baker 
> wrote:
> > >>>
> > >>> Please review the ASF policy on signing releases [1].  I think these
> > points are pertinent:
> > >>>
> > >>> - The release manager signs the release.  That provides the
> > verification that the release binaries were in fact created by the
> release
> > manager and have not been modified.  Multiple signatures are not required
> > or even possible sometimes.
> > >>>
> > >>> - The KEYS file in git[2] is a convenience for keeping [3] up to
> > date.  In fact, the KEYS file is a secondary check for a fingerprint at
> > id.apache.org (see [4] for how ASF checks signatures on releases).
> > >>>
> > >>> To me I don’t see a strict necessity to include the KEYS file commit
> > in the release tag.  It’s on the release branch and it will be merged to
> > /develop.
> > >>>
> > >>> $.02,
> > >>> Anthony
> > >>>
> > >>> [1] http://apache.org/dev/release-signing.html
> > >>> [2] https://github.com/apache/geode/blob/develop/KEYS
> > >>> [3] https://dist.apache.org/repos/dist/release/geode/KEYS
> > >>> [4] https://mirror-vm.apache.org/~henkp/checker/faq.html
> > >>>
> >  On Apr 30, 2018, at 10:31 AM, Galen O'Sullivan <
> gosulli...@pivotal.io>
> > wrote:
> > 
> >  -1
> > 
> >  I don't see Mike's key in the KEYS file on either rel/v1.6.0.RC1 (
> > 5ce726bd7b4f8d2648fd011a807a1bcc624ddfa5) or on develop.
> > 
> >  It seems odd to me to add a new key and use it to sign the release
> > without using an already-existing key to sign the release as well. If
> > someone's trying to verify a source tag, there isn't a chain of
> signatures
> > with the last signer of the release signing a commit with the addition of
> > the next new key.
> > 
> >  Galen
> > >
> >
> >
>

Re: Geospatial command support in Redis adapter

[Swapnil Bawaskar]

Thanks Aditya!
I believe this will indeed be a useful contribution!

We as a community have an agreement that each commit should have a JIRA
ticket. If you can register on
https://issues.apache.org/jira/secure/Dashboard.jspa, and let us know your
username, someone here will give you permission to create a JIRA ticket.

On Thu, Jul 26, 2018 at 12:38 PM Aditya Anchuri  wrote:

> Hello,
>
> I recently forked geode on github and started working on adding geospatial
> commands (supported in Redis >3.2) to the Redis adapter (GEOADD, GEOPOS,
> GEOHASH, GEODIST, GEORADIUS and GEORADIUSBYMEMBER). I am wrapping up my
> work on this, but before I make a PR I would like to know if this is a
> feature that people are interested in having? If so, would there be a JIRA
> ticket created? This is my first time contributing to Geode.
>
> Thanks,
> -Aditya
>

Re: Concerning Apache Geode 1.7.0

[Swapnil Bawaskar]

The artifacts probably have not been uploaded yet. I would wait for a vote
thread on the release candidate.

On Wed, Sep 5, 2018 at 2:10 PM John Blum  wrote:

> I see that the branch for Apache Geode 1.7.0 (release/1.7.0 [1]) has been
> created in preparation for the upcoming the 1.7 release.
>
> I am trying to get an early sense/feel for the required changes in SDG when
> I rebase it on 1.7.
>
> Previously, I was able to obtain Geode snapshots from...
>
>
> 
>apache-snapshotshttps://repository.apache.org/content/repositories/snapshots 
>
>
> But currently, Maven is unable to resolve the 1.7 JARs
>
>
> $ mvn dependency:resolve
> [INFO] Scanning for projects...
> [INFO]
> [INFO] -< org.springframework.data:spring-data-geode
> >-
> [INFO] Building Spring Data Geode 2.2.0.BUILD-SNAPSHOT
> [INFO] [ jar
> ]-
> [WARNING] The POM for org.apache.geode:geode-core:jar:1.7.0 is missing, no
> dependency information available
> [WARNING] The POM for org.apache.geode:geode-cq:jar:1.7.0 is missing, no
> dependency information available
> [WARNING] The POM for org.apache.geode:geode-lucene:jar:1.7.0 is missing,
> no dependency information available
> [WARNING] The POM for org.apache.geode:geode-wan:jar:1.7.0 is missing, no
> dependency information available
> [INFO]
> 
> [INFO] BUILD FAILURE
> [INFO]
> 
> [INFO] Total time: 0.617 s
> [INFO] Finished at: 2018-09-05T14:07:30-07:00
> [INFO]
> 
> [ERROR] Failed to execute goal on project spring-data-geode: Could not
> resolve dependencies for project
> org.springframework.data:spring-data-geode:jar:2.2.0.BUILD-SNAPSHOT: *The
> following artifacts could not be resolved*:
> org.apache.geode:geode-core:jar:1.7.0, org.apache.geode:geode-cq:jar:1.7.0,
> org.apache.geode:geode-lucene:jar:1.7.0,
> org.apache.geode:geode-wan:jar:1.7.0: Failure to find
> org.apache.geode:geode-core:jar:1.7.0 *in
> https://repository.apache.org/content/repositories/snapshots
> * was cached
> in the local repository, resolution will not be reattempted until the
> update interval of apache-snapshots has elapsed or updates are forced ->
> [Help 1]
>
>
> Is this the correct repo and are the 1.7 snapshots currently available?
>
> Thanks,
>
> --
> -John
>
> [1] https://github.com/apache/geode/tree/release/1.7.0
>

Re: [Discuss] showcasing community work

[Swapnil Bawaskar]

Would it make sense to put these under an "extensions" page, rather than a
"community" page?

On Tue, Oct 16, 2018 at 5:05 PM Sai Boorlagadda 
wrote:

> Thanks, John for your inputs. I will be adding the page to capture these
> details.
> If there are no objections then I would go ahead and add a new section to
> http://geode.apache.org/community/
>
> Sai
>
> On Thu, Oct 11, 2018 at 12:07 PM John Blum  wrote:
>
> > This is a very nice idea Sai.
> >
> > Perhaps a page with a table containing...
> >
> > * Project Name
> > * Project Description (Summary)
> > * Project License
> > * Project URL (to GitHub page, Website, etc, where users can find out
> more
> > information)
> >
> > Food for thought.
> >
> >
> > On Thu, Oct 11, 2018 at 12:04 PM, Sai Boorlagadda <
> > sai.boorlaga...@gmail.com
> > > wrote:
> >
> > > I would like to discuss what's best to capture all the community work
> in
> > > one place. There are some good plugins/extensions that are built by the
> > > community which does not necessarily be merged into the mainstream but
> > > would be a great value for others in the community. Capturing all these
> > at
> > > one place would a great help for someone looking for a solution.
> > >
> > > Also, I would like to capture (if any) requirements (Eg: licenses etc)
> > that
> > > these developers need to adhere to showcase their work. Are there any
> > > Apache guidelines for such work?
> > >
> > > There are many ways to do this. if everyone agrees about the idea of
> > > capturing all at one place then I propose we add a page/section
> capturing
> > > both the requirements and a table of references URLs to those
> open-source
> > > work. That way it easy for someone to update the page and create a PR
> for
> > > the website.
> > >
> > > Sai
> > >
> >
> >
> >
> > --
> > -John
> > john.blum10101 (skype)
> >
>

FOSDEM

[Swapnil Bawaskar]

If anyone intending to travel to FOSDEM, please sign up for a booth slot
for Geode.

https://cwiki.apache.org/confluence/display/COMDEV/FOSDEM+2019

Thanks!

Re: Removal of distributed transaction tests

[Swapnil Bawaskar]

Implementation for distributed transactions was started but has not been
completed yet. The tests may only be testing what has been implemented so
far, so I would vote not to remove these tests.
On Thu, Feb 9, 2017 at 1:57 PM Kirk Lund  wrote:

> My reasoning is simple: Apache Geode does not currently have a distributed
> transactions feature.
>
>
> On Thu, Feb 9, 2017 at 12:03 PM, Dan Smith  wrote:
>
> > What's wrong with these tests? Are you saying there is coverage
> elsewhere?
> >
> > -Dan
> >
> > On Thu, Feb 9, 2017 at 12:02 PM, Kirk Lund  wrote:
> >
> > > Do we really want our nightly build and precheckin runs to waste time
> > > running these tests?
> > >
> > > DistributedTransactionDUnitTest
> > > DistTXDebugDUnitTest
> > > DistTXOrderDUnitTest
> > > DistTXPersistentDebugDUnitTest
> > > DistTXRestrictionsDUnitTest
> > > DistTXWithDeltaDUnitTest
> > > DistributedTransactionDUnitTest
> > > PRDistTXDUnitTest
> > > PRDistTXWithVersionsDUnitTest
> > >
> > > There are also quite a few non-dunit tests for distributed
> transactions.
> > >
> > > -Kirk
> > >
> >
>

Re: GeodeRedisAdapter improvments/feedback

[Swapnil Bawaskar]

The Redis adapter was designed so that we can scale all the Redis data
structures horizontally. If you bring the data structures to region entry
level, there is no reason for anyone to use our implementation over Redis.

On Tue, Feb 14, 2017 at 3:15 PM Jason Huynh  wrote:

> Hi Hitesh,
>
> Not sure about everyone else, but I had a hard time reading this,  however
> I think I figured out what you were describing... the only part I still am
> unsure about is  Feedback/vote: both behaviour is desirable.  Do you mean
> you want feedback and voting on whether both behaviors are desired?  As in
> old implementation and new implementation?
>
> 2,3,4)  The new implementation would mean all the data for a specific data
> structure is contained in a single bucket.  So the individual data
> structures are not quite scalable.  How would you allow scaling of a single
> data structure?
>
> On Tue, Feb 14, 2017 at 3:05 PM Real Wes  wrote:
>
> In what format do you want the feedback Hitesh?  For now I’ll just comment:
>
> 1. Redis Type String
> No comments except that a future Geode value-add would be to extend the
> Jedis client so that the K/V’s are not compressed. In this way OQL and CQ
> will work.  The tradeoff of this is that the data cannot be read by a
> native redis client but for Geode users it’s great. Call the new client
> Geodis.
>
> 2. List/ Hash/ Set/ SortedSet
> Creating a separate region for each creates a constraint that the keys are
> limited to the characters for region names, which are A-z/0-9/ - and _.
> Everything else is out. Redis users might start asking questions why their
> list named ++^^/## throws an error. Your suggestion to make it a key rather
> than a region solves this. Furthermore, creating a new region every time a
> new Redis collection is created is going to be slow. I’m not sure why a
> region was created but I’m sure it made sense to the developer at the time.
>
> 7. Default Config
> Can’t we configure a gfsh option to default to the region types we want?
> Customer A will want PARTITION but Customer B will want
> PARTITION_REDUNDANT_EXPIRATION_PERSISTENT.  I wonder if we can consider a
> geode> create region —redisType=PARTITION_REDUNDANT_EXPIRATION_PERSISTENT
> that makes _all_ Redis regions of that type?
>
>
>
> On Feb 14, 2017, at 5:36 PM, Hitesh Khamesra  hitesh...@yahoo.com>> wrote:
>
> Current GeodeRedisAdapter implementation is based on
> https://cwiki.apache.org/confluence/display/GEODE/Geode+Redis+Adapter+Proposal
> .
> We are looking for some feedback on Redis commands and their mapping to
> geode region.
>
> 1. Redis Type String
>   a. Usage Set k1 v1
>   b. Current implementation creates "STRING_REGION" geode-partition-region
> upfront
>   c. This k1/v1 are geode-region key/value
>   d. Any feedback?
>
> 2. List Type
>   a. usage "rpush mylist A"
>   b. Current implementation maps each list to geode-partition-region(i.e.
> mylist is geode-partition-region); with the ability to get item from
> head/tail
>   c. Feedback/vote
>   -- List type operation at region-entry level;
>   -- region-key = "mylist"
>   -- region-value = Arraylist (will support all redis list ops)
>   d. Feedback/vote: both behavior is desirable
>
>
> 3. Hashes
>   a. this represents field-value or java bean object
>   b. usage "hmset user1000 username antirez birthyear 1977 verified 1"
>   c. Current implementation maps each hashes to
> geode-partition-region(i.e. user1000 is geode-partition-region)
>   d. Feedback/vote
> -- Should we map hashes to region-entry
> -- region-key = user1000
> -- region-value = map
> -- This will provide java bean sort to behaviour with 10s of
> field-value
> -- Personally I would prefer this..
>   e. Feedback/vote: both behaviour is desirable
>
> 4. Sets
>   a. This represents unique keys in set
>   b. usage "sadd myset 1 2 3"
>   c. Current implementation maps each sadd to geode-partition-region(i.e.
> myset is geode-partition-region)
>   d. Feedback/vote
> -- Should we map set to region-entry
> -- region-key = myset
> -- region-value = Hashset
>   e. Feedback/vote: both behaviour is desirable
>
> 5. SortedSets
>   a. This represents unique keys in set with score (usecase Query top-10)
>   b. usage "zadd hackers 1940 "Alan Kay""
>   c. Current implementation maps each zadd to geode-partition-region(i.e.
> hackers is geode-partition-region)
>   d. Feedback/vote
> -- Should we map set to region-entry
> -- region-key = hackers
> -- region-value = Sorted Hashset
>   e. Feedback/vote: both behaviour is desirable
>
> 6. HyperLogLogs
>   a. A HyperLogLog is a probabilistic data structure used in order to
> count unique things (technically this is referred to estimating the
> cardinality of a set).
>   b. usage "pfadd hll a b c d"
>   c. Current implementation creates "HLL_REGION" geode-partition-region
> upfront
>   d. hll becomes region-key and value is HLL object
>   e. any feedback?
>
> 7. Default config for geode-reg

PROXY and CACHING_PROXY regions on Client

[Swapnil Bawaskar]

GEODE-1887  was filed to
make sure that the user experience while using Geode is similar to RDBMS
and other data products out there. While reviewing the pull request
 I realized that we need to make
other operations propagate to the server as well. These include:
- invalidateRegion()
- destroyRegion()
- getSnapshotService()
- getEntry()
- keySet()
- values()
- isDestroyed()
- containsValueForKey()
- containsKey()
- containsValue()
- entrySet()

Also, rather than have a user "create" a PROXY region, which is just a
handle to a server side region, I would like to propose that
clientCache.getRegion("name") actually creates and returns a PROXY region
even if one was not created earlier/through cache.xml. So, in summary, the
workflow on the client would be:

ClientCacheFactory cacheFactory = new ClientCacheFactory();
cacheFactory.addPoolLocator("localhost", 10334);
ClientCache clientCache = cacheFactory.create();

Region students = clientCache.getRegion("students");
students.put("student1", "foo");
assert students.size() == 1;

If a client wants to have a near cache, they can still "create" a
CACHING_PROXY region.

For a CACHING_PROXY, I propose that we leave the default implementation
unchanged, i.e. all operations work locally on the client (except CRUD
operations that are always propagated to the server). In the case where the
client wishes to perform operations on the server, I propose that we
introduce a new method:

/**
 * @return
 */
Region serverView();

so that all operations on the returned view (Region) are performed on the
server.

In the longer term, we should break up Region into two interfaces, one that
has methods that only work on the client (like registerInterest and
serverView()) and other for the server.

Thanks!
Swapnil.

Re: PROXY and CACHING_PROXY regions on Client

[Swapnil Bawaskar]

@John The intention behind this proposal is to make Geode client
development easy for new users. So, looking at this as a new user, I would
say that having to "create" a PROXY region only to find out that it does
nothing on the server, is more confusing than an overloaded getRegion().

To summarize, the proposal for getRegion() is:
1. lookup if the region exists already and return it; this applies to
regions that have been created through API and cache.xml. This is the
current behavior.
2. If the region does not exist:
2.a. check if it exists on the server, if so create a PROXY region under
the covers and return it. Do this only on the client
2.b. If it does not exist on the server, throw an exception.



On Wed, Feb 15, 2017 at 9:38 AM John Blum  wrote:

> @Eric-
>
> Hmm...
>
> Well, I'd argue that it is still confusing to "*overload*" the purpose of
> getRegion("path") to dually "*get*" (the primary function/purpose) and also
> "*create*" (secondary).
>
> I'd also say that the getRegion("path") API call is not exclusive to a
> *ClientCache*, particularly since getRegion("path") is on RegionService
> <
> http://data-docs-samples.cfapps.io/docs-gemfire/latest/javadocs/japi/com/gemstone/gemfire/cache/RegionService.html#getRegion(java.lang.String)
> >
> [1],
> which both ClientCache and Cache implement, indirectly through
> GemFireCache,
> I might add.  Therefore, getRegion("path") has a completely different
> meaning server-side (or in the embedded "peer cache" UC).
>
> -j
>
> [1]
>
> http://data-docs-samples.cfapps.io/docs-gemfire/latest/javadocs/japi/com/gemstone/gemfire/cache/RegionService.html#getRegion(java.lang.String)
>
> On Wed, Feb 15, 2017 at 9:29 AM, Anthony Baker  wrote:
>
> > Introducing an API like this gives us the opportunity to split the
> > client/server region API’s.  I don’t think we should return Region, but
> > something specific to “server view”.  How would those API’s operate on a
> > CACHING_PROXY?
> >
> > Anthony
> >
> > > On Feb 15, 2017, at 6:44 AM, Swapnil Bawaskar 
> > wrote:
> > >
> > > /**
> > > * @return
> > > */
> > > Region serverView();
> > >
> >
> >
>
>
> --
> -John
> john.blum10101 (skype)
>

Re: PROXY and CACHING_PROXY regions on Client

[Swapnil Bawaskar]

Re: breaking existing code; we could throw UnsupportedOperationException
for these two methods:
- invalidateRegion()
- destroyRegion()

I do not see anyone using/depending on invalidateRegion, since the behavior
currently is a no-op.
destroyRegion currently only gets rid of the handle to the region, so it is
most likely being used when the client is shutting down, so fixing existing
applications should be straight forward.

On Wed, Feb 15, 2017 at 12:27 PM Swapnil Bawaskar 
wrote:

> @John The intention behind this proposal is to make Geode client
> development easy for new users. So, looking at this as a new user, I would
> say that having to "create" a PROXY region only to find out that it does
> nothing on the server, is more confusing than an overloaded getRegion().
>
> To summarize, the proposal for getRegion() is:
> 1. lookup if the region exists already and return it; this applies to
> regions that have been created through API and cache.xml. This is the
> current behavior.
> 2. If the region does not exist:
> 2.a. check if it exists on the server, if so create a PROXY region under
> the covers and return it. Do this only on the client
> 2.b. If it does not exist on the server, throw an exception.
>
>
>
> On Wed, Feb 15, 2017 at 9:38 AM John Blum  wrote:
>
> @Eric-
>
> Hmm...
>
> Well, I'd argue that it is still confusing to "*overload*" the purpose of
> getRegion("path") to dually "*get*" (the primary function/purpose) and also
> "*create*" (secondary).
>
> I'd also say that the getRegion("path") API call is not exclusive to a
> *ClientCache*, particularly since getRegion("path") is on RegionService
> <
> http://data-docs-samples.cfapps.io/docs-gemfire/latest/javadocs/japi/com/gemstone/gemfire/cache/RegionService.html#getRegion(java.lang.String)
> >
> [1],
> which both ClientCache and Cache implement, indirectly through
> GemFireCache,
> I might add.  Therefore, getRegion("path") has a completely different
> meaning server-side (or in the embedded "peer cache" UC).
>
> -j
>
> [1]
>
> http://data-docs-samples.cfapps.io/docs-gemfire/latest/javadocs/japi/com/gemstone/gemfire/cache/RegionService.html#getRegion(java.lang.String)
>
> On Wed, Feb 15, 2017 at 9:29 AM, Anthony Baker  wrote:
>
> > Introducing an API like this gives us the opportunity to split the
> > client/server region API’s.  I don’t think we should return Region, but
> > something specific to “server view”.  How would those API’s operate on a
> > CACHING_PROXY?
> >
> > Anthony
> >
> > > On Feb 15, 2017, at 6:44 AM, Swapnil Bawaskar 
> > wrote:
> > >
> > > /**
> > > * @return
> > > */
> > > Region serverView();
> > >
> >
> >
>
>
> --
> -John
> john.blum10101 (skype)
>
>

Re: Requesting more JIRA permission

[Swapnil Bawaskar]

+1

On Wed, Feb 15, 2017 at 4:29 PM Jared Stewart  wrote:

> +1
>
> > On Feb 15, 2017, at 4:27 PM, Michael William Dodge 
> wrote:
> >
> > +1
> >
> >> On 15 Feb, 2017, at 16:17, Addison Huddy  wrote:
> >>
> >> Hello Apache Geode Community,
> >>
> >> I'm writing to request a higher level of permission on JIRA tickets in
> an
> >> effort to increase organization and clarity within our community.
> >>
> >> As we grow our community and focus our mutual development efforts, it is
> >> important to stay organized. For example:
> >>
> >>  - how we organize jira's and sub-jiras
> >>  - Updating the status of a ticket
> >>  - What we call the ticket so that it is informative
> >>
> >> In my daily work on Geode at Pivotal Inc, I find myself wanting to
> organize
> >> things within the community, but don't have the permissions to edit
> tickets
> >> besides by own.
> >>
> >> I fully recognize that with more ticket permissions comes a greater
> >> responsibility to communicate with the community and uphold our
> democratic
> >> model.  I will make sure to keep and promote these values.
> >>
> >> *Please +1 this request if you agree to increase my JIRA permissions so
> I
> >> can edit JIRAs.*
> >>
> >> Thanks,
> >> Addison
> >
>
>

Re: GeodeRedisAdapter improvments/feedback

[Swapnil Bawaskar]

I think we as a community need to determine what value do we want to add
with the Redis adapter. Redis already does a great job storing small data
structures in memory and sharding them. We do a great job of making sure
that these data structures are horizontally scalable; why would we want to
replicate what another open source project is already implementing?

Having said that, I would like to see a configuration property that lets
the user chose between a single server vs a distributed collection.


> I think we could have the following options:
>
>  1. have a property that could be set to use either single server
> collections over use the current distributed collection
>  2. have first class collection implementations that are distributed by
> nature, as using key:value as the hammer for all does not make sense
>

I don't think these options are mutually exclusive. We should make lists
and SortedSets first class data structures in Geode alongside regions.

Re: [jira] [Created] (GEODE-2435) Redis adapter MULTI behavior is different from Redis

[Swapnil Bawaskar]

Indeed, all you need to do is set -Dgemfire.ALLOW_PERSISTENT_TRANSACTIONS
to true

On Tue, Feb 21, 2017 at 5:22 PM Michael Stolz  wrote:

> This looks like the persistence with transactions issue that Gemfire has.
> There is a system property to allow it to work
>
> --
> Mike Stolz
> Principal Engineer - Gemfire Product Manager
> Mobile: 631-835-4771 <(631)%20835-4771>
>
> On Feb 6, 2017 4:41 PM, "Galen O'Sullivan (JIRA)"  wrote:
>
> > Galen O'Sullivan created GEODE-2435:
> > ---
> >
> >  Summary: Redis adapter MULTI behavior is different from
> Redis
> >  Key: GEODE-2435
> >  URL: https://issues.apache.org/jira/browse/GEODE-2435
> >  Project: Geode
> >   Issue Type: Bug
> > Reporter: Galen O'Sullivan
> >
> >
> > {{WATCH}} isn't implemented properly, but this is about returning an
> error
> > instead of nil when we have a {{MULTI}} fail:
> > {code}
> > $ redis-cli -p 11212
> > 127.0.0.1:11212> set a b
> > OK
> > 127.0.0.1:11212> watch a
> > (error) ERR Keys cannot be watched or unwatched because GemFire watches
> > all keys by default for transactions
> > 127.0.0.1:11212> lpush la boo
> > (integer) 1
> > (2.09s)
> > 127.0.0.1:11212> multi
> > OK
> > 127.0.0.1:11212> lpush la z
> > QUEUED
> > 127.0.0.1:11212> lpush la x
> > QUEUED
> > {code}
> > At this point, we {{lpush la foo}} in a different client, then:
> > {code}
> > 127.0.0.1:11212> exec
> > 1) (error) ERR The server had an internal error please try again
> > 2) (error) ERR The server had an internal error please try again
> > 127.0.0.1:11212>
> > {code}
> >
> > whereas a Redis instance will simply return nil instead of an error.
> >
> > Looking in the logs, I see this:
> > {code}
> > [error 2017/02/06 13:21:39.493 PST server2
> 
> > tid=0x2a] GeodeRedisServer-Unexpected error handler for [id: 0x3ddf9f21,
> /
> > 127.0.0.1:58862 => /127.0.0.1:11212]
> > java.lang.UnsupportedOperationException: Operations on persist-backup
> > regions are not allowed because this thread has an active transaction
> > at org.apache.geode.internal.cache.TXRegionState.(
> > TXRegionState.java:60)
> > at org.apache.geode.internal.cache.TXBucketRegionState.<
> > init>(TXBucketRegionState.java:29)
> > at org.apache.geode.internal.cache.TXState.writeRegion(
> > TXState.java:252)
> > at org.apache.geode.internal.cache.TXState.txWriteRegion(
> > TXState.java:1110)
> > at org.apache.geode.internal.cache.TXState.txReadEntry(
> > TXState.java:1365)
> > at org.apache.geode.internal.cache.TXState.txReadEntry(
> > TXState.java:1344)
> > at org.apache.geode.internal.cache.TXState.
> > getDeserializedValue(TXState.java:1414)
> > at org.apache.geode.internal.cache.TXStateProxyImpl.
> > getDeserializedValue(TXStateProxyImpl.java:352)
> > at org.apache.geode.internal.cache.LocalRegion.get(
> > LocalRegion.java:1394)
> > at org.apache.geode.internal.cache.PartitionedRegionDataStore.
> > getLocally(PartitionedRegionDataStore.java:2047)
> > at org.apache.geode.internal.cache.PartitionedRegion.
> > getFromBucket(PartitionedRegion.java:4022)
> > at org.apache.geode.internal.cache.PartitionedRegion.
> > findObjectInSystem(PartitionedRegion.java:3399)
> > at org.apache.geode.internal.cache.TXState.findObject(
> > TXState.java:1540)
> > at org.apache.geode.internal.cache.TXStateProxyImpl.
> > findObject(TXStateProxyImpl.java:614)
> > at org.apache.geode.internal.cache.PartitionedRegion.get(
> > PartitionedRegion.java:3160)
> > at org.apache.geode.internal.cache.LocalRegion.get(
> > LocalRegion.java:1330)
> > at org.apache.geode.internal.cache.AbstractRegion.get(
> > AbstractRegion.java:282)
> > at org.apache.geode.redis.internal.executor.list.
> > ListExecutor.pushElements(ListExecutor.java:70)
> > at org.apache.geode.redis.internal.executor.list.
> > PushExecutor.executeCommand(PushExecutor.java:47)
> > at org.apache.geode.redis.internal.ExecutionHandlerContext.
> > executeWithTransaction(ExecutionHandlerContext.java:244)
> > at org.apache.geode.redis.internal.ExecutionHandlerContext.
> > executeCommand(ExecutionHandlerContext.java:191)
> > at org.apache.geode.redis.internal.ExecutionHandlerContext.
> > channelRead(ExecutionHandlerContext.java:137)
> > at io.netty.channel.DefaultChannelHandlerContext.
> > invokeChannelRead(DefaultChannelHandlerContext.java:368)
> > at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(
> > DefaultChannelHandlerContext.java:353)
> > at io.netty.handler.codec.ByteToMessageDecoder.channelRead(
> > ByteToMessageDecoder.java:173)
> > at io.netty.channel.DefaultChannelHandlerContext.
> > invokeChannelRead(DefaultChannelHandlerContext.java:368)
> > at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(
> > DefaultChannelHandl

Re: [DISCUSS] changes to Redis implementation

[Swapnil Bawaskar]

Accepting this pull request as-is will break backwards compatibility. I
think if the new behavior is desired, it should be based on some
configuration.

On Mon, Feb 27, 2017 at 11:40 AM Bruce Schuchardt 
wrote:

> I think in this case it would be replacing the messages used to
> replicate changes with ones that send the operation + parameters instead
> of the modified entry's new value.  That could be done by creating a new
> subclass of BucketRegion.  Stick the operation+params into the
> EntryEventImpl that follows the operation through the system and in the
> BucketRegion subclass pull out that data and send it instead of the
> event's newValue.
>
> Le 2/27/2017 à 11:13 AM, Real Wes a écrit :
> > I'm not following what a "simple operation replication framework" is and
> how it applies to the Redis API. If you replicate operations, you still
> need to update the data at some point, causing a synchronous replication
> event so as to provide HA. What is, in more detail, a "simple operation
> replication framework"?
> >
> > Regards,
> > Wes Williams
> > Sent from mobile phone
> >
> >> On Feb 27, 2017, at 2:07 PM, Bruce Schuchardt 
> wrote:
> >>
> >> What I like about using the existing delta-prop mechanism is it will
> also extend to client subscriptions & WAN.  It would take a lot of work to
> propagate Redis commands through those paths.
> >>
> >>> Le 2/27/2017 à 10:35 AM, Hitesh Khamesra a écrit :
> >>> The simplicity of Redis API making this problem (delta update)most
> apparent. But,  I would imagine many Geode apps has a similar use case.
> >>>
> >>> -Hitesh
> >>>
> >>>From: Michael Stolz 
> >>>   To: dev@geode.apache.org
> >>>   Sent: Monday, February 27, 2017 9:06 AM
> >>>   Subject: Re: [DISCUSS] changes to Redis implementation
> >>> It does seem like the operations will often be much smaller than
> the data
> >>> they are operating on.
> >>> It is almost the classic "move the code to the data" pattern.
> >>>
> >>> --
> >>> Mike Stolz
> >>> Principal Engineer, GemFire Product Manager
> >>> Mobile: +1-631-835-4771 <(631)%20835-4771>
> >>>
> >>> On Mon, Feb 27, 2017 at 10:51 AM, Udo Kohlmeyer  >
> >>> wrote:
> >>>
>  Delta could provide us a mechanism to replicate only what is required.
> 
>  I wonder if we could not create a simple operation replication
> framework.
>  Rather than writing a potential large amounts of code for delta, we
>  replicate only the operation.
> 
> 
> > On 2/27/17 07:18, Wes Williams wrote:
> >
> > Replicating a whole collection because of 1 change does not really
> make
> >> too much sense.<<
> > I agree but won't delta replication prevent sending the entire
> collection
> > across the wire?
> >
> > *Wes Williams | Pivotal Advisory **Data Engineer*
> > 781.606.0325 <(781)%20606-0325>
> > http://pivotal.io/big-data/pivotal-gemfire
> >
> > On Mon, Feb 27, 2017 at 10:08 AM, Udo Kohlmeyer <
> ukohlme...@pivotal.io>
> > wrote:
> >
> > I've quickly gone through the changes for the pull request.
> >> The most significant change of this pull request is that the
> collections
> >> that initially were regions are single collections (not
> distributed).
> >> That
> >> said, this is something that we've been discussing. The one thing
> that I
> >> wonder about is, what will the performance look like when the
> collections
> >> become really large? Replicating a whole collection because of 1
> change
> >> does not really make too much sense.
> >>
> >> Maybe this implementation becomes the catalyst for future
> improvements.
> >>
> >> --Udo
> >>
> >>
> >>
> >> On 2/24/17 15:25, Bruce Schuchardt wrote:
> >>
> >> Gregory Green has posted a pull request that warrants discussion. It
> >>> improves performance for Sets and Hashes by altering the storage
> format
> >>> for
> >>> these collections.  As such it will not permit a rolling upgrade,
> though
> >>> the Redis adapter is labelled "experimental" so maybe that's okay.
> >>>
> >>> https://github.com/apache/geode/pull/404
> >>>
> >>> The PR also fixes GEODE-2469, inability to handle hash keys having
> >>> colons.
> >>>
> >>> There was some discussion about altering the storage format that
> was
> >>> initiated by Hitesh.  Personally I think Gregory's changes are
> better
> >>> than
> >>> the current implementation and we should accept them, though I
> haven't
> >>> gone
> >>> through the code changes extensively.
> >>>
> >>>
> >>>
> >>>
>
>

Re: export logs and stats

[Swapnil Bawaskar]

+1 for restricting the file extensions and looking at what the user has
defined.

On Tue, Feb 28, 2017 at 4:46 PM Jinmei Liao  wrote:

> Yeah, before we were simply looking at System.getProperty("user.dir") to
> get all the logs/stats, now I think we should use whatever user defined in
> the config file and do a getParent to get the parent directory to search
> for the files.
>
> On Tue, Feb 28, 2017 at 4:34 PM, Dan Smith  wrote:
>
> > I think maybe I didn't quite understand the original proposal. Are you
> > saying you won't even look at the directory or filename the user
> specifies,
> > but just grab all the files that happen to be in the working directory
> and
> > end in .log? I don't think that's going to do the right thing for most
> > users.
> >
> > Very commonly users do direct their logs to separate directory, at least
> a
> > subdirectory of their working directory. And they often put all of their
> > logs into that directory - application logs, gemfire logs, etc. So I
> think
> > you really do need to look at the directory and filename they are
> > specifying.
> >
> > Now, if you are just saying that their filename has to end in .log, that
> > seems like a reasonable restriction. It seems like maybe we already have
> > that restriction if you're hitting an IndexOutOfBoundError.
> >
> > -Dan
> >
> > On Tue, Feb 28, 2017 at 4:20 PM, Jinmei Liao  wrote:
> >
> > > Darrel, it seems if user defines a log file to be simply "serverLog" (a
> > > filename with no "."), when rolling over the log file once file size is
> > > reached, we get a IndexOutOfBoundError when it's trying to figure out
> > > what's the rolled-over filename should be.
> > >
> > > And if user defines a log file name to be "server.log.gz", when rolling
> > > over, the new filename seems to be "server.log_01_01.gz". Does not look
> > > like we handle the ".gz' suffix correctly. For now, we do want to
> enforce
> > > the log/stats filename ends with appropriate suffix.
> > >
> > > On Tue, Feb 28, 2017 at 3:16 PM, Darrel Schneider <
> dschnei...@pivotal.io
> > >
> > > wrote:
> > >
> > > > It sounds like you will pick up any file in the working directory
> that
> > > ends
> > > > with ".log" or ".log.gz".
> > > > But what if the geode server is sharing a directory with something
> else
> > > > that is also writing files with these extensions?
> > > > Or if multiple geode servers are running in the same directory?
> > > > I think it would be better to use the configured log file name and
> stat
> > > > archive name and use that to find the logs and stats to gather. The
> > > rolling
> > > > code has already been written that will find all the existing logs
> and
> > > > stats. In any bugs in that code need to be fixed since it would break
> > the
> > > > code that removes old files based on disk space. So it seems like you
> > > > should be able to use this same code to get a list of the files to
> > copy.
> > > >
> > > >
> > > > On Tue, Feb 28, 2017 at 2:57 PM, Dan Smith 
> wrote:
> > > >
> > > > > I'm a bit confused by (1). Isn't it actually more complicated for
> you
> > > to
> > > > > restrict log collection to a relative path? Why not just look for
> log
> > > > files
> > > > > no matter where they are written to? I also don't really follow the
> > > > > argument about why a user that writes to /var/logs is not going to
> > want
> > > > to
> > > > > use this command. Won't all users want to be able to gather their
> > logs
> > > > > using this command?
> > > > >
> > > > > 2 seems reasonable. It seems like we should restrict the file names
> > if
> > > we
> > > > > are going to have this limitation.
> > > > >
> > > > > -Dan
> > > > >
> > > > > On Tue, Feb 28, 2017 at 2:43 PM, Jinmei Liao 
> > > wrote:
> > > > >
> > > > > > Hello community,
> > > > > >
> > > > > > We are currently trying to improve what "export logs" should do.
> > > > > Currently
> > > > > > export logs only export the logs(filtered by logLevel and start
> and
> > > end
> > > > > > date) to each individual member's file system. We want to make
> all
> > > the
> > > > > > member's logs exported to a central location  and if you are
> > > connecting
> > > > > > using http, it will be exported to your local file system. This
> is
> > to
> > > > > > facilitate gathering logs in the cloud environment.
> > > > > >
> > > > > > That said, for the first round of implementation, we would like
> to
> > > > impose
> > > > > > these restrictions to this command:
> > > > > > 1) it will only look for the logs/stats in each members working
> > > > directory
> > > > > > only.
> > > > > > 2) it will only look for files that ends with .log, .log.gz, .gfs
> > or
> > > > > > .gfs.gz.
> > > > > >
> > > > > > Background for 1): if you started your locator/server with
> > "log-file"
> > > > or
> > > > > > "statistics-archive-file" with an absolute path, it will write
> > these
> > > > > files
> > > > > > to that location, but if you simply give it a relative path, the
> > > files
> > > > > wi

Re: Geode log levels vs. log4j log levels

[Swapnil Bawaskar]

+1 to making the change and updating the docs to reflect only the new log
levels.

On Wed, Mar 15, 2017 at 10:45 AM Jinmei Liao  wrote:

> Hi, all,
>
> Geode, before using log4j for logging has implemented its own LogWriters
> and logLevels. Geode log levels has values like "finest, finer, fine,
> config, info, warning, error, severe".  We've moved away from using these
> log writers to favor using log4j logging. So, mostly, geode logs have log4j
> log statements with levels like "fatal, error, warn, info, debug".
>
> But by examining our existing commands that has log-level options,
> specifically "start server", "start locator", "alter runtime", "change
> log-level", these commands are still accepting log-levels with geode log
> levels, and under the cover, it has a mapping to the log4j levels. Since we
> are moving away from geode's own logging, I would like to propose changing
> these commands to use log4j log levels explicitly.
>
> To avoid breaking existing commands, we can provide a mapping to the log4j
> levels after user provides an "old" log level, but at least I would like to
> have the auto complete start showing the log4j levels instead of the geode
> log levels and move towards getting rid of these old levels completely.
>
> --
> Cheers
>
> Jinmei
>

Re: GEODE-2714 Proposal for new api on Function interface (Please read)

[Swapnil Bawaskar]

Here is the link: https://issues.apache.org/jira/browse/GEODE-2714

Hi Hitesh,

Before executing a function, we build a list of buckets that the function
needs to execute on and then we build the function context using only those
buckets. So, as long as the function is using the context to get the local
data set, it should not see the data twice, even if rebalance was in
progress.

When you saw this behavior, was the function using the function context?

Thanks!


On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart  wrote:

> Can you give a link to the proposal?
>
> Thanks,
> Jared
> > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra
>  wrote:
> >
> > Please let us know your opinion on that.
> > Thanks.Hitesh
>
>

Re: GEODE-2714 Proposal for new api on Function interface (Please read)

[Swapnil Bawaskar]

For step 2 above, won't we end up recomputing the list of buckets?

I agree, however,  that we do have the problem of moving primary,
especially in the middle of iterating over data. We already have an
optimizeForWrite() method on Function that makes the function execute on
the primary. Do we want to make locking part of this method, rather than
introducing a new method?

On Fri, Mar 24, 2017 at 2:10 PM Hitesh Khamesra  wrote:

> Hi Swapnil:
>
> Right, and then we execute that function on that node. Now assume during
> that function execution time(first user thread) two things happens
> simultaneously..
>
> 1. we move that bucket to other node.
> 2. And other user thread execute function with same set of keys on other
> node, which first  thread is still executing.
>
> Basically we need to make sure while function execution we don't move the
> primary bucket.
>
> Thanks.
> Hitesh
> --
> *From:* Swapnil Bawaskar 
> *To:* dev@geode.apache.org; Hitesh Khamesra 
> *Sent:* Friday, March 24, 2017 1:55 PM
> *Subject:* Re: GEODE-2714 Proposal for new api on Function interface
> (Please read)
>
> Here is the link: https://issues.apache.org/jira/browse/GEODE-2714
>
> Hi Hitesh,
>
> Before executing a function, we build a list of buckets that the function
> needs to execute on and then we build the function context using only those
> buckets. So, as long as the function is using the context to get the local
> data set, it should not see the data twice, even if rebalance was in
> progress.
>
> When you saw this behavior, was the function using the function context?
>
> Thanks!
>
>
> On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart  wrote:
>
> Can you give a link to the proposal?
>
> Thanks,
> Jared
> > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra
>  wrote:
> >
> > Please let us know your opinion on that.
> > Thanks.Hitesh
>
>
>
>