Re: [DISCUSS] Redundancy Gfsh Commands

[Joris Melchior]

+1

I like this idea and Kirk's suggestion to use the CompletableFuture as a
standard for asynchronous operations.

On Mon, Mar 30, 2020 at 2:47 PM Donal Evans  wrote:

> Hey everyone,
>
> An RFC for adding gfsh commands to allow users to restore redundancy to
> partitioned regions and to easily check the redundancy status of
> partitioned regions has been posted:
> https://cwiki.apache.org/confluence/display/GEODE/Redundancy+Gfsh+Commands
> .
>
> Please review and comment on this RFC by Monday, 04/06/2020.
>
> Please also note that this RFC has been revised from an earlier draft
> version that some of you may have seen, so the content may have changed.
>
> Thanks,
> Donal
>


-- 
*Joris Melchior *
CF Engineering
Pivotal Toronto
416 877 5427

“Programs must be written for people to read, and only incidentally for
machines to execute.” – *Hal Abelson*

[DISCUSS] Replace UDP messaging for membership with TCP

[Dan Smith]

Hi all,

We created a RFC for replacing our UDP messaging  in Geode with a TCP based
solution. This will address the issues we have supporting our current udp
encryption solution, along with helping us move away from jgroups, which
currently can't be upgraded.

Please review and comment by 4/7/2020.

https://cwiki.apache.org/confluence/display/GEODE/Replace+UDP+messaging+for+membership+with+TCP

Thanks!
-Dan

Re: [DISCUSS] Replace UDP messaging for membership with TCP

[Mark Hanson]

Can we document the protocol this time?

Thanks,
Mark

> On Mar 31, 2020, at 10:17 AM, Dan Smith  wrote:
> 
> Hi all,
> 
> We created a RFC for replacing our UDP messaging  in Geode with a TCP based
> solution. This will address the issues we have supporting our current udp
> encryption solution, along with helping us move away from jgroups, which
> currently can't be upgraded.
> 
> Please review and comment by 4/7/2020.
> 
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FGEODE%2FReplace%2BUDP%2Bmessaging%2Bfor%2Bmembership%2Bwith%2BTCP&data=02%7C01%7Chansonm%40vmware.com%7C5419f5b82bd94c98484a08d7d597724e%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637212718744094716&sdata=yF7o7Cns%2B1QOGQKKvXhJmVtnZGIiCjsuOskNjIEd8Ag%3D&reserved=0
> 
> Thanks!
> -Dan

Re: RFC: Add C Bindings to Geode Native Client

[Matthew Reddington]

I would like to request the addition of two new repositories under Apache in 
order to implement this RFC and to take advantage of it in practice. That would 
be Apache/geode-c-client and Apache/geode-dot-net-core-client.

> On Mar 30, 2020, at 3:33 PM, Jacob Barrett  wrote:
> 
> https://github.com/pivotal-jbarrett/geode-native/tree/ee34cfbb5bddb55f5f890bb013c75d7780a787ae/ccache
>  
> 
> 
> Quick stab at a POC for thread exception handling.
> 
> -Jake
> 

[ANNOUNCE] Apache Geode 1.12.0

[Ernest Burghardt]

The Apache Geode community is pleased to announce the availability of
Apache Geode 1.12.0.

Apache Geode is a data management platform that provides a database-like
consistency model, reliable transaction processing and a shared-nothing
architecture to maintain very low latency performance with high concurrency
processing.

Geode {version} contains a number of improvements and bug fixes. It
includes performance improvements in OQL order-by and distinct queries
in client/server when security is enabled. New GFSH commands were
added to get/set cluster config and to destroy gateway receivers. A
new post processor was added to the new client protocol. Pulse now
supports legacy SSL options. Auto-reconnecting members no more reuse
old addresses and IDs. Duplicated or member-specific receivers are
removed from cluster config during rolling upgrades. Users are
encouraged to upgrade to the latest release.
For the full list of changes please review the release
notes:https://cwiki.apache.org/confluence/display/GEODE/
Release+Notes#ReleaseNotes-1.12.0

The release artifacts can be downloaded from the project
website:http://geode.apache.org/releases/

The release documentation is available
at:http://geode.apache.org/docs/guide/112/about_geode.html

We would like to thank all the contributors that made the release possible.
Regards,
Ernie Burghardt on behalf of the Apache Geode team

Re: [DISCUSS] RFC: Shipping Geode patch releases

[Owen Nichols]

With the release of Geode 1.12.0 (awesome!), we have a new branch naming 
convention starting today:

release/1.12.0 is now support/1.12

As per this RFC, support/1.12 and an accompanying pipeline[1] will be 
maintained until Dec 31 2020.

Critical fixes may be proposed at any time.  The fix should already be on 
develop.  If it is not a clean&simple cherry-pick, please prepare a PR against 
desired support branch(es).

The usual three “+1” votes are needed to bring a proposed fix.

Anyone in the Geode community may at any time propose making a patch release 
from a support branch.

-Owen

[1] 
https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-12-main

> On Mar 13, 2020, at 4:10 PM, Anthony Baker  wrote:
> 
> Based on the consensus in this thread, we will move forward with this
> proposal.  We can work out the exact mechanics after we release
> v1.12.0 and move that into a support mode.
> 
> Thanks for all the feedback.
> 
> Anthony
> 
> 
> On Fri, Feb 21, 2020 at 5:30 PM Anthony Baker  wrote:
>> 
>> Hi everyone,
>> 
>> I'd like to propose shipping patch releases of Geode as a way to
>> improve our engagement and support of our user community.  Please
>> review the details in the linked RFC [1] and kindly offer your
>> thoughts in this thread.
>> 
>> 
>> Thanks,
>> Anthony
>> 
>> [1] https://cwiki.apache.org/confluence/display/GEODE/Shipping+patch+releases

Errored: apache/geode-examples#432 (release/1.12.0 - 0b27ac9)

[Travis CI]

Build Update for apache/geode-examples
-

Build: #432
Status: Errored

Duration: 20 secs
Commit: 0b27ac9 (release/1.12.0)
Author: Ernie Burghardt
Message: Revert "temporarily point to staging repo for CI purposes"

View the changeset: 
https://github.com/apache/geode-examples/compare/2e32684883b7...0b27ac9b9c22

View the full build log and details: 
https://travis-ci.org/github/apache/geode-examples/builds/669368329?utm_medium=notification&utm_source=email

--

You can unsubscribe from build emails from the apache/geode-examples repository 
going to 
https://travis-ci.org/account/preferences/unsubscribe?repository=11483039&utm_medium=notification&utm_source=email.
Or unsubscribe from *all* email updating your settings at 
https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notification&utm_source=email.
Or configure specific recipients for build notifications in your .travis.yml 
file. See https://docs.travis-ci.com/user/notifications.

Re: RFC: Add C Bindings to Geode Native Client

[Jacob Barrett]

Given that the C-binding will be tightly coupled with the C++ layer and written 
in C++ I don’t think it make sense to have its own repository. In order for the 
C-binding need to access internal, non-exported methods, for things like 
serialization it will need to be statically linked to the C++ library. If we 
use separate repositories and builds then we need to build a static library for 
the dependency in the C-binding library. This seems overly complicated to me 
for little to no gain. Can you please elaborate on the the pros of having a 
separate repository for the C-binding?

For sure agree on the .net core repo. Tough depending on the mixed mode binary 
support we still might run into the dependency issues on static library. 

-Jake


> On Mar 31, 2020, at 11:23 AM, Matthew Reddington  
> wrote:
> 
> I would like to request the addition of two new repositories under Apache in 
> order to implement this RFC and to take advantage of it in practice. That 
> would be Apache/geode-c-client and Apache/geode-dot-net-core-client.
> 
>> On Mar 30, 2020, at 3:33 PM, Jacob Barrett  wrote:
>> 
>> https://github.com/pivotal-jbarrett/geode-native/tree/ee34cfbb5bddb55f5f890bb013c75d7780a787ae/ccache
>>  
>> 
>> 
>> Quick stab at a POC for thread exception handling.
>> 
>> -Jake
>> 
> 

Re: RFC: Add C Bindings to Geode Native Client

[Blake Bender]

Just want to make sure I understand what you're after here.  We should have
a "ccache" directory or similar in the geode-native repo, where we build C
bindings for the client, then we should compile them into a shared library
containing all of the code, and export/make visible only the C interface?
So the geode native installer will contain shared libraries representing 3
copies of all the code, with the static library in the build tree making a
total of 4?

I'm starting to be concerned with the overall inefficiency of this plan.
 Is this the best we can do?

Thanks,

Blake


On Tue, Mar 31, 2020 at 12:05 PM Jacob Barrett  wrote:

> Given that the C-binding will be tightly coupled with the C++ layer and
> written in C++ I don’t think it make sense to have its own repository. In
> order for the C-binding need to access internal, non-exported methods, for
> things like serialization it will need to be statically linked to the C++
> library. If we use separate repositories and builds then we need to build a
> static library for the dependency in the C-binding library. This seems
> overly complicated to me for little to no gain. Can you please elaborate on
> the the pros of having a separate repository for the C-binding?
>
> For sure agree on the .net core repo. Tough depending on the mixed mode
> binary support we still might run into the dependency issues on static
> library.
>
> -Jake
>
>
> > On Mar 31, 2020, at 11:23 AM, Matthew Reddington 
> wrote:
> >
> > I would like to request the addition of two new repositories under
> Apache in order to implement this RFC and to take advantage of it in
> practice. That would be Apache/geode-c-client and
> Apache/geode-dot-net-core-client.
> >
> >> On Mar 30, 2020, at 3:33 PM, Jacob Barrett  wrote:
> >>
> >>
> https://github.com/pivotal-jbarrett/geode-native/tree/ee34cfbb5bddb55f5f890bb013c75d7780a787ae/ccache
> <
> https://github.com/pivotal-jbarrett/geode-native/tree/ee34cfbb5bddb55f5f890bb013c75d7780a787ae/ccache
> >
> >>
> >> Quick stab at a POC for thread exception handling.
> >>
> >> -Jake
> >>
> >
>
>

Re: RFC: Add C Bindings to Geode Native Client

[Dan Smith]

Once we do have agreement on what new repositories we want, I think any pmc
member should be able to create them on gitbox.apache.org. Deleting them if
we decide we don't want them is harder :)

-Dan

On Tue, Mar 31, 2020, 12:26 PM Blake Bender  wrote:

> Just want to make sure I understand what you're after here.  We should have
> a "ccache" directory or similar in the geode-native repo, where we build C
> bindings for the client, then we should compile them into a shared library
> containing all of the code, and export/make visible only the C interface?
> So the geode native installer will contain shared libraries representing 3
> copies of all the code, with the static library in the build tree making a
> total of 4?
>
> I'm starting to be concerned with the overall inefficiency of this plan.
>  Is this the best we can do?
>
> Thanks,
>
> Blake
>
>
> On Tue, Mar 31, 2020 at 12:05 PM Jacob Barrett 
> wrote:
>
> > Given that the C-binding will be tightly coupled with the C++ layer and
> > written in C++ I don’t think it make sense to have its own repository. In
> > order for the C-binding need to access internal, non-exported methods,
> for
> > things like serialization it will need to be statically linked to the C++
> > library. If we use separate repositories and builds then we need to
> build a
> > static library for the dependency in the C-binding library. This seems
> > overly complicated to me for little to no gain. Can you please elaborate
> on
> > the the pros of having a separate repository for the C-binding?
> >
> > For sure agree on the .net core repo. Tough depending on the mixed mode
> > binary support we still might run into the dependency issues on static
> > library.
> >
> > -Jake
> >
> >
> > > On Mar 31, 2020, at 11:23 AM, Matthew Reddington <
> mredding...@pivotal.io>
> > wrote:
> > >
> > > I would like to request the addition of two new repositories under
> > Apache in order to implement this RFC and to take advantage of it in
> > practice. That would be Apache/geode-c-client and
> > Apache/geode-dot-net-core-client.
> > >
> > >> On Mar 30, 2020, at 3:33 PM, Jacob Barrett 
> wrote:
> > >>
> > >>
> >
> https://github.com/pivotal-jbarrett/geode-native/tree/ee34cfbb5bddb55f5f890bb013c75d7780a787ae/ccache
> > <
> >
> https://github.com/pivotal-jbarrett/geode-native/tree/ee34cfbb5bddb55f5f890bb013c75d7780a787ae/ccache
> > >
> > >>
> > >> Quick stab at a POC for thread exception handling.
> > >>
> > >> -Jake
> > >>
> > >
> >
> >
>

Re: RFC: Add C Bindings to Geode Native Client

[Matthew Reddington]

A separate repo is our interpretation of the comments generated by this RFC. 
It’s easier to combine repositories later than it is to take them apart. The 
take away of this library is the ABI; that it will presently link against the 
C++ library is an implementation detail. We consider these separate projects 
and that they should be organized as such.

> On Mar 31, 2020, at 12:25 PM, Blake Bender  wrote:
> 
> Just want to make sure I understand what you're after here.  We should have
> a "ccache" directory or similar in the geode-native repo, where we build C
> bindings for the client, then we should compile them into a shared library
> containing all of the code, and export/make visible only the C interface?
> So the geode native installer will contain shared libraries representing 3
> copies of all the code, with the static library in the build tree making a
> total of 4?
> 
> I'm starting to be concerned with the overall inefficiency of this plan.
> Is this the best we can do?
> 
> Thanks,
> 
> Blake
> 
> 
> On Tue, Mar 31, 2020 at 12:05 PM Jacob Barrett  wrote:
> 
>> Given that the C-binding will be tightly coupled with the C++ layer and
>> written in C++ I don’t think it make sense to have its own repository. In
>> order for the C-binding need to access internal, non-exported methods, for
>> things like serialization it will need to be statically linked to the C++
>> library. If we use separate repositories and builds then we need to build a
>> static library for the dependency in the C-binding library. This seems
>> overly complicated to me for little to no gain. Can you please elaborate on
>> the the pros of having a separate repository for the C-binding?
>> 
>> For sure agree on the .net core repo. Tough depending on the mixed mode
>> binary support we still might run into the dependency issues on static
>> library.
>> 
>> -Jake
>> 
>> 
>>> On Mar 31, 2020, at 11:23 AM, Matthew Reddington 
>> wrote:
>>> 
>>> I would like to request the addition of two new repositories under
>> Apache in order to implement this RFC and to take advantage of it in
>> practice. That would be Apache/geode-c-client and
>> Apache/geode-dot-net-core-client.
>>> 
 On Mar 30, 2020, at 3:33 PM, Jacob Barrett  wrote:
 
 
>> https://github.com/pivotal-jbarrett/geode-native/tree/ee34cfbb5bddb55f5f890bb013c75d7780a787ae/ccache
>> <
>> https://github.com/pivotal-jbarrett/geode-native/tree/ee34cfbb5bddb55f5f890bb013c75d7780a787ae/ccache
>>> 
 
 Quick stab at a POC for thread exception handling.
 
 -Jake
 
>>> 
>> 
>> 

Re: RFC: Add C Bindings to Geode Native Client

[Jacob Barrett]

> On Mar 31, 2020, at 12:25 PM, Blake Bender  wrote:
> 
> Just want to make sure I understand what you're after here.  We should have
> a "ccache" directory or similar in the geode-native repo, where we build C
> bindings for the client, then we should compile them into a shared library
> containing all of the code, and export/make visible only the C interface?

Correct.

> So the geode native installer will contain shared libraries representing 3
> copies of all the code, with the static library in the build tree making a
> total of 4?

Let me see if I am counting the same things you are:
Out of the existing geode-native repo:
1 - C++ dynamic library (only C++ symbols exported)
2 - C dynamic library (only C symbols export)
3 - .NET Framework mixed mode assembly (deprecated, only .NET symbols exported)

Out of the new geode-net repo: (keeping it short, dropping the dot)
4 - .NET Core mixed mode assembly
or
4 - .NET Core assembly with dependency on C dynamic library

This also supposes that we have one installer for all this, maybe we have 
multiple installers, maybe we select what we want to install, maybe it doesn’t 
matter because in the end Geode doesn’t even produce an installer.

> I'm starting to be concerned with the overall inefficiency of this plan.
> Is this the best we can do?

Can you describe the inefficiency you are thinking about. 

I see having the C dynamic library statically linked the C++ bits as a much 
smaller footprint to whatever language it is eventually bound to. All the C++ 
templated symbols that are unused by the C binding layer should be left our by 
the linker. I won’t have to copy two dynamic libraries with my C-binding based 
application. Think about .NET core, if we can’t do mixed mode assemblies then 
it must be dynamically dependent on the C dynamic library, if the C library was 
dynamic to the C++ then we have to have that library also in the path. Recall 
how hard it is for some to manage having .NET and SSLimpl in the same path, now 
it's worse. If the inefficiency is disk space, that’s cheeper than support time 
on getting library paths correct.

-Jake

Re: RFC: Add C Bindings to Geode Native Client

[Jacob Barrett]

> On Mar 31, 2020, at 1:48 PM, Matthew Reddington  
> wrote:
> 
> A separate repo is our interpretation of the comments generated by this RFC.

Can you please quote specific statements that you interpreted to suggest 
separate repositories. I would like to understand where this interpretation 
comes from. 

> We consider these separate projects and that they should be organized as such.

Who is we? Does the we reflect the owners of the comments you are referencing 
above?

In the end Geode is a single project in the eyes of ASF.

-Jake

Re: RFC: Add C Bindings to Geode Native Client

[Blake Bender]

We in this instance means the native client team.  As far as specific
comments, I'm going to suggest we not go down that road, because this feels
a little more adversarial to me than it needs to be already.  Suffice to
say that from my own perspective, in both what you wrote and what I got
from our in-person conversation on Monday, your answer to the general
question "Should the C bindings be part of the native client?" appeared to
be no, thus a separate repository seemed a perfectly reasonable
assumption.  I had hoped to do this in the geode-native repo to begin with,
so your assent to this makes life easier.

As far as my concerns about inefficiency, what I meant is essentially yes
we have multiple copies of the same code in the release, and this always
makes me a little uneasy.  I've seen a lot of compatibility bugs in my
career having to do with different products having different versions of
the same code, so my natural inclination is to avoid it when possible.
Having both C++ and C-style functions exported from the same library
doesn't give me any heartburn at all, so simply compiling the C bindings
into the existing shared library just means one less copy of the code in
the installation.  I fear I am in the minority in this opinion, however,
and it's not something I'm really doctrinaire about, so I'll defer.

So are we good here?  I'd like to get the RFC wrapped up and move on to
building this.

Thanks,

Blake


On Tue, Mar 31, 2020 at 2:03 PM Jacob Barrett  wrote:

>
>
> > On Mar 31, 2020, at 1:48 PM, Matthew Reddington 
> wrote:
> >
> > A separate repo is our interpretation of the comments generated by this
> RFC.
>
> Can you please quote specific statements that you interpreted to suggest
> separate repositories. I would like to understand where this interpretation
> comes from.
>
> > We consider these separate projects and that they should be organized as
> such.
>
> Who is we? Does the we reflect the owners of the comments you are
> referencing above?
>
> In the end Geode is a single project in the eyes of ASF.
>
> -Jake
>
>

Re: RFC: Add C Bindings to Geode Native Client

[Jacob Barrett]

> On Mar 31, 2020, at 3:06 PM, Blake Bender  wrote:
> 
> We in this instance means the native client team.  As far as specific
> comments, I'm going to suggest we not go down that road, because this feels
> a little more adversarial to me than it needs to be already.

Sorry it feels adversarial. From below I think there is a misunderstanding of 
my preferences.

>  Suffice to
> say that from my own perspective, in both what you wrote and what I got
> from our in-person conversation on Monday, your answer to the general
> question "Should the C bindings be part of the native client?" appeared to
> be no, thus a separate repository seemed a perfectly reasonable
> assumption.  I had hoped to do this in the geode-native repo to begin with,
> so your assent to this makes life easier.

This may be the point of confusion because I have never intended to give the 
impression that the C-bindings should be separate from the geode-native repo. 
My examples even integrate it with the geode-native project. I do believe it 
should be separate sources and separate includes. I would not want to be doing 
a C++ project and have C functions clouding my IDE or  vice versa. Perhaps that 
is where the confusion comes from. 

> As far as my concerns about inefficiency, what I meant is essentially yes
> we have multiple copies of the same code in the release, and this always
> makes me a little uneasy.  I've seen a lot of compatibility bugs in my
> career having to do with different products having different versions of
> the same code, so my natural inclination is to avoid it when possible.
> Having both C++ and C-style functions exported from the same library
> doesn't give me any heartburn at all, so simply compiling the C bindings
> into the existing shared library just means one less copy of the code in
> the installation.  I fear I am in the minority in this opinion, however,
> and it's not something I'm really doctrinaire about, so I'll defer.
I would really like to understand your concerns but I don’t understand how 
combining the symbols into a single file resolves the versioning issue? Can 
your help me understand what the different produces with different versions 
means and how it would apply to this case?

If the C and C++ symbols are both exported from the same shared library would 
you have a common include directory as well or would you spit the includes? I 
could live with a combine library but not a combined include headers.

> So are we good here?  I'd like to get the RFC wrapped up and move on to
> building this.

Do you feel there is a consensus? I feel like there is a lot left that isn’t 
either in the original RFC, hasn’t been discussed here or is still a sticking 
point. You could update the RFC with what we have discussed and see if 
consensus is reached.

Sticking points:
* Single or split shared libraries
* Single or split includes
* Single or split source repository

Undefined:
* Exception handling (I gave one example but didn’t get feedback or consensus)
* Namespacing/Prefixing
* Pattern and naming conventions for new, delete and class methods.
* Handling of (de)serialization hand off or callbacks into non-C code

Agreed:
* Strong types with opaque struct pointers
* No complete structs in the API/ABI

-Jake

Re: [DISCUSS] Replace UDP messaging for membership with TCP

[Anthony Baker]

Echo’ing my comment here:

When we move from a reliable UDP implementation to one based on TCP, we need to 
think about how to provide reliability on top of TCP.  If you dig into TCP, 
you’ll find that it tries really hard (sometimes up to 15 minutes!!) but 
doesn’t guarantee message delivery.  Does this matter in practice?  Yes it 
does--I’ve worked on geode issues where a faulty network cable eventually 
caused a cluster hang because of a dropped TCP packet.

One technique for dealing with this is to pair requests and responses.  If a 
response is not received within a timeout, close the socket and assume any 
incomplete requests must be resent.


While DTLS looks interesting, it does impose some constraints on packets that 
could lead to poor performance.  I think overall we’ll be better suited to 
focus on TCP for secure transport.

Anthony



> On Mar 31, 2020, at 10:17 AM, Dan Smith  wrote:
> 
> Hi all,
> 
> We created a RFC for replacing our UDP messaging  in Geode with a TCP based
> solution. This will address the issues we have supporting our current udp
> encryption solution, along with helping us move away from jgroups, which
> currently can't be upgraded.
> 
> Please review and comment by 4/7/2020.
> 
> https://cwiki.apache.org/confluence/display/GEODE/Replace+UDP+messaging+for+membership+with+TCP
> 
> Thanks!
> -Dan