RE: [PROPOSAL]: Include GEODE-7832, GEODE-7853 & GEODE-7863 in Geode 1.12.0

[Alberto Bustamante Reyes]

+1

De: Donal Evans 
Enviado: jueves, 19 de marzo de 2020 2:14
Para: dev@geode.apache.org 
Asunto: Re: [PROPOSAL]: Include GEODE-7832, GEODE-7853 & GEODE-7863 in Geode 
1.12.0

+1

On Wed, Mar 18, 2020 at 4:53 PM Owen Nichols  wrote:

> +3
>
> > On Mar 18, 2020, at 4:52 PM, Ju@N  wrote:
> >
> > Hello devs,
> >
> > I'd like to propose including the fixes for *GEODE-7832 [1]*, *GEODE-7853
> > [2]* and *GEODE-7863 [3]* in release 1.12.0.
> > All the changes are related to the work we have been doing in order to
> > bring the performance closer to the baseline (*Geode 1.10*), we are not
> > quite there yet but it would be good to include these fixes into the
> > release anyways.
> > Best regards.
> >
> > [1]: https://issues.apache.org/jira/browse/GEODE-7832
> > [2]: https://issues.apache.org/jira/browse/GEODE-7853
> > [3]: https://issues.apache.org/jira/browse/GEODE-7863
> >
> > --
> > Ju@N
> > --
> > Ju@N
>
>

Re: [DISCUSS] Client side configuration for a SNI proxy

[Bruce Schuchardt]

+1

On 3/18/20, 4:13 PM, "Dan Smith"  wrote:

One addendum to this proposal:

This proposed method of configuring a SNI proxy server requires that the
client sends the SNI hostname as part of the TLS handshake. In the previous
proposal we suggested it would only be sent if a SNI Proxy is configured.
Now we propose that a geode client will *always* send the SNI hostname when
it initiates a TLS connection to a locator or server.

This will help us implement the SniSocketFactory. It may also help with
other use cases such as presenting different SSL certificates for different
hostnames, which is the original reason this SNI hostname is part of TLS.
The SSLParameterExtension will still take precedence, so I think this
should not break the original use case for SSLParameterExtension [1]

-Dan

[1]

https://lists.apache.org/thread.html/0e08a5d0a480115ace6412e16ad45023adf0c29406ec2095a4d19d30%40%3Cdev.geode.apache.org%3E

On Mon, Mar 16, 2020 at 3:33 PM Dan Smith  wrote:

> Hi all,
>
> A new RFC is up for this feature
> 
https://cwiki.apache.org/confluence/display/GEODE/Client+side+configuration+for+a+SNI+proxy.
>
>
> Please review and comment by this Friday, 3/20/2020.
>
> This hopefully addresses some of the concerns with the previous RFC for
> this feature. The new proposal is for a more general SocketFactory 
property
> that users can implement, along the lines of what Jake and Owen suggested.
>
> -Dan
>

Re: [DISCUSS] Client side configuration for a SNI proxy

[Joris Melchior]

+1

On Mon, Mar 16, 2020 at 6:33 PM Dan Smith  wrote:

> Hi all,
>
> A new RFC is up for this feature
>
> https://cwiki.apache.org/confluence/display/GEODE/Client+side+configuration+for+a+SNI+proxy
> .
>
>
> Please review and comment by this Friday, 3/20/2020.
>
> This hopefully addresses some of the concerns with the previous RFC for
> this feature. The new proposal is for a more general SocketFactory property
> that users can implement, along the lines of what Jake and Owen suggested.
>
> -Dan
>


-- 
*Joris Melchior *
CF Engineering
Pivotal Toronto
416 877 5427

“Programs must be written for people to read, and only incidentally for
machines to execute.” – *Hal Abelson*

Re: WAN replication issue in cloud native environments

[Bruce Schuchardt]

I wonder if an approach similar to the SNI hostname PoolFactory changes would 
work for this non-TLS gateway.  The client needs to differentiate between the 
different servers so that it doesn’t declare all of them dead should one of 
them fail.  If the pool knew about the gateway it could direct all traffic 
there and the servers wouldn’t need to set a hostname-for-clients.

 

It’s not an ideal solution since the gateway wouldn’t know which server the 
client wanted to contact and there are sure to be other problems like creating 
a backup queue for subscriptions.  But that’s the case with the 
hostname-for-clients approach, too.

 

 

From: Alberto Bustamante Reyes 
Date: Wednesday, March 18, 2020 at 8:35 AM
To: Dan Smith , "dev@geode.apache.org" 
Cc: Bruce Schuchardt , Jacob Barrett 
, Anilkumar Gingade , Charlie Black 

Subject: RE: WAN replication issue in cloud native environments

 

Hi all,

 

As Bruce suggested me, I have created a wiki page describing the problem we are 
trying to solve: 
https://cwiki.apache.org/confluence/display/GEODE/Allow+same+host+and+port+for+all+gateway+receivers

 

Please let me know if further clarifications are needed.

 

Also, I have closed the PR I have been using until now, and created a new one 
with the current status of the solution, with one commit per issue described in 
the wiki: https://github.com/apache/geode/pull/4824

 

Thanks in advance!

De: Alberto Bustamante Reyes 
Enviado: lunes, 9 de marzo de 2020 11:24
Para: Dan Smith 
Cc: dev@geode.apache.org ; Bruce Schuchardt 
; Jacob Barrett ; Anilkumar 
Gingade ; Charlie Black 
Asunto: RE: WAN replication issue in cloud native environments 

 

Thanks for point that out Dan. Sorry for the misunderstanding, as I only found 
that "affinity" (setServerAffinityLocation method) on the client code I thought 
you were talking about it.
Anyway, I did some more tests and it does not solve our problem...

I tried configuring the service affinity on k8s, but it breaks the first part 
of the solution (the changes implemented on LocatorLoadSnapshot that solves the 
problem of the replication) and senders do not connect to other receivers when 
the one they were connected to is down.

The only alternative we have in mind to try to solve the ping problem is to 
keep on investigating if changing the ping task creation could be a solution 
(the changes implemented are clearly breaking something, so the solution is not 
complete yet).







De: Dan Smith 
Enviado: jueves, 5 de marzo de 2020 21:03
Para: Alberto Bustamante Reyes 
Cc: dev@geode.apache.org ; Bruce Schuchardt 
; Jacob Barrett ; Anilkumar 
Gingade ; Charlie Black 
Asunto: Re: WAN replication issue in cloud native environments

I think there is some confusion here.

The client side class ExecutablePool has a method called 
setServerAffinityLocation. It looks like that is used for some internal 
transaction code to make sure transactions go to the same server. I don't think 
it makes any sense for the gateway to be messing with this setting.

What I was talking about was session affinity in your proxy server. For 
example, if you are using k8s, session affinity as defined in this page - 
https://kubernetes.io/docs/concepts/services-networking/service/

"If you want to make sure that connections from a particular client are passed 
to the same Pod each time, you can select the session affinity based on the 
client’s IP addresses by setting service.spec.sessionAffinity to “ClientIP” 
(the default is “None”)"

I think setting session affinity might help your use case, because it sounds 
like you are having issues with the proxy directing pings to a different server 
than the data.

-Dan

On Thu, Mar 5, 2020 at 4:20 AM Alberto Bustamante Reyes 
 wrote:
I think that was what I did when I tried, but I realized I had a failure in the 
code. Now that I have tried again, reverting the change of executing ping by 
endpoint, and applying the server affinity, the connections are much more 
stable! Looks promising 🙂

I suppose that if I want to introduce this change, setting the server affinity 
in the gateway sender should be introduced as a new option in the sender 
configuration, right?

De: Dan Smith mailto:dsm...@pivotal.io>>
Enviado: jueves, 5 de marzo de 2020 4:41
Para: Alberto Bustamante Reyes 
Cc: dev@geode.apache.org 
mailto:dev@geode.apache.org>>; Bruce Schuchardt 
mailto:bschucha...@pivotal.io>>; Jacob Barrett 
mailto:jbarr...@pivotal.io>>; Anilkumar Gingade 
mailto:aging...@pivotal.io>>; Charlie Black 
mailto:cbl...@pivotal.io>>
Asunto: Re: WAN replication issue in cloud native environments

Oh, sorry, I meant server affinity with the proxy itself. So that it will 
always route traffic from the same gateway sender to the same gateway receiver. 
Hopefully that would ensure that pings go to the same receiver data is sent to.

-Dan

On Wed, Mar 4, 2020, 1:31 AM Alberto Busta

Re: Tips on using AsyncInvocation in DUnit Tests

[Kirk Lund]

Sure thing!

package org.apache.geode.examples;

import static org.apache.geode.test.dunit.VM.getVM;
import static org.assertj.core.api.Assertions.assertThat;

import java.io.Serializable;
import java.util.concurrent.Future;

import org.junit.Rule;
import org.junit.Test;

import org.apache.geode.test.dunit.AsyncInvocation;
import org.apache.geode.test.dunit.rules.DistributedRule;

public class ExampleDistributedTest implements Serializable {

  @Rule
  public DistributedRule distributedRule = new DistributedRule(1);

  /**
   * Awaits completion of AsyncInvocation with Void type.
   */
  @Test
  public void awaitCompletionOfAsyncInvocation() throws Exception {
AsyncInvocation asyncActionInVM0 = getVM(0).invokeAsync(() ->
doAsyncAction());

// do other actions in parallel

asyncActionInVM0.await();
  }

  /**
   * Awaits completion of AsyncInvocation and then returns its result.
   */
  @Test
  public void getResultOfAsyncInvocation() throws Exception {
AsyncInvocation asyncActionInVM0 =
getVM(0).invokeAsync(() -> doAsyncActionWithReturnValue());

// do other actions in parallel

Serializable result = asyncActionInVM0.get();
assertThat(result).isNotNull();
  }

  /**
   * Awaits completion of AsyncInvocation as a Future with Void type.
   */
  @Test
  public void awaitCompletionOfFuture() throws Exception {
Future asyncActionInVM0 = getVM(0).invokeAsync(() ->
doAsyncAction());

// do other actions in parallel

asyncActionInVM0.get();
  }

  /**
   * Awaits completion of AsyncInvocation as a Future and then returns its
result.
   */
  @Test
  public void getResultOfFuture() throws Exception {
Future asyncActionInVM0 =
getVM(0).invokeAsync(() -> doAsyncActionWithReturnValue());

// do other actions in parallel

Serializable result = asyncActionInVM0.get();
assertThat(result).isNotNull();
  }

  private void doAsyncAction() {
// do some work that will need to execute asynchronously
  }

  private Serializable doAsyncActionWithReturnValue() {
// do some work that will need to execute asynchronously
return "Result";
  }
}

On Wed, Mar 18, 2020 at 3:17 PM Anilkumar Gingade 
wrote:

> Thanks Kirk. Can you add an example here...
>
> On Wed, Mar 18, 2020 at 11:12 AM Kirk Lund  wrote:
>
> > Tips on using AsyncInvocation:
> >
> > * Always use await() or get()
> > * Both check and throw any remote exceptions
> > * Both use GeodeAwaitility Timeout and will throw TimeoutException if
> it’s
> > exceeded
> > * Use await() for Void types and get() when expecting a non-null value
> >
> > Recent improvements:
> >
> > Timeout now gets a remote stack trace to use as the cause and dumps stack
> > traces for that JVM’s threads.
> >
> > You can also declare your instance of AsyncInvocation as a Future and
> > simply use the standard Java API for Futures. This basically means the
> test
> > will invoke get() for both Void and non-Void types.
> >
> > AsyncInvocation handles everything for you when you invoke await() or
> get()
> > -- there is no need to invoke any of the deprecated APIs on
> > AsyncInvocation:
> > * Both use the GeodeAwaitility Timeout and throw TImeoutException
> > * If Timeout occurs, AsyncInvocation will use the remote stack trace of
> the
> > stuck thread as the cause and it will also print all threads stacks for
> > that DUnit VM to facilitate debugging
> > * Both will check for a remote failure and rethrow it
> >
>

Re: [DISCUSS] Client side configuration for a SNI proxy

[Jason Huynh]

+1

On Thu, Mar 19, 2020 at 7:27 AM Joris Melchior  wrote:

> +1
>
> On Mon, Mar 16, 2020 at 6:33 PM Dan Smith  wrote:
>
> > Hi all,
> >
> > A new RFC is up for this feature
> >
> >
> https://cwiki.apache.org/confluence/display/GEODE/Client+side+configuration+for+a+SNI+proxy
> > .
> >
> >
> > Please review and comment by this Friday, 3/20/2020.
> >
> > This hopefully addresses some of the concerns with the previous RFC for
> > this feature. The new proposal is for a more general SocketFactory
> property
> > that users can implement, along the lines of what Jake and Owen
> suggested.
> >
> > -Dan
> >
>
>
> --
> *Joris Melchior *
> CF Engineering
> Pivotal Toronto
> 416 877 5427
>
> “Programs must be written for people to read, and only incidentally for
> machines to execute.” – *Hal Abelson*
> 
>

Re: [PROPOSAL]: Include GEODE-7832, GEODE-7853 & GEODE-7863 in Geode 1.12.0

[Anilkumar Gingade]

+1 The changes and the risk looks minimal.

On Thu, Mar 19, 2020 at 2:16 AM Alberto Bustamante Reyes
 wrote:

> +1
> 
> De: Donal Evans 
> Enviado: jueves, 19 de marzo de 2020 2:14
> Para: dev@geode.apache.org 
> Asunto: Re: [PROPOSAL]: Include GEODE-7832, GEODE-7853 & GEODE-7863 in
> Geode 1.12.0
>
> +1
>
> On Wed, Mar 18, 2020 at 4:53 PM Owen Nichols  wrote:
>
> > +3
> >
> > > On Mar 18, 2020, at 4:52 PM, Ju@N  wrote:
> > >
> > > Hello devs,
> > >
> > > I'd like to propose including the fixes for *GEODE-7832 [1]*,
> *GEODE-7853
> > > [2]* and *GEODE-7863 [3]* in release 1.12.0.
> > > All the changes are related to the work we have been doing in order to
> > > bring the performance closer to the baseline (*Geode 1.10*), we are not
> > > quite there yet but it would be good to include these fixes into the
> > > release anyways.
> > > Best regards.
> > >
> > > [1]: https://issues.apache.org/jira/browse/GEODE-7832
> > > [2]: https://issues.apache.org/jira/browse/GEODE-7853
> > > [3]: https://issues.apache.org/jira/browse/GEODE-7863
> > >
> > > --
> > > Ju@N
> > > --
> > > Ju@N
> >
> >
>

Re: 2020 Apache Geode Summit - Call for Papers!

[Anthony Baker]

Thanks for sharing Bob!  I highly encourage you to submit a talk and/or plan on 
attending.  We always see lots of users, expert practitioners, and project 
committers at this event.

Related, the CFP for ApacheCon NA is also open [1].  Submit your talk there as 
well!


Thanks,
Anthony

[1] https://www.apachecon.com/acna2020/cfp.html 




> On Mar 19, 2020, at 3:12 PM, Bob Glithero  wrote:
> 
> Do you have a great Apache Geode story to share?  Call for papers for the 
> 2020 Apache Geode Summit is now open!
>  
> Speaking at Geode Summit is a great way to connect with hundreds of your 
> peers, become more involved in the Geode community, and have a public 
> platform for you to share your story of the future of the premier in-memory 
> data platform.
>  
> Call for Papers
>  
> Submit your talks here: 
>  
> https://docs.google.com/spreadsheets/d/1fHzogly2D0sU6DQVHJsY4tBi8NDRcPAYn4vCtbjVJKg/edit?usp=sharing
>  
> 
>  
> Target talk length is 40 min with Q&A.  For 2020, we are also considering 
> 5-minute TED talk style presentations that we can aggregate into one session 
> (hit reply if you’re interested).   We have a soft deadline of May 1 for 
> submissions, because talks will begin being publicized in early June.  If 
> you’re accepted, we’ll reach out for bios, headshots, etc.
>  
> If you’d like your talk to be considered as a SpringOne breakout, please tag 
> it “Spring” in the spreadsheet link above.
>  
> We are looking for your most interesting, most informative, most advanced, 
> and most generally applicable talks on Apache Geode and the tools, 
> technologies, and techniques in the Geode ecosystem. Winning submissions will 
> make it clear that you understand how to take your knowledge and experience 
> and transfer it to Geode Summit attendees. For example:
>  
> A deep dive into Geode features 
> Patterns you have developed while using Geode
> Development techniques, libraries, and debugging techniques that have helped 
> you build event-driven systems
> An experience report about the Geode-based system you’ve built, including 
> takeaways that the audience can put into practice right away
> Beginner-level introductions to Geode
> Specific explanations of how Geode enabled you to build a solution unique to 
> your industry, and applicable to others with similar use cases
> Examples of Geode with streaming, event-processing, asynchronous 
> applications, or Kubernetes are especially welcome
> We especially encourage members of underrepresented groups to submit talks!
>  
> We’re looking for unique perspectives and thought-provoking discussions that 
> will inform, challenge, and enlighten our attendees. So do your best to stand 
> out by submitting relevant, engaging, creative, and entertaining proposals.  
> The SpringOne Platform site (spring.io ) has other ideas 
> for developing standout content.
>  
> Note:  due to COVID-19 outbreak, we may have to convert to a virtual 
> conference.  We will continue to monitor and update.  As you plan your talk, 
> please consider how it would work in a virtual format.
>  
> Cheers,
> The Geode Summit Team