Re: Requests taking too long if one member of the cluster fails

[Mario Salazar de Torres]

Hi everyone,

Regarding what @Jacob Barrett mentioned about the 
geode-native timeout handling, yes, I am aware of that, we are working on 
identifying any problems to open PRs.
But my feeling is that this PR https://github.com/apache/geode-native/pull/695 
will dramatically improve things there 🙂

Regarding parametrization, we've been testing several parametrizations and 
things look really promising, there are some minor things to tweak, but we 
barely notice the impact.
Regarding allowing a to configure Geode as a non primary/secondary (a.k.a 
multi-master) distributed system, thin is I've been reading, just out of 
curiosity, and it turns out, it is feasible. I.E: Google 
Spanner
Different thing is that it is something that can be easily implemented in 
Geode, or even maybe that's not something we've want. Still, I think that's a 
conversation for another forum.

So really thanks everyone that helped 🙂
BR,
Mario.

From: Anthony Baker 
Sent: Monday, November 23, 2020 6:25 PM
To: dev@geode.apache.org 
Cc: miguel.g.gar...@ericsson.com 
Subject: Re: Requests taking too long if one member of the cluster fails

Yes, lowering the member timeout is one approach I’ve seen taken for 
applications that demand ultra low latency.  These workloads need to provide 
not just low “average” or even p99 latency, but put a hard limit on the max 
value.

When you do this you need to ensure coherency across at all aspects of timeouts 
(eg client read timeouts and retries).  You need to ensure that GC pauses don’t 
cause instability in the cluster.  For example, if a GC pause is greater than 
the member timeout, you should go back and re-tune your heap settings to drive 
down GC.  If you are running in a container of VM you need to ensure sufficient 
resources so that the GemFIre process is never paused.

All this presupposes a stable and performant network infrastructure.

Anthony


On Nov 21, 2020, at 1:40 PM, Mario Salazar de Torres 
mailto:mario.salazar.de.tor...@est.tech>> 
wrote:

So, what I've tried here is to set a really low member-timeout, which results 
the server holding the secondary copy becoming the primary owner in around 
<600ms. That's quite a huge improvement,
but I wanted to ask you if setting this member-timeout too low might carry 
unforeseen consequences.

Re: Committing broken javadocs to develop

[Mark Hanson]

Can we make lack of comments an error?

On 11/20/20, 4:47 PM, "Jacob Barrett"  wrote:

I think we can make javadoc warnings an error in Gradle. I have fixed 
several modules to treat java warnings as errors already. Perhaps we can just 
extend that effort.

> On Nov 20, 2020, at 10:39 AM, Kirk Lund  wrote:
> 
> Please remember to:
> * fix all broken javadocs before submitting a PR
> * always review javadocs when reviewing PRs
> 
> Thanks,
> Kirk

Re: Geode - store and query JSON documents

[Anilkumar Gingade]

Ankit,

Here is how to query col2.
"SELECT d.col2 FROM /JsonRegion v, v.data d, d.col2 c where c.k21 = '22'";

You can find example on how to query nested collections:
https://geode.apache.org/docs/guide/18/getting_started/querying_quick_reference.html

When you want to select a nested collection and inspect its value; you need to 
create iterator in the from clause (E.g.  d.col2 in the above query)

You can find other ways to query arrays in the above sample.

-Anil.



On 11/23/20, 10:02 PM, "ankit Soni"  wrote:

Hi Anil,

Thanks a lot for your reply. This really helps to proceed. The query shared
by you worked but I need a slight variation of it, i.e where clause
contains col2 (data.col2.k21 = '22') which is array unlike col1
(object).

FYI: value is stored in cache.
PDX[28847624, __GEMFIRE_JSON]{
data=[PDX[28847624, __GEMFIRE_JSON] {
col1=PDX[28626794, __GEMFIRE_JSON] {k11=aaa, k12=true, k13=,
k14=2020-12-31T00..}
Col2=[PDX[25385544, __GEMFIRE_JSON]{k21=, k22=true}]}]}
Based on OQL querying doc shared, tried few ways but no luck on querying
based on Col2.

It will be really helpful if you share updated query.

Thanks
Ankit.

On Tue, Nov 24, 2020, 2:42 AM Anilkumar Gingade  wrote:

> Ankit,
>
> Here is how you can query your JSON object.
>
> String queryStr = "SELECT d.col1 FROM /JsonRegion v, v.data d where
> d.col1.k11 = 'aaa'";
>
> As replied earlier; the data is stored as PdxInstance type in the cache.
> In the PdxInstance, the data is stored as top level or nested collection 
of
> objects/values based on input JSON object structure.
> The query engine queries on the PdxInstance type and returns the value.
>
> To see, how the PdxInstance data looks like in the cache, you can print
> the returned value from querying the region values:
> E.g.:
>  String queryStr = "SELECT v FROM /JsonRegion v";
>  SelectResults results = (SelectResults)
> QueryService().newQuery(queryStr).execute();
>   Object[] value = results.asList().toArray();
>   System.out.println(" Projected value: " + value[0]);
>
> You can find sample queries on different type of objects (collections,
> etc) at:
>
> 
https://geode.apache.org/docs/guide/18/getting_started/querying_quick_reference.html
>
> Also in order to determine where the time is getting spent, can you
> separate out object creation through JSONFormatter from put operation.
> E.g.:
> PdxInstance pdxInstance = JSONFormatter.fromJSON(jsonDoc_2);
> // Time taken to format:
> region.put("1", pdxInstance);
> // Time taken to add to cache:
>
> And measure the time separately. It will help to see if the time is spent
> in getting the PdxInstance or in doing puts. Also, can you measure the 
time
> in avg.
> E.g. Say time measured for puts from 1000 to 2000 and avg time for those
> puts.
>
> -Anil.
>
>
> On 11/23/20, 11:27 AM, "ankit Soni"  wrote:
>
>  Hello geode-dev,
>
> I am *evaluating usage of Geode (1.12) with storing JSON documents and
> querying the same*. I am able to store the json records successfully 
in
> geode but seeking guidance on how to query them.
> More details on code and sample json is,
>
>
> *Sample client-code*
>
> import org.apache.geode.cache.client.ClientCache;
> import org.apache.geode.cache.client.ClientCacheFactory;
> import org.apache.geode.cache.client.ClientRegionShortcut;
> import org.apache.geode.pdx.JSONFormatter;
> import org.apache.geode.pdx.PdxInstance;
>
> public class MyTest {
>
> *//NOTE: Below is truncated json, single json document can max
> contain an array of col1...col30 (30 diff attributes) within data. *
> public final static  String jsonDoc_2 = "{" +
> "\"data\":[{" +
> "\"col1\": {" +
> "\"k11\": \"aaa\"," +
> "\"k12\":true," +
> "\"k13\": ," +
> "\"k14\": \"2020-12-31:00:00:00\"" +
> "}," +
> "\"col2\":[{" +
> "\"k21\": \"22\"," +
> "\"k22\": true" +
> "}]" +
> "}]" +
> "}";
>
> * //NOTE: Col1col30 are mix of JSONObject ({}) and JSONArray
> ([]) as shown above in jsonDoc_2;*
>
> public static void main(String[] args){
>
> //create client-cache
> ClientCache cache = new