Re: privacy protection

[Anthony Baker]

If you want to encrypt values at the client, you might want to check out this 
talk:
https://springoneplatform.io/2018/sessions/implementing-pii-encryption-with-pdx-serialization

Anthony


> On Jan 21, 2020, at 10:45 AM, Michael Oleske  wrote:
> 
> Something to consider is what scenarios do you want to protect against.
> Full disk encryption protects against a drive pull attack, such as an
> attack walks into a data storage room full disks, pulls a disk from the
> room, and then makes a run for it.  Since the full disk is encrypted, the
> attacker will have to break that.  Encrypting values protects against
> insiders from seeing values, such as a rogue administrator that is trying
> to get personally identifiable information (they would be able to decrypt
> the disk, but wouldn't be able to decrypt the values).  I'm personally not
> sure what Geode needs to do or could be doing versus what's best practice
> when building applications that use Geode/when running Geode.
> 
> -michael
> 
> On Tue, Jan 21, 2020 at 10:33 AM Dan Smith  wrote:
> 
>> Hi Mario,
>> 
>> It's something we've talked about before, I think it would be a good idea.
>> The two workarounds I have seen done are
>> - full disk encryption - storing the geode disk files on an encrypted disk
>> - encrypting values  - At the client side, converting values into encrypted
>> bytes before storing them in geode.
>> 
>> But it would nice to just support encryption at the disk store level.
>> 
>> -Dan
>> 
>> On Tue, Jan 21, 2020 at 4:21 AM Udo Kohlmeyer  wrote:
>> 
>>> Hi there Mario,
>>> 
>>> The idea of encryption-at-rest, is something that has been on my radar
>>> for at least 4yrs now.
>>> 
>>> I would not mind having a chat about what your requirements are and how
>>> we can get this into Geode.
>>> 
>>> Looking forward to chatting to you about this.
>>> 
>>> --Udo
>>> 
>>> On 1/15/20 2:20 AM, Mario Kevo wrote:
 Hi geode-dev,
 
 Is it possible somehow to protect all files that containing user
>> data(or
>>> user data itself) being stored in disk for Geode.
 This includes all persistence data(OpLogs), backups and possible other
>>> files containing user data.
 Also protection is needed for all of the files potentailly used for
>>> replication and cluster high availability mechanism.
 
 If this feature is not available, do you have it in the plan already?
 Do you included it in Geode roadmap?
 
 BR,
 Mario
 
 
>>> 
>> 

Re: disable statistic archival

[Kirk Lund]

Try setting STATISTIC_SAMPLING_ENABLED to false to disable statistic
sampling.

I think we should delete "An empty string (default) disables statistic
archival." from the javadocs for STATISTIC_ARCHIVE_FILE to avoid confusion
and redundancy with STATISTIC_SAMPLING_ENABLED.

See below for the javadocs on both properties.

  /**
   * The static String definition of the "statistic-archive-file"
property 
   * 
   * Description: The file that statistic samples are written to. An
empty string (default)
   * disables statistic archival.
   * 
   * Default: ""
   */
  String STATISTIC_ARCHIVE_FILE = "statistic-archive-file";

  /**
   * The static String definition of the
"statistic-sampling-enabled" property 
   * 
   * Description: "true" causes the statistics to be sampled
periodically and operating
   * system statistics to be fetched each time a sample is taken. "false"
disables sampling which
   * also disables operating system statistic collection. Non OS statistics
will still be recorded
   * in memory and can be viewed by administration tools. However, charts
will show no activity and
   * no statistics will be archived while sampling is disabled. Starting in
7.0 the default value
   * has been changed to true. If statistic sampling is disabled it will
also cause various metrics
   * seen in gfsh and pulse to always be zero.
   * 
   * Default: "true"
   * 
   * Allowed values: true|false
   */
  String STATISTIC_SAMPLING_ENABLED = "statistic-sampling-enabled";

On Tue, Jan 21, 2020 at 1:06 AM Mario Kevo  wrote:

> Hi,
>
> We are trying to disable archiving statistic in the file by providing
> empty string to --statistic-archive-file. This option doesn't work.
> From the documentation it should work:
> The file to which the running system member writes statistic samples. For
> example: “StatisticsArchiveFile.gfs”. An empty string disables archiving.
> I opened ticket(GEODE-7714<
> https://issues.apache.org/jira/browse/GEODE-7714>) and try to fix it, but
> without success.
>
> As alter runtime command update properties and cache, it checks if any of
> these parameters change, but if we set this property to an empty string it
> failed with message
> Please provide a relevant parameter(s).
> We can change for this parameter that it can be an empty string but how
> this command works, it goes over all parameters and checks if it is
> changed. In that case if we provide something like
> alter runtime --member=server it will be successiful but shouldn't as we
> didn't provide any parameter.
>
> So the proposal is that we need to add a new parameter called
> --statistic-archiving-enabled  which can be true or false. In case it is
> true we need to provide also --statistic-archive-file.
>
> Any thougths?
>
> BR,
> Mario
>
>

Re: disable statistic archival

[Dave Barnes]

I'm getting the impression that the user guide could be clearer with regard
to the interactions between

   - enable-statistics
   - statistic-sampling-enabled
   - statistic-archive-file



On Wed, Jan 22, 2020 at 9:30 AM Kirk Lund  wrote:

> Try setting STATISTIC_SAMPLING_ENABLED to false to disable statistic
> sampling.
>
> I think we should delete "An empty string (default) disables statistic
> archival." from the javadocs for STATISTIC_ARCHIVE_FILE to avoid confusion
> and redundancy with STATISTIC_SAMPLING_ENABLED.
>
> See below for the javadocs on both properties.
>
>   /**
>* The static String definition of the "statistic-archive-file"
> property * name="statistic-archive-file"/a>
>* 
>* Description: The file that statistic samples are written to. An
> empty string (default)
>* disables statistic archival.
>* 
>* Default: ""
>*/
>   String STATISTIC_ARCHIVE_FILE = "statistic-archive-file";
>
>   /**
>* The static String definition of the
> "statistic-sampling-enabled" property * name="statistic-sampling-enabled"/a>
>* 
>* Description: "true" causes the statistics to be sampled
> periodically and operating
>* system statistics to be fetched each time a sample is taken. "false"
> disables sampling which
>* also disables operating system statistic collection. Non OS statistics
> will still be recorded
>* in memory and can be viewed by administration tools. However, charts
> will show no activity and
>* no statistics will be archived while sampling is disabled. Starting in
> 7.0 the default value
>* has been changed to true. If statistic sampling is disabled it will
> also cause various metrics
>* seen in gfsh and pulse to always be zero.
>* 
>* Default: "true"
>* 
>* Allowed values: true|false
>*/
>   String STATISTIC_SAMPLING_ENABLED = "statistic-sampling-enabled";
>
> On Tue, Jan 21, 2020 at 1:06 AM Mario Kevo  wrote:
>
> > Hi,
> >
> > We are trying to disable archiving statistic in the file by providing
> > empty string to --statistic-archive-file. This option doesn't work.
> > From the documentation it should work:
> > The file to which the running system member writes statistic samples. For
> > example: “StatisticsArchiveFile.gfs”. An empty string disables archiving.
> > I opened ticket(GEODE-7714<
> > https://issues.apache.org/jira/browse/GEODE-7714>) and try to fix it,
> but
> > without success.
> >
> > As alter runtime command update properties and cache, it checks if any of
> > these parameters change, but if we set this property to an empty string
> it
> > failed with message
> > Please provide a relevant parameter(s).
> > We can change for this parameter that it can be an empty string but how
> > this command works, it goes over all parameters and checks if it is
> > changed. In that case if we provide something like
> > alter runtime --member=server it will be successiful but shouldn't as we
> > didn't provide any parameter.
> >
> > So the proposal is that we need to add a new parameter called
> > --statistic-archiving-enabled  which can be true or false. In case it is
> > true we need to provide also --statistic-archive-file.
> >
> > Any thougths?
> >
> > BR,
> > Mario
> >
> >
>

RE: WAN replication issue in cloud native environments

[Alberto Bustamante Reyes]

Thanks Naba & Jacob for your comments!



@Naba: I have been implementing a solution as you suggested, and I think it 
would be convenient if the client knows the memberId of the server it is 
connected to.

(current code is here: https://github.com/apache/geode/pull/4616 )

For example, in:

LocatorLoadSnapshot::getReplacementServerForConnection(ServerLocation 
currentServer, String group, Set excludedServers)

In this method, client has sent the ServerLocation , but if that object does 
not contain the memberId, I dont see how to guarantee that the replacement that 
will be returned is not the same server the client is currently connected.
Inside that method, this other method is called:

LocatorLoadSnapshot::isCurrentServerMostLoaded(currentServer, groupServers)

where groupServers is a "Map" object. If 
the keys of that map have the same host and port, they are only different on 
the memberId. But as you dont know it (you just have currentServer which 
contains host and port), you cannot get the correct LoadHolder value, so you 
cannot know if your server is the most loaded.

So I think client needs to know the memberId of the server.

@Jacob: I think the solution finally implies that client have to know the 
memberId, I think we could simplify the maps.


BR/

Alberto B.



De: Jacob Barrett 
Enviado: miércoles, 22 de enero de 2020 7:29
Para: dev@geode.apache.org 
Cc: Anilkumar Gingade ; Charlie Black 
Asunto: Re: WAN replication issue in cloud native environments


> On Jan 21, 2020, at 1:24 PM, Nabarun Nag  wrote:
>
> Suggestion:
> - Instead, can we create a new class that contains the memberID and
> ServerLocation and that new class object is added as a key in the
> connectionMap.

I poked around a bit in this code and the ServerLocation is also in the 
LoadHolder class so we can simplify this even more by just using the member ID 
as the key in all these maps. When we need the ServerLocation we can get that 
from the LoadHolder.

The addServer call comes from a caller that has the CacheServerProfile, which 
has the member ID. The updateLoad caller is a DistributedMessage which has a 
sender member that is the member ID. Lastly, removeServer caller has a 
CacheServerProfile as well we can again get the member ID.

-Jake

Re: WAN replication issue in cloud native environments

[Jacob Barrett]

> On Jan 22, 2020, at 9:51 AM, Alberto Bustamante Reyes 
>  wrote:
> 
> Thanks Naba & Jacob for your comments!
> 
> 
> 
> @Naba: I have been implementing a solution as you suggested, and I think it 
> would be convenient if the client knows the memberId of the server it is 
> connected to.
> 
> (current code is here: https://github.com/apache/geode/pull/4616 
>  )
> 
> For example, in:
> 
> LocatorLoadSnapshot::getReplacementServerForConnection(ServerLocation 
> currentServer, String group, Set excludedServers)
> 
> In this method, client has sent the ServerLocation , but if that object does 
> not contain the memberId, I dont see how to guarantee that the replacement 
> that will be returned is not the same server the client is currently 
> connected.
> Inside that method, this other method is called:


Given that your setup is masquerading multiple members behind the same host and 
port (ServerLocation) it doesn’t matter. When the pool opens a new socket to 
the replacement server it will be to the shared hostname and port and the 
Kubenetes service at that host and port will just pick a backend host. In the 
solution we suggested we preserved that behavior since the k8s service can’t 
determine which backend member to route the connection to based on the member 
id.


> LocatorLoadSnapshot::isCurrentServerMostLoaded(currentServer, groupServers)
> 
> where groupServers is a "Map" object. 
> If the keys of that map have the same host and port, they are only different 
> on the memberId. But as you dont know it (you just have currentServer which 
> contains host and port), you cannot get the correct LoadHolder value, so you 
> cannot know if your server is the most loaded.

Again, given your use case the behavior of this method is lost when a new 
connection is establish by the pool through the shared hostname anyway. 

> @Jacob: I think the solution finally implies that client have to know the 
> memberId, I think we could simplify the maps.

The client isn’t keeping these load maps, the locator is, and the locator knows 
all the member ids. The client end only needs to know the host/port 
combination. In your example where the wan replication (a client to the remote 
cluster) connects to the shared host/port service and get randomly routed to 
one of the backend servers in that service.

All of this locator balancing code is unnecessarily in this model where 
something else is choosing the final destination. The goal of our proposed 
changes was to recognize that all we need is to make sure the locator keeps the 
shared ServerLocation alive in its responses to clients by tracking the members 
associated and reducing that set to the set of unit ServerLocations. In your 
case that will always reduce to 1 ServerLocation for N number of members, as 
long as 1 member is still up.

-Jake

[DISCUSS] Cut 1.12.0 branch on February 3rd, 2020

[Ernest Burghardt]

Hello Geode,

I'll be your release manager for Geode 1.12 and the plan is to cut the
branch on February 3rd, 2020 (we'll avoid Groundhog Day and LeapDay as
well...)

As a reminder, only critical fixes will be allowed once the release branch
is created, so it is helpful to have develop as stable as possible before
branching.

Please discuss any concerns regarding this plan/date.

Thanks,
Ernie

Old geode-benchmark PRs

[Dan Smith]

Hi,

I noticed we have some old outstanding PRs for the geode-benchmarks
project. Are any of these things we want to merge or should we close them
out?

https://github.com/apache/geode-benchmarks/pulls

-Dan

Re: disable statistic archival

[Kirk Lund]

Dave, In your list, I think "enable-statistics" should be
"enable-time-statistics". But, yes you're right!

On Wed, Jan 22, 2020 at 9:46 AM Dave Barnes  wrote:

> I'm getting the impression that the user guide could be clearer with regard
> to the interactions between
>
>- enable-statistics
>- statistic-sampling-enabled
>- statistic-archive-file
>
>
>
> On Wed, Jan 22, 2020 at 9:30 AM Kirk Lund  wrote:
>
> > Try setting STATISTIC_SAMPLING_ENABLED to false to disable statistic
> > sampling.
> >
> > I think we should delete "An empty string (default) disables statistic
> > archival." from the javadocs for STATISTIC_ARCHIVE_FILE to avoid
> confusion
> > and redundancy with STATISTIC_SAMPLING_ENABLED.
> >
> > See below for the javadocs on both properties.
> >
> >   /**
> >* The static String definition of the "statistic-archive-file"
> > property  >* name="statistic-archive-file"/a>
> >* 
> >* Description: The file that statistic samples are written to.
> An
> > empty string (default)
> >* disables statistic archival.
> >* 
> >* Default: ""
> >*/
> >   String STATISTIC_ARCHIVE_FILE = "statistic-archive-file";
> >
> >   /**
> >* The static String definition of the
> > "statistic-sampling-enabled" property  >* name="statistic-sampling-enabled"/a>
> >* 
> >* Description: "true" causes the statistics to be sampled
> > periodically and operating
> >* system statistics to be fetched each time a sample is taken. "false"
> > disables sampling which
> >* also disables operating system statistic collection. Non OS
> statistics
> > will still be recorded
> >* in memory and can be viewed by administration tools. However, charts
> > will show no activity and
> >* no statistics will be archived while sampling is disabled. Starting
> in
> > 7.0 the default value
> >* has been changed to true. If statistic sampling is disabled it will
> > also cause various metrics
> >* seen in gfsh and pulse to always be zero.
> >* 
> >* Default: "true"
> >* 
> >* Allowed values: true|false
> >*/
> >   String STATISTIC_SAMPLING_ENABLED = "statistic-sampling-enabled";
> >
> > On Tue, Jan 21, 2020 at 1:06 AM Mario Kevo  wrote:
> >
> > > Hi,
> > >
> > > We are trying to disable archiving statistic in the file by providing
> > > empty string to --statistic-archive-file. This option doesn't work.
> > > From the documentation it should work:
> > > The file to which the running system member writes statistic samples.
> For
> > > example: “StatisticsArchiveFile.gfs”. An empty string disables
> archiving.
> > > I opened ticket(GEODE-7714<
> > > https://issues.apache.org/jira/browse/GEODE-7714>) and try to fix it,
> > but
> > > without success.
> > >
> > > As alter runtime command update properties and cache, it checks if any
> of
> > > these parameters change, but if we set this property to an empty string
> > it
> > > failed with message
> > > Please provide a relevant parameter(s).
> > > We can change for this parameter that it can be an empty string but how
> > > this command works, it goes over all parameters and checks if it is
> > > changed. In that case if we provide something like
> > > alter runtime --member=server it will be successiful but shouldn't as
> we
> > > didn't provide any parameter.
> > >
> > > So the proposal is that we need to add a new parameter called
> > > --statistic-archiving-enabled  which can be true or false. In case it
> is
> > > true we need to provide also --statistic-archive-file.
> > >
> > > Any thougths?
> > >
> > > BR,
> > > Mario
> > >
> > >
> >
>

Re: Old geode-benchmark PRs

[Donal Evans]

Two of those PRs are mine, so perhaps I can give a bit of context for
people who might look at them. The oldest of the two, "Feature/Add PdxType
benchmark and additional framework flexibility" was an attempt to quantify
and maintain the improvement in performance for PdxType creation when large
numbers of PdxTypes already exist, and to allow the passing of additional
system properties to the VMs hosting the servers in order to change the log
level and prevent the benchmark measuring how long it takes to log PdxType
creation rather than actual time taken to create new PdxTypes. This PR has
been open for a very long time, so it's possible that the changes regarding
passing additional system properties to the VMs are now outdated or
unnecessary, but the actual benchmarks themselves still have some value.

The second PR, "Added benchmarks for aggregate functions" contains 16 new
benchmarks related to aggregate OQL queries, (8 each for Partitioned and
Replicated regions), which were added following work in that area by the
Commons team. The build is currently marked as failing, but this is due to
a timeout rather than an actual build failure, as the number of benchmarks
added increased the total time to build beyond the currently configured
timeout. Adding such a large number of additional benchmarks will probably
also noticeably increase the time it takes benchmarks to run, which bears
consideration.

I hope this helps shed some light for people who may look over those PRs.

On Wed, Jan 22, 2020 at 11:36 AM Dan Smith  wrote:

> Hi,
>
> I noticed we have some old outstanding PRs for the geode-benchmarks
> project. Are any of these things we want to merge or should we close them
> out?
>
> https://github.com/apache/geode-benchmarks/pulls
>
> -Dan
>

[Polite Request] Clean up branches on Geode

[Michael Oleske]

There's a lot of branches on Geode that look like they can be deleted
because they have either been merged or closed.  Below is a list.  Please
consider deleting your branch if you have created one of the below.

Thanks!
-michael

The following branches on Geode have been merged

feature/GEODE-7212 Updated 2 months ago by karensmolermiller - #4392 -
Merged
feature/GEODE-7477 Updated 2 months ago by Eric Shu - #4352 - Merged
revert-4204-feature/GEODE-7157 Updated 3 months ago by mhansonp - #4283 -
Merged
revert-4201-GEODE-7326-add-cache-gets-timer Updated 3 months ago by
mhansonp - #4222 - Merged
revert-4189-revert-4086-feature/GEODE-6807 Updated 3 months ago by mivanac
- #4212 - Merged
revert-4086-feature/GEODE-6807 Updated 3 months ago by echobravopapa -
#4189 - Merged
revert-4046-feature/GEODE-7179 Updated 4 months ago by nabarunnag - #4054 -
Merged
revert-4034-feature/GEODE-7128 Updated 4 months ago by nabarunnag - #4052 -
Merged
feature/GEODE-7130 Updated 5 months ago by pivotal-eshu - #3973 - Merged
feature/GEODE-6812 Updated 8 months ago by pivotal-eshu - #3682 - Merged
feature/GEODE-6802 Updated 8 months ago by pivotal-eshu - #3623 - Merged

The following branches on Geode have been closed
feature/GEODE-7576 Updated last month by Eric Shu - #4477 - Closed
feature/GEODE-7277 Updated 3 months ago by gesterzhou - #4149 - Closed
better-classpath Updated 14 months ago by metatype - #2904 - Closed

Re: Old geode-benchmark PRs

[Jacob Barrett]

I hate to just close what’s there but it also likely needs some massaging. 
Additions to the benchmarks risk destabilizing the CI if the benchmark has a 
wide variance. I don’t have time right now to spend on tuning new benchmarks. 
If someone else does please step up and take it on.


> On Jan 22, 2020, at 12:31 PM, Donal Evans  wrote:
> 
> Two of those PRs are mine, so perhaps I can give a bit of context for
> people who might look at them. The oldest of the two, "Feature/Add PdxType
> benchmark and additional framework flexibility" was an attempt to quantify
> and maintain the improvement in performance for PdxType creation when large
> numbers of PdxTypes already exist, and to allow the passing of additional
> system properties to the VMs hosting the servers in order to change the log
> level and prevent the benchmark measuring how long it takes to log PdxType
> creation rather than actual time taken to create new PdxTypes. This PR has
> been open for a very long time, so it's possible that the changes regarding
> passing additional system properties to the VMs are now outdated or
> unnecessary, but the actual benchmarks themselves still have some value.
> 
> The second PR, "Added benchmarks for aggregate functions" contains 16 new
> benchmarks related to aggregate OQL queries, (8 each for Partitioned and
> Replicated regions), which were added following work in that area by the
> Commons team. The build is currently marked as failing, but this is due to
> a timeout rather than an actual build failure, as the number of benchmarks
> added increased the total time to build beyond the currently configured
> timeout. Adding such a large number of additional benchmarks will probably
> also noticeably increase the time it takes benchmarks to run, which bears
> consideration.
> 
> I hope this helps shed some light for people who may look over those PRs.
> 
> On Wed, Jan 22, 2020 at 11:36 AM Dan Smith  wrote:
> 
>> Hi,
>> 
>> I noticed we have some old outstanding PRs for the geode-benchmarks
>> project. Are any of these things we want to merge or should we close them
>> out?
>> 
>> https://github.com/apache/geode-benchmarks/pulls
>> 
>> -Dan
>> 

Re: [Polite Request] Clean up branches on Geode

[Jacob Barrett]

Also, please use your fork for branches!

> On Jan 22, 2020, at 3:14 PM, Michael Oleske  wrote:
> 
> There's a lot of branches on Geode that look like they can be deleted
> because they have either been merged or closed.  Below is a list.  Please
> consider deleting your branch if you have created one of the below.
> 
> Thanks!
> -michael
> 
> The following branches on Geode have been merged
> 
> feature/GEODE-7212 Updated 2 months ago by karensmolermiller - #4392 -
> Merged
> feature/GEODE-7477 Updated 2 months ago by Eric Shu - #4352 - Merged
> revert-4204-feature/GEODE-7157 Updated 3 months ago by mhansonp - #4283 -
> Merged
> revert-4201-GEODE-7326-add-cache-gets-timer Updated 3 months ago by
> mhansonp - #4222 - Merged
> revert-4189-revert-4086-feature/GEODE-6807 Updated 3 months ago by mivanac
> - #4212 - Merged
> revert-4086-feature/GEODE-6807 Updated 3 months ago by echobravopapa -
> #4189 - Merged
> revert-4046-feature/GEODE-7179 Updated 4 months ago by nabarunnag - #4054 -
> Merged
> revert-4034-feature/GEODE-7128 Updated 4 months ago by nabarunnag - #4052 -
> Merged
> feature/GEODE-7130 Updated 5 months ago by pivotal-eshu - #3973 - Merged
> feature/GEODE-6812 Updated 8 months ago by pivotal-eshu - #3682 - Merged
> feature/GEODE-6802 Updated 8 months ago by pivotal-eshu - #3623 - Merged
> 
> The following branches on Geode have been closed
> feature/GEODE-7576 Updated last month by Eric Shu - #4477 - Closed
> feature/GEODE-7277 Updated 3 months ago by gesterzhou - #4149 - Closed
> better-classpath Updated 14 months ago by metatype - #2904 - Closed

Re: [Polite Request] Clean up branches on Geode

[Owen Nichols]

GitHub does have a setting to auto-delete PR branches once merged, but I think 
it’s safe to assume this community prefers to [forget to] delete them manually 
;)


> On Jan 22, 2020, at 3:14 PM, Michael Oleske  wrote:
> 
> There's a lot of branches on Geode that look like they can be deleted
> because they have either been merged or closed.  Below is a list.  Please
> consider deleting your branch if you have created one of the below.
> 
> Thanks!
> -michael
> 
> The following branches on Geode have been merged
> 
> feature/GEODE-7212 Updated 2 months ago by karensmolermiller - #4392 -
> Merged
> feature/GEODE-7477 Updated 2 months ago by Eric Shu - #4352 - Merged
> revert-4204-feature/GEODE-7157 Updated 3 months ago by mhansonp - #4283 -
> Merged
> revert-4201-GEODE-7326-add-cache-gets-timer Updated 3 months ago by
> mhansonp - #4222 - Merged
> revert-4189-revert-4086-feature/GEODE-6807 Updated 3 months ago by mivanac
> - #4212 - Merged
> revert-4086-feature/GEODE-6807 Updated 3 months ago by echobravopapa -
> #4189 - Merged
> revert-4046-feature/GEODE-7179 Updated 4 months ago by nabarunnag - #4054 -
> Merged
> revert-4034-feature/GEODE-7128 Updated 4 months ago by nabarunnag - #4052 -
> Merged
> feature/GEODE-7130 Updated 5 months ago by pivotal-eshu - #3973 - Merged
> feature/GEODE-6812 Updated 8 months ago by pivotal-eshu - #3682 - Merged
> feature/GEODE-6802 Updated 8 months ago by pivotal-eshu - #3623 - Merged
> 
> The following branches on Geode have been closed
> feature/GEODE-7576 Updated last month by Eric Shu - #4477 - Closed
> feature/GEODE-7277 Updated 3 months ago by gesterzhou - #4149 - Closed
> better-classpath Updated 14 months ago by metatype - #2904 - Closed

Question regarding a ConnectionPoolDUnitTest failure

[Mark Hanson]

Hi All, 

I could use a little help understanding the expected behavior for a test case.

We are registering for events and we are getting destroys for destroys, creates 
for creates etc. However, in the case of a recreate, it seems like we are 
getting an entry event of LOCAL_LOAD_CREATE and some INVALIDATE's as well.

I am not sure why that this the case and 
ConnectionPoolDUnitTest.test014InvalidateAndDestroyPropagation seems to have 
the expectation of only getting creates as well, which it mostly does. 
Sometimes though it gets these unexpected invalidates.

An example is this 
EntryEventImpl[op=INVALIDATE;region=/root/test014InvalidateAndDestroyPropagation;key=2;callbackArg=null;originRemote=true;originMember=612f3aa1e54b(:loner):53030:faeb21ce;callbacksInvoked;id=EventID[id=25
 bytes;threadID=2;sequenceID=556];isFromServer],

Note the callbackArg=null, this is odd, usually it is populated.

Any help someone could provide would be a great help. This seems like a bug.

Thanks,
Mark

Odg: disable statistic archival

[Mario Kevo]

@Kirk, in case we change --enable-statistic(this flag 
refer to STATISTIC_SAMPLING_ENABLED ) to false we lose some metrics in gfsh and 
pulse.
What if we want to keep statistic on, but not archive it to file.
I don't see now how to disable archiving statistics to file, but keep 
statistics in gfsh and pulse.

Yes, the user guide could be clearer.



Šalje: Kirk Lund 
Poslano: 22. siječnja 2020. 20:44
Prima: geode 
Predmet: Re: disable statistic archival

Dave, In your list, I think "enable-statistics" should be
"enable-time-statistics". But, yes you're right!

On Wed, Jan 22, 2020 at 9:46 AM Dave Barnes  wrote:

> I'm getting the impression that the user guide could be clearer with regard
> to the interactions between
>
>- enable-statistics
>- statistic-sampling-enabled
>- statistic-archive-file
>
>
>
> On Wed, Jan 22, 2020 at 9:30 AM Kirk Lund  wrote:
>
> > Try setting STATISTIC_SAMPLING_ENABLED to false to disable statistic
> > sampling.
> >
> > I think we should delete "An empty string (default) disables statistic
> > archival." from the javadocs for STATISTIC_ARCHIVE_FILE to avoid
> confusion
> > and redundancy with STATISTIC_SAMPLING_ENABLED.
> >
> > See below for the javadocs on both properties.
> >
> >   /**
> >* The static String definition of the "statistic-archive-file"
> > property  >* name="statistic-archive-file"/a>
> >* 
> >* Description: The file that statistic samples are written to.
> An
> > empty string (default)
> >* disables statistic archival.
> >* 
> >* Default: ""
> >*/
> >   String STATISTIC_ARCHIVE_FILE = "statistic-archive-file";
> >
> >   /**
> >* The static String definition of the
> > "statistic-sampling-enabled" property  >* name="statistic-sampling-enabled"/a>
> >* 
> >* Description: "true" causes the statistics to be sampled
> > periodically and operating
> >* system statistics to be fetched each time a sample is taken. "false"
> > disables sampling which
> >* also disables operating system statistic collection. Non OS
> statistics
> > will still be recorded
> >* in memory and can be viewed by administration tools. However, charts
> > will show no activity and
> >* no statistics will be archived while sampling is disabled. Starting
> in
> > 7.0 the default value
> >* has been changed to true. If statistic sampling is disabled it will
> > also cause various metrics
> >* seen in gfsh and pulse to always be zero.
> >* 
> >* Default: "true"
> >* 
> >* Allowed values: true|false
> >*/
> >   String STATISTIC_SAMPLING_ENABLED = "statistic-sampling-enabled";
> >
> > On Tue, Jan 21, 2020 at 1:06 AM Mario Kevo  wrote:
> >
> > > Hi,
> > >
> > > We are trying to disable archiving statistic in the file by providing
> > > empty string to --statistic-archive-file. This option doesn't work.
> > > From the documentation it should work:
> > > The file to which the running system member writes statistic samples.
> For
> > > example: “StatisticsArchiveFile.gfs”. An empty string disables
> archiving.
> > > I opened ticket(GEODE-7714<
> > > https://issues.apache.org/jira/browse/GEODE-7714>) and try to fix it,
> > but
> > > without success.
> > >
> > > As alter runtime command update properties and cache, it checks if any
> of
> > > these parameters change, but if we set this property to an empty string
> > it
> > > failed with message
> > > Please provide a relevant parameter(s).
> > > We can change for this parameter that it can be an empty string but how
> > > this command works, it goes over all parameters and checks if it is
> > > changed. In that case if we provide something like
> > > alter runtime --member=server it will be successiful but shouldn't as
> we
> > > didn't provide any parameter.
> > >
> > > So the proposal is that we need to add a new parameter called
> > > --statistic-archiving-enabled  which can be true or false. In case it
> is
> > > true we need to provide also --statistic-archive-file.
> > >
> > > Any thougths?
> > >
> > > BR,
> > > Mario
> > >
> > >
> >
>

Re: Odg: disable statistic archival

[Owen Nichols]

Would it work to archive them to file /dev/null

On Wed, Jan 22, 2020 at 11:36 PM Mario Kevo  wrote:

> @Kirk, in case we change --enable-statistic(this
> flag refer to STATISTIC_SAMPLING_ENABLED ) to false we lose some metrics in
> gfsh and pulse.
> What if we want to keep statistic on, but not archive it to file.
> I don't see now how to disable archiving statistics to file, but keep
> statistics in gfsh and pulse.
>
> Yes, the user guide could be clearer.
>
>
> 
> Šalje: Kirk Lund 
> Poslano: 22. siječnja 2020. 20:44
> Prima: geode 
> Predmet: Re: disable statistic archival
>
> Dave, In your list, I think "enable-statistics" should be
> "enable-time-statistics". But, yes you're right!
>
> On Wed, Jan 22, 2020 at 9:46 AM Dave Barnes  wrote:
>
> > I'm getting the impression that the user guide could be clearer with
> regard
> > to the interactions between
> >
> >- enable-statistics
> >- statistic-sampling-enabled
> >- statistic-archive-file
> >
> >
> >
> > On Wed, Jan 22, 2020 at 9:30 AM Kirk Lund  wrote:
> >
> > > Try setting STATISTIC_SAMPLING_ENABLED to false to disable statistic
> > > sampling.
> > >
> > > I think we should delete "An empty string (default) disables statistic
> > > archival." from the javadocs for STATISTIC_ARCHIVE_FILE to avoid
> > confusion
> > > and redundancy with STATISTIC_SAMPLING_ENABLED.
> > >
> > > See below for the javadocs on both properties.
> > >
> > >   /**
> > >* The static String definition of the
> "statistic-archive-file"
> > > property  > >* name="statistic-archive-file"/a>
> > >* 
> > >* Description: The file that statistic samples are written
> to.
> > An
> > > empty string (default)
> > >* disables statistic archival.
> > >* 
> > >* Default: ""
> > >*/
> > >   String STATISTIC_ARCHIVE_FILE = "statistic-archive-file";
> > >
> > >   /**
> > >* The static String definition of the
> > > "statistic-sampling-enabled" property  > >* name="statistic-sampling-enabled"/a>
> > >* 
> > >* Description: "true" causes the statistics to be sampled
> > > periodically and operating
> > >* system statistics to be fetched each time a sample is taken.
> "false"
> > > disables sampling which
> > >* also disables operating system statistic collection. Non OS
> > statistics
> > > will still be recorded
> > >* in memory and can be viewed by administration tools. However,
> charts
> > > will show no activity and
> > >* no statistics will be archived while sampling is disabled.
> Starting
> > in
> > > 7.0 the default value
> > >* has been changed to true. If statistic sampling is disabled it
> will
> > > also cause various metrics
> > >* seen in gfsh and pulse to always be zero.
> > >* 
> > >* Default: "true"
> > >* 
> > >* Allowed values: true|false
> > >*/
> > >   String STATISTIC_SAMPLING_ENABLED = "statistic-sampling-enabled";
> > >
> > > On Tue, Jan 21, 2020 at 1:06 AM Mario Kevo 
> wrote:
> > >
> > > > Hi,
> > > >
> > > > We are trying to disable archiving statistic in the file by providing
> > > > empty string to --statistic-archive-file. This option doesn't work.
> > > > From the documentation it should work:
> > > > The file to which the running system member writes statistic samples.
> > For
> > > > example: “StatisticsArchiveFile.gfs”. An empty string disables
> > archiving.
> > > > I opened ticket(GEODE-7714<
> > > > https://issues.apache.org/jira/browse/GEODE-7714>) and try to fix
> it,
> > > but
> > > > without success.
> > > >
> > > > As alter runtime command update properties and cache, it checks if
> any
> > of
> > > > these parameters change, but if we set this property to an empty
> string
> > > it
> > > > failed with message
> > > > Please provide a relevant parameter(s).
> > > > We can change for this parameter that it can be an empty string but
> how
> > > > this command works, it goes over all parameters and checks if it is
> > > > changed. In that case if we provide something like
> > > > alter runtime --member=server it will be successiful but shouldn't as
> > we
> > > > didn't provide any parameter.
> > > >
> > > > So the proposal is that we need to add a new parameter called
> > > > --statistic-archiving-enabled  which can be true or false. In case it
> > is
> > > > true we need to provide also --statistic-archive-file.
> > > >
> > > > Any thougths?
> > > >
> > > > BR,
> > > > Mario
> > > >
> > > >
> > >
> >
>