Broken: apache/geode#6352 (develop - 804ecfa)

[Travis CI]

Build Update for apache/geode
-

Build: #6352
Status: Broken

Duration: 20 minutes and 2 seconds
Commit: 804ecfa (develop)
Author: jinmeiliao
Message: GEODE-4625: name collision check when create region through gfsh 
(#1483)

View the changeset: 
https://github.com/apache/geode/compare/ce0d6a0f5a1c...804ecfabef7a

View the full build log and details: 
https://travis-ci.org/apache/geode/builds/345047542?utm_source=email&utm_medium=notification

--

You can configure recipients for build notifications in your .travis.yml file. 
See https://docs.travis-ci.com/user/notifications

Broken: jinmeiliao/geode#236 (createRegion - ae94531)

[Travis CI]

Build Update for jinmeiliao/geode
-

Build: #236
Status: Broken

Duration: 16 minutes and 56 seconds
Commit: ae94531 (createRegion)
Author: Jinmei Liao
Message: add security test

View the changeset: 
https://github.com/jinmeiliao/geode/compare/04079fe98af7...ae9453177177

View the full build log and details: 
https://travis-ci.org/jinmeiliao/geode/builds/345047352?utm_source=email&utm_medium=notification

--

You can configure recipients for build notifications in your .travis.yml file. 
See https://docs.travis-ci.com/user/notifications

Geode unit tests completed in 'develop/DistributedTest' with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/DistributedTest/builds/159

[SECURITY] CVE-2017-15692 Apache Geode unsafe deserialization in TcpServer

[Anthony Baker]

CVE-2017-15692 Apache Geode unsafe deserialization in TcpServer

Severity:  Important

Vendor: The Apache Software Foundation

Versions Affected:  Apache Geode 1.0.0 through 1.3.0

Description:
The TcpServer within the Geode locator opens a network port that
deserializes data.  If an unprivileged user gains access to the Geode
locator, they may be able to cause remote code execution if certain
classes are present on the classpath.

A malicious user can send a network message to the Geode locator and
execute code if certain classes are present on the classpath.

Mitigation:
Users of the affected versions should upgrade to Apache Geode 1.4.0 or
later.  In addition, users should set the flag
validate-serializable-objects.

Credit:
This issue was reported responsibly to the Apache Geode Security Team
by Man Yue Mo from Semmle.

References:
[1] https://issues.apache.org/jira/browse/GEODE-3923
[2] 
https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-SecurityVulnerabilities

[SECURITY] CVE-2017-15693 Apache Geode unsafe deserialization of application objects

[Anthony Baker]

CVE-2017-15693 Apache Geode unsafe deserialization of application objects

Severity:  Important

Vendor: The Apache Software Foundation

Versions Affected:  Apache Geode 1.0.0 through 1.3.0

Description:
The Geode server stores application objects in serialized form.
Certain cluster operations and API invocations cause these objects to
be deserialized.  An user with DATA:WRITE access to the cluster may be
able to cause remote code execution if certain classes are present on
the classpath.

Mitigation:
Users of the affected versions should upgrade to Apache Geode 1.4.0 or
later.  In addition, users should set the flags
validate-serializable-objects and serializable-object-filter.

Credit:
This issue was reported responsibly to the Apache Geode Security Team
by Man Yue Mo from Semmle.

References:
[1] https://issues.apache.org/jira/browse/GEODE-3923
[2] 
https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-SecurityVulnerabilities

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #837 was SUCCESSFUL (with 2331 tests)

[Spring CI]

---
Spring Data GemFire > Nightly-ApacheGeode > #837 was successful.
---
Scheduled
2333 tests in total.

https://build.spring.io/browse/SGF-NAG-837/





--
This message is automatically generated by Atlassian Bamboo

Re: [Proposal] Thread monitoring mechanism

[Barry Oglesby]

A lot of the Geode thread pools are defined in ClusterDistributionManager.
Most of these use custom ThreadPoolExecutors like:

SerialQueuedExecutorWithDMStats
PooledExecutorWithDMStats
FunctionExecutionPooledExecutor

These classes all extend ThreadPoolExecutor and override beforeExecute and
afterExecute. These methods are currently used by helper classes to update
the stats before and after a thread executes. Potentially these same
methods could be used to add and remove a thread from a monitor. For
example, there could be a FunctionExecutionThreadMonitor that is created as
part of the FunctionExecutionPooledExecutor whose job it would be to
monitor FunctionExecution threads. The beforeExecute method would add the
thread to the monitor; the afterExecute would remove the thread from the
monitor.

I would be mindful about the performance impact of adding these monitors,
though.


Thanks,
Barry Oglesby


On Wed, Feb 21, 2018 at 11:41 AM, Gregory Vortman <
gregory.vort...@amdocs.com> wrote:

> That's the point exactly to have a single very thin and generic mechanism
> to cover all threads/threads pool. Nothing is specific in this solution.
> Regards
>
>
> -Original Message-
> From: Jason Huynh [jhu...@pivotal.io]
> Received: Wednesday, 21 Feb 2018, 20:54
> To: dev@geode.apache.org [dev@geode.apache.org]
> CC: u...@geode.apache.org [u...@geode.apache.org]
> Subject: Re: [Proposal] Thread monitoring mechanism
>
> I am assuming this would be for all thread/thread pools and not specific
> to Function threads.  I wonder what the impact would be for put/get
> operations or are we going to target specific operations.
>
>
>
> On Tue, Feb 20, 2018 at 1:04 AM Gregory Vortman <
> gregory.vort...@amdocs.com> wrote:
> Hello team,
> One of the most severe issues hitting our real time application is thread
> stuck for multiple reasons, such as long lasting locks, deadlocks, threads
> which wait for reply forever in case of packet drop issue etc...
> Such kind of stuck are under Radar of the existing system health check
> methods.
> In mission critical applications, this will be resulted as an immediate
> outage.
>
> As a short we are implementing kind of internal watch dog mechanism for
> stuck detector:
>There is a registration object
>Function executor having start/end hooks to
> register/unregister the thread via the registration object
> Customized Monitoring scheduled thread is spawned on startup. The thread
> to wake up every N seconds, to scan the registration map and to detect
> unregistered threads for a long time (configurable).
> Once such threads has been detected, process stack is taken and thread
> stack statistic metric is provided.
>
> This helps us to monitor, detect and take fast decision about the action
> which should be taken - usually it is member bounce decision (consistency
> issue is possible, in our case it is better than deny of service).
> The above solution is not touching GEODE core code, but implemented in
> boundaries of customized code only.
>
> I would like to raise a proposal to introduce a long term generic thread
> monitoring mechanism, to detect threads which are stuck for any reason.
> To maintain a monitoring object having a start/end methods to be invoked
> similarly to FunctionStats.startFunctionExecution and FunctionStats.
> endFunctionExecution.
>
> Your feedback would be appreciated
>
> Thank you for cooperation.
> Best regards!
>
> Gregory Vortman
>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>

Geode unit tests completed in 'develop/FlakyTest' with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/FlakyTest/builds/254

Re: [Proposal] Thread monitoring mechanism

[Anilkumar Gingade]

Good idea, we need better tools (echo systems) to manage/monitor Geode
resources.
In Geode many times the work is handed to other low-level threads
(messaging) or new threads/runnables; it will be nice to have some
mechanism to associate main work-thread to low level thread; that will give
better indication on who is waiting on whom.

-Anil




On Fri, Feb 23, 2018 at 3:08 PM, Barry Oglesby  wrote:

> A lot of the Geode thread pools are defined in ClusterDistributionManager.
> Most of these use custom ThreadPoolExecutors like:
>
> SerialQueuedExecutorWithDMStats
> PooledExecutorWithDMStats
> FunctionExecutionPooledExecutor
>
> These classes all extend ThreadPoolExecutor and override beforeExecute and
> afterExecute. These methods are currently used by helper classes to update
> the stats before and after a thread executes. Potentially these same
> methods could be used to add and remove a thread from a monitor. For
> example, there could be a FunctionExecutionThreadMonitor that is created as
> part of the FunctionExecutionPooledExecutor whose job it would be to
> monitor FunctionExecution threads. The beforeExecute method would add the
> thread to the monitor; the afterExecute would remove the thread from the
> monitor.
>
> I would be mindful about the performance impact of adding these monitors,
> though.
>
>
> Thanks,
> Barry Oglesby
>
>
> On Wed, Feb 21, 2018 at 11:41 AM, Gregory Vortman <
> gregory.vort...@amdocs.com> wrote:
>
> > That's the point exactly to have a single very thin and generic mechanism
> > to cover all threads/threads pool. Nothing is specific in this solution.
> > Regards
> >
> >
> > -Original Message-
> > From: Jason Huynh [jhu...@pivotal.io]
> > Received: Wednesday, 21 Feb 2018, 20:54
> > To: dev@geode.apache.org [dev@geode.apache.org]
> > CC: u...@geode.apache.org [u...@geode.apache.org]
> > Subject: Re: [Proposal] Thread monitoring mechanism
> >
> > I am assuming this would be for all thread/thread pools and not specific
> > to Function threads.  I wonder what the impact would be for put/get
> > operations or are we going to target specific operations.
> >
> >
> >
> > On Tue, Feb 20, 2018 at 1:04 AM Gregory Vortman <
> > gregory.vort...@amdocs.com> wrote:
> > Hello team,
> > One of the most severe issues hitting our real time application is thread
> > stuck for multiple reasons, such as long lasting locks, deadlocks,
> threads
> > which wait for reply forever in case of packet drop issue etc...
> > Such kind of stuck are under Radar of the existing system health check
> > methods.
> > In mission critical applications, this will be resulted as an immediate
> > outage.
> >
> > As a short we are implementing kind of internal watch dog mechanism for
> > stuck detector:
> >There is a registration object
> >Function executor having start/end hooks to
> > register/unregister the thread via the registration object
> > Customized Monitoring scheduled thread is spawned on startup. The thread
> > to wake up every N seconds, to scan the registration map and to detect
> > unregistered threads for a long time (configurable).
> > Once such threads has been detected, process stack is taken and thread
> > stack statistic metric is provided.
> >
> > This helps us to monitor, detect and take fast decision about the action
> > which should be taken - usually it is member bounce decision (consistency
> > issue is possible, in our case it is better than deny of service).
> > The above solution is not touching GEODE core code, but implemented in
> > boundaries of customized code only.
> >
> > I would like to raise a proposal to introduce a long term generic thread
> > monitoring mechanism, to detect threads which are stuck for any reason.
> > To maintain a monitoring object having a start/end methods to be invoked
> > similarly to FunctionStats.startFunctionExecution and FunctionStats.
> > endFunctionExecution.
> >
> > Your feedback would be appreciated
> >
> > Thank you for cooperation.
> > Best regards!
> >
> > Gregory Vortman
> >
> > This message and the information contained herein is proprietary and
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer <
> > https://www.amdocs.com/about/email-disclaimer>
> > This message and the information contained herein is proprietary and
> > confidential and subject to the Amdocs policy statement,
> >
> > you may review at https://www.amdocs.com/about/email-disclaimer <
> > https://www.amdocs.com/about/email-disclaimer>
> >
>

Geode unit tests completed in 'develop/DistributedTest' with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/DistributedTest/builds/162