[jira] [Resolved] (GEODE-8213) C++ native client performance bottleneck in access to serialization registry
[ https://issues.apache.org/jira/browse/GEODE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto Gomez resolved GEODE-8213. -- Resolution: Fixed > C++ native client performance bottleneck in access to serialization registry > > > Key: GEODE-8213 > URL: https://issues.apache.org/jira/browse/GEODE-8213 > Project: Geode > Issue Type: Improvement > Components: native client >Reporter: Alberto Gomez >Assignee: Alberto Gomez >Priority: Major > Fix For: 1.14.0 > > > It's been observed that when the number of threads used in a Geode client > using PdxSerialization is greater than 8, there is an important drop in > performance. > Analysing the client process behavior with perf, it has been observed a very > high CPU consumption by a spinlock > (apache::geode::util::concurrent::spinlock_mutex::lock) used when accessing > the serialization registry . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (GEODE-8508) Update .lgtm.yml to latest (1.13) Geode release
Alberto Bustamante Reyes created GEODE-8508: --- Summary: Update .lgtm.yml to latest (1.13) Geode release Key: GEODE-8508 URL: https://issues.apache.org/jira/browse/GEODE-8508 Project: Geode Issue Type: Task Components: native client Reporter: Alberto Bustamante Reyes As done in GEODE-8018 for Geode 1.12, now that Geode 1.13 is out it is needed to update LGTM config file to use the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GEODE-8508) Update .lgtm.yml to latest (1.13) Geode release
[ https://issues.apache.org/jira/browse/GEODE-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated GEODE-8508: -- Labels: pull-request-available (was: ) > Update .lgtm.yml to latest (1.13) Geode release > --- > > Key: GEODE-8508 > URL: https://issues.apache.org/jira/browse/GEODE-8508 > Project: Geode > Issue Type: Task > Components: native client >Reporter: Alberto Bustamante Reyes >Priority: Major > Labels: pull-request-available > > As done in GEODE-8018 for Geode 1.12, now that Geode 1.13 is out it is > needed to update LGTM config file to use the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8508) Update .lgtm.yml to latest (1.13) Geode release
[ https://issues.apache.org/jira/browse/GEODE-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198366#comment-17198366 ] ASF GitHub Bot commented on GEODE-8508: --- alb3rtobr opened a new pull request #652: URL: https://github.com/apache/geode-native/pull/652 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Update .lgtm.yml to latest (1.13) Geode release > --- > > Key: GEODE-8508 > URL: https://issues.apache.org/jira/browse/GEODE-8508 > Project: Geode > Issue Type: Task > Components: native client >Reporter: Alberto Bustamante Reyes >Priority: Major > > As done in GEODE-8018 for Geode 1.12, now that Geode 1.13 is out it is > needed to update LGTM config file to use the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (GEODE-8508) Update .lgtm.yml to latest (1.13) Geode release
[ https://issues.apache.org/jira/browse/GEODE-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto Bustamante Reyes reassigned GEODE-8508: --- Assignee: Alberto Bustamante Reyes > Update .lgtm.yml to latest (1.13) Geode release > --- > > Key: GEODE-8508 > URL: https://issues.apache.org/jira/browse/GEODE-8508 > Project: Geode > Issue Type: Task > Components: native client >Reporter: Alberto Bustamante Reyes >Assignee: Alberto Bustamante Reyes >Priority: Major > Labels: pull-request-available > > As done in GEODE-8018 for Geode 1.12, now that Geode 1.13 is out it is > needed to update LGTM config file to use the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GEODE-8018) Update .lgtm.yml to latest (1.12) Geode release
[ https://issues.apache.org/jira/browse/GEODE-8018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alberto Bustamante Reyes resolved GEODE-8018. - Resolution: Fixed > Update .lgtm.yml to latest (1.12) Geode release > --- > > Key: GEODE-8018 > URL: https://issues.apache.org/jira/browse/GEODE-8018 > Project: Geode > Issue Type: Task > Components: native client >Reporter: Blake Bender >Priority: Major > > Geode 1.12 is out now, so we need to update LGTM to the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198391#comment-17198391
]
ASF GitHub Bot commented on GEODE-8506:
---
bschuchardt commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r491013014
##
File path:
geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
##
@@ -295,19 +302,48 @@ void releaseBuffer(BufferPool.BufferType type, ByteBuffer
buffer) {
/**
* Releases a previously acquired buffer.
*/
- private void releaseBuffer(ByteBuffer bb, boolean send) {
-if (bb.isDirect()) {
- BBSoftReference bbRef = new BBSoftReference(bb, send);
- if (bb.capacity() <= SMALL_BUFFER_SIZE) {
+ private void releaseBuffer(ByteBuffer buffer, boolean send) {
+if (buffer.isDirect()) {
+ buffer = getPoolableBuffer(buffer);
+ BBSoftReference bbRef = new BBSoftReference(buffer, send);
+ if (buffer.capacity() <= SMALL_BUFFER_SIZE) {
bufferSmallQueue.offer(bbRef);
- } else if (bb.capacity() <= MEDIUM_BUFFER_SIZE) {
+ } else if (buffer.capacity() <= MEDIUM_BUFFER_SIZE) {
bufferMiddleQueue.offer(bbRef);
} else {
bufferLargeQueue.offer(bbRef);
}
} else {
- updateBufferStats(-bb.capacity(), send, false);
+ updateBufferStats(-buffer.capacity(), send, false);
+}
+ }
+
+ /**
+ * If we hand out a buffer that is larger than the requested size we create a
+ * "slice" of the buffer having the requested capacity and hand that out
instead.
+ * When we put the buffer back in the pool we need to find the original,
non-sliced,
+ * buffer. This is held in DirectBuffer in its "attachment" field, which is
a public
+ * method, though DirectBuffer is package-private.
+ */
+ @VisibleForTesting
+ public ByteBuffer getPoolableBuffer(ByteBuffer buffer) {
Review comment:
That seems like an expensive change that would disturb a lot of code.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198393#comment-17198393
]
ASF GitHub Bot commented on GEODE-8506:
---
bschuchardt commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r491016215
##
File path:
geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
##
@@ -98,14 +101,18 @@ private ByteBuffer acquireDirectBuffer(int size, boolean
send) {
if (useDirectBuffers) {
if (size <= MEDIUM_BUFFER_SIZE) {
-return acquirePredefinedFixedBuffer(send, size);
+result = acquirePredefinedFixedBuffer(send, size);
Review comment:
I didn't change that private method and AFAIK the comment is still
accurate
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198394#comment-17198394
]
ASF GitHub Bot commented on GEODE-8506:
---
Bill commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r491018765
##
File path:
geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
##
@@ -295,19 +302,48 @@ void releaseBuffer(BufferPool.BufferType type, ByteBuffer
buffer) {
/**
* Releases a previously acquired buffer.
*/
- private void releaseBuffer(ByteBuffer bb, boolean send) {
-if (bb.isDirect()) {
- BBSoftReference bbRef = new BBSoftReference(bb, send);
- if (bb.capacity() <= SMALL_BUFFER_SIZE) {
+ private void releaseBuffer(ByteBuffer buffer, boolean send) {
+if (buffer.isDirect()) {
+ buffer = getPoolableBuffer(buffer);
+ BBSoftReference bbRef = new BBSoftReference(buffer, send);
+ if (buffer.capacity() <= SMALL_BUFFER_SIZE) {
bufferSmallQueue.offer(bbRef);
- } else if (bb.capacity() <= MEDIUM_BUFFER_SIZE) {
+ } else if (buffer.capacity() <= MEDIUM_BUFFER_SIZE) {
bufferMiddleQueue.offer(bbRef);
} else {
bufferLargeQueue.offer(bbRef);
}
} else {
- updateBufferStats(-bb.capacity(), send, false);
+ updateBufferStats(-buffer.capacity(), send, false);
+}
+ }
+
+ /**
+ * If we hand out a buffer that is larger than the requested size we create a
+ * "slice" of the buffer having the requested capacity and hand that out
instead.
+ * When we put the buffer back in the pool we need to find the original,
non-sliced,
+ * buffer. This is held in DirectBuffer in its "attachment" field, which is
a public
+ * method, though DirectBuffer is package-private.
+ */
+ @VisibleForTesting
+ public ByteBuffer getPoolableBuffer(ByteBuffer buffer) {
Review comment:
That’s fine. But I did want to run it by you.
Subclassing from a byte buffer class would entail making large (lots of
methods), new classes huh?
The alternative would be to return something that is not a byte buffer, but
rather has-a byte buffer. In that case things that call our `BufferPool`
methods would get one of those (non-byte-buffer) things, and that would perturb
lots of calling code.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198395#comment-17198395
]
ASF GitHub Bot commented on GEODE-8506:
---
Bill commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r490654223
##
File path:
geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
##
@@ -98,14 +101,18 @@ private ByteBuffer acquireDirectBuffer(int size, boolean
send) {
if (useDirectBuffers) {
if (size <= MEDIUM_BUFFER_SIZE) {
-return acquirePredefinedFixedBuffer(send, size);
+result = acquirePredefinedFixedBuffer(send, size);
Review comment:
The comment on method `acquirePredefinedFixedBuffer()` now lies. It says:
```
/**
* Acquire direct buffer with predefined default capacity (4096 or 32768)
*/
```
But that method no longer returns buffers with predefined _capacity_.
~Also that method's name seems wrong since it doesn't return a buffer with a
predefined capacity at all, rather, it returns a buffer of the specified
`size`.~ brainf*rt
If I am understanding the method in question, perhaps this would be more
accurate (also eliminates the word "fixed" from the method name since I don't
think that is pertinent):
```
/**
* Acquire direct buffer of size <= MEDIUM_BUFFER_SIZE
*
* Caller ensures that size <= MEDIUM_BUFFER_SIZE on entry. This method
* will not check that!
*/
private ByteBuffer acquireSmallToMediumBuffer(boolean send, int size) {
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198418#comment-17198418
]
ASF GitHub Bot commented on GEODE-8506:
---
bschuchardt commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r491044218
##
File path:
geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
##
@@ -98,14 +101,18 @@ private ByteBuffer acquireDirectBuffer(int size, boolean
send) {
if (useDirectBuffers) {
if (size <= MEDIUM_BUFFER_SIZE) {
-return acquirePredefinedFixedBuffer(send, size);
+result = acquirePredefinedFixedBuffer(send, size);
Review comment:
I would prefer to just remove the magic numbers from the comment. I
don't think the method rename is warranted.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198420#comment-17198420
]
ASF GitHub Bot commented on GEODE-8506:
---
bschuchardt commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r491045634
##
File path:
geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
##
@@ -295,19 +302,48 @@ void releaseBuffer(BufferPool.BufferType type, ByteBuffer
buffer) {
/**
* Releases a previously acquired buffer.
*/
- private void releaseBuffer(ByteBuffer bb, boolean send) {
-if (bb.isDirect()) {
- BBSoftReference bbRef = new BBSoftReference(bb, send);
- if (bb.capacity() <= SMALL_BUFFER_SIZE) {
+ private void releaseBuffer(ByteBuffer buffer, boolean send) {
+if (buffer.isDirect()) {
+ buffer = getPoolableBuffer(buffer);
+ BBSoftReference bbRef = new BBSoftReference(buffer, send);
+ if (buffer.capacity() <= SMALL_BUFFER_SIZE) {
bufferSmallQueue.offer(bbRef);
- } else if (bb.capacity() <= MEDIUM_BUFFER_SIZE) {
+ } else if (buffer.capacity() <= MEDIUM_BUFFER_SIZE) {
bufferMiddleQueue.offer(bbRef);
} else {
bufferLargeQueue.offer(bbRef);
}
} else {
- updateBufferStats(-bb.capacity(), send, false);
+ updateBufferStats(-buffer.capacity(), send, false);
+}
+ }
+
+ /**
+ * If we hand out a buffer that is larger than the requested size we create a
+ * "slice" of the buffer having the requested capacity and hand that out
instead.
+ * When we put the buffer back in the pool we need to find the original,
non-sliced,
+ * buffer. This is held in DirectBuffer in its "attachment" field, which is
a public
+ * method, though DirectBuffer is package-private.
+ */
+ @VisibleForTesting
+ public ByteBuffer getPoolableBuffer(ByteBuffer buffer) {
Review comment:
Subclassing ByteBuffer isn't possible due to the class having final
methods. Introducing a new class seems desirable but would affect a lot of
code and this needs to be backported to 1.12 and 1.13 support branches. In
such cases it's best to keep code changes to a minimum.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (GEODE-8267) serverRestartsAfterOneLocatorDies hangs
[
https://issues.apache.org/jira/browse/GEODE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ernest Burghardt reassigned GEODE-8267:
---
Assignee: (was: Ernest Burghardt)
> serverRestartsAfterOneLocatorDies hangs
> ---
>
> Key: GEODE-8267
> URL: https://issues.apache.org/jira/browse/GEODE-8267
> Project: Geode
> Issue Type: Bug
> Components: configuration, locator, membership
>Reporter: Bill Burcham
>Priority: Major
>
> hang:
> [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/DistributedTestOpenJDK11/builds/275#A]
>
> The test hung in serverRestartsAfterOneLocatorDies after another failure in
> the same test class.
> Here's the hung thread:
> {noformat}
> "Test worker" #27 prio=5 os_prio=0 cpu=5016.73ms elapsed=5638.52s
> tid=0x7f01c8ad4800 nid=0x18 runnable [0x7f019872c000]"Test worker"
> #27 prio=5 os_prio=0 cpu=5016.73ms elapsed=5638.52s tid=0x7f01c8ad4800
> nid=0x18 runnable [0x7f019872c000] java.lang.Thread.State: RUNNABLE at
> java.net.SocketInputStream.socketRead0(java.base@11.0.7/Native Method) at
> java.net.SocketInputStream.socketRead(java.base@11.0.7/SocketInputStream.java:115)
> at
> java.net.SocketInputStream.read(java.base@11.0.7/SocketInputStream.java:168)
> at
> java.net.SocketInputStream.read(java.base@11.0.7/SocketInputStream.java:140)
> at
> java.io.BufferedInputStream.fill(java.base@11.0.7/BufferedInputStream.java:252)
> at
> java.io.BufferedInputStream.read(java.base@11.0.7/BufferedInputStream.java:271)
> - locked <0xd08fe7a0> (a java.io.BufferedInputStream) at
> java.io.DataInputStream.readByte(java.base@11.0.7/DataInputStream.java:270)
> at
> sun.rmi.transport.StreamRemoteCall.executeCall(java.rmi@11.0.7/StreamRemoteCall.java:240)
> at sun.rmi.server.UnicastRef.invoke(java.rmi@11.0.7/UnicastRef.java:164) at
> java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(java.rmi@11.0.7/RemoteObjectInvocationHandler.java:217)
> at
> java.rmi.server.RemoteObjectInvocationHandler.invoke(java.rmi@11.0.7/RemoteObjectInvocationHandler.java:162)
> at com.sun.proxy.$Proxy53.executeMethodOnObject(Unknown Source) at
> org.apache.geode.test.dunit.VM.executeMethodOnObject(VM.java:607) at
> org.apache.geode.test.dunit.VM.invoke(VM.java:450) at
> org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:268)
> at
> org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:261)
> at
> org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:256)
> at
> org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest.serverRestartsAfterOneLocatorDies(ClusterConfigLocatorRestartDUnitTest.java:114)
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.7/Native
> Method) {noformat}
> Here's the previous test failure, which may have affected the test that hung:
> {code:java}
> org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest
> > serverRestartHangsWaitingForStartupMessageResponse FAILED
> org.junit.runners.model.TestTimedOutException: test timed out after
> 30 milliseconds
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
> at java.net.SocketInputStream.read(SocketInputStream.java:168)
> at java.net.SocketInputStream.read(SocketInputStream.java:140)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:271)
> at java.io.DataInputStream.readByte(DataInputStream.java:270)
> at
> sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:240)
> at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:164)
> at
> java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:217)
> at
> java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:162)
> at com.sun.proxy.$Proxy53.executeMethodOnObject(Unknown Source)
> at org.apache.geode.test.dunit.VM.executeMethodOnObject(VM.java:607)
> at org.apache.geode.test.dunit.VM.invoke(VM.java:437)
> at
> org.apache.geode.test.junit.rules.VMProvider.invoke(VMProvider.java:94)
> at
> org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest.serverRestartHangsWaitingForStartupMessageResponse(ClusterConfigLocatorRestartDUnitTest.java:176)
> {code}
> Seems like 300s should be long enough so I fear there may be a real problem
> here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8477) getMembersNotShuttingDown() doesn't use consistent set of shutdown members
[
https://issues.apache.org/jira/browse/GEODE-8477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198425#comment-17198425
]
Ernest Burghardt commented on GEODE-8477:
-
[~burcham] do we have a failing test that shows this issue?
> getMembersNotShuttingDown() doesn't use consistent set of shutdown members
> --
>
> Key: GEODE-8477
> URL: https://issues.apache.org/jira/browse/GEODE-8477
> Project: Geode
> Issue Type: Bug
> Components: membership
>Affects Versions: 1.12.0, 1.13.0, 1.14.0
>Reporter: Bill Burcham
>Priority: Major
>
> {{GMSMembership.getMembersNotShuttingDown()}} correctly accesses
> {{latestView}} under the protection of the read lock {{latestViewReadLock}}
> associated with the read-write lock {{latestViewLock}} ✓
> However that method also accesses the {{shutdownMembers}} collection, a
> {{(Bounded)LinkedHashMap}}. That class does not provide concurrency
> protection. Furthermore {{shutdownMembers}} is not read/written consistently
> under the protection of any synchronization ✕
> The result is that in this method, {{shutdownMembers}} may be corrupt and its
> contents may be arbitrarily stale.
> One solution is to put all reads and writes of {{shutdownMembers}} under the
> protection of the same read-write lock as {{latestView}}. If we do this we
> can eliminate existing synchronization on {{shutdownMembers}} in
> {{shutdownMessageReceived()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198429#comment-17198429
]
ASF GitHub Bot commented on GEODE-8506:
---
Bill commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r491065581
##
File path:
geode-core/src/test/java/org/apache/geode/internal/net/BufferPoolTest.java
##
@@ -133,27 +135,42 @@ public void checkBufferSizeAfterAllocation() throws
Exception {
@Test
public void checkBufferSizeAfterAcquire() throws Exception {
+// allocate a small buffer and a larger buffer. Check their sizes, etc and
then
+// release and reacquire them. They should be from separate buffer pools
so there
+// should still be a small buffer and a larger buffer.
Review comment:
I'm retracting that comment since these variable names existed before
this PR.
Any such changes should be pursued in connection with a refactoring ticket.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198430#comment-17198430
]
ASF GitHub Bot commented on GEODE-8506:
---
Bill commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r491065723
##
File path:
geode-core/src/test/java/org/apache/geode/internal/net/BufferPoolTest.java
##
@@ -133,27 +135,42 @@ public void checkBufferSizeAfterAllocation() throws
Exception {
@Test
public void checkBufferSizeAfterAcquire() throws Exception {
+// allocate a small buffer and a larger buffer. Check their sizes, etc and
then
+// release and reacquire them. They should be from separate buffer pools
so there
+// should still be a small buffer and a larger buffer.
ByteBuffer buffer = bufferPool.acquireDirectReceiveBuffer(100);
ByteBuffer newBuffer =
bufferPool.acquireDirectReceiveBuffer(1);
-assertThat(buffer.capacity()).isGreaterThanOrEqualTo(4096);
-assertThat(newBuffer.capacity()).isGreaterThanOrEqualTo(32768);
+assertThat(buffer.capacity()).isEqualTo(100);
+assertThat(newBuffer.capacity()).isEqualTo(1);
+assertThat(buffer.isDirect()).isTrue();
+assertThat(newBuffer.isDirect()).isTrue();
+assertThat(bufferPool.getPoolableBuffer(buffer).capacity())
+.isGreaterThanOrEqualTo(4096);
Review comment:
I'm retracting that comment since these constants existed before this PR.
Any such changes should be pursued in connection with a refactoring ticket.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[
https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198431#comment-17198431
]
ASF GitHub Bot commented on GEODE-8506:
---
Bill commented on a change in pull request #5525:
URL: https://github.com/apache/geode/pull/5525#discussion_r491066133
##
File path:
geode-core/src/main/java/org/apache/geode/internal/net/BufferPool.java
##
@@ -98,14 +101,18 @@ private ByteBuffer acquireDirectBuffer(int size, boolean
send) {
if (useDirectBuffers) {
if (size <= MEDIUM_BUFFER_SIZE) {
-return acquirePredefinedFixedBuffer(send, size);
+result = acquirePredefinedFixedBuffer(send, size);
Review comment:
I'm retracting that comment since this method name existed before this
PR and was arguably already wrong before this PR.
Any such changes should be pursued in connection with a refactoring ticket.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BufferPool returns byte buffers that may be much larger than requested
> --
>
> Key: GEODE-8506
> URL: https://issues.apache.org/jira/browse/GEODE-8506
> Project: Geode
> Issue Type: Improvement
> Components: membership
>Reporter: Bruce J Schuchardt
>Assignee: Bruce J Schuchardt
>Priority: Major
> Labels: pull-request-available
>
> BufferPool manages several pools of direct-memory ByteBuffers. When asked
> for a ByteBuffer of size X you may receive a buffer that is any size greater
> than or equal to X. For users of this pool this is unexpected behavior and
> is causing some trouble.
> MessageStreamer, for instance, performs message "chunking" based on the size
> of a socket's buffer size. It requests a byte buffer of that size and then
> fills it over and over again with message chunks to be written to the socket.
> But it does this based on the buffer's capacity, which may be much larger
> than the expected buffer size. This results in incorrect chunking and
> requires larger buffers in the receiver of these message chunks.
> BufferPool should always return a buffer that has exactly the requested
> capacity. It could be a _slice_ of a pooled buffer, for instance. That
> would let it hand out a larger buffer while not confusing the code that
> requested the buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (GEODE-8509) remove reflection on ByteBuffer in Geode comms
Bruce J Schuchardt created GEODE-8509: - Summary: remove reflection on ByteBuffer in Geode comms Key: GEODE-8509 URL: https://issues.apache.org/jira/browse/GEODE-8509 Project: Geode Issue Type: Improvement Components: membership, messaging Reporter: Bruce J Schuchardt In GEODE-8506 we introduced reflection on byte buffers to ascertain the underlying buffer of a _slice_. We should remove this reflection and change BufferPool to return an object that knows about the _slice_ and its underlying buffer. This will improve performance on every buffer-release operation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce J Schuchardt resolved GEODE-8506. --- Fix Version/s: 1.14.0 Resolution: Fixed > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198539#comment-17198539 ] ASF GitHub Bot commented on GEODE-8506: --- bschuchardt merged pull request #5525: URL: https://github.com/apache/geode/pull/5525 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198541#comment-17198541 ] ASF subversion and git services commented on GEODE-8506: Commit a11b9c076a72609ff00802c010b6e32262228776 in geode's branch refs/heads/develop from Bruce Schuchardt [ https://gitbox.apache.org/repos/asf?p=geode.git;h=a11b9c0 ] GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525) * GEODE-8506: BufferPool returns byte buffers that may be much larger than requested Create a "slice" of the acquired buffer to return to the caller instead of returning a buffer larger than what was requested. On return we fish out the parent buffer and put it back in the pool. * cache reflection method, remove magic numbers in test and BufferPool The one dunit failure seems to be GEODE-7710 > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198542#comment-17198542 ] ASF subversion and git services commented on GEODE-8506: Commit a11b9c076a72609ff00802c010b6e32262228776 in geode's branch refs/heads/develop from Bruce Schuchardt [ https://gitbox.apache.org/repos/asf?p=geode.git;h=a11b9c0 ] GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525) * GEODE-8506: BufferPool returns byte buffers that may be much larger than requested Create a "slice" of the acquired buffer to return to the caller instead of returning a buffer larger than what was requested. On return we fish out the parent buffer and put it back in the pool. * cache reflection method, remove magic numbers in test and BufferPool The one dunit failure seems to be GEODE-7710 > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-7710) JMXMBeanReconnectDUnitTest fails intermittently because one locator is missing the LockServiceMXBean
[
https://issues.apache.org/jira/browse/GEODE-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198543#comment-17198543
]
ASF subversion and git services commented on GEODE-7710:
Commit a11b9c076a72609ff00802c010b6e32262228776 in geode's branch
refs/heads/develop from Bruce Schuchardt
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=a11b9c0 ]
GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525)
* GEODE-8506: BufferPool returns byte buffers that may be much larger than
requested
Create a "slice" of the acquired buffer to return to the caller instead
of returning a buffer larger than what was requested. On return we fish
out the parent buffer and put it back in the pool.
* cache reflection method, remove magic numbers in test and BufferPool
The one dunit failure seems to be GEODE-7710
> JMXMBeanReconnectDUnitTest fails intermittently because one locator is
> missing the LockServiceMXBean
>
>
> Key: GEODE-7710
> URL: https://issues.apache.org/jira/browse/GEODE-7710
> Project: Geode
> Issue Type: Bug
> Components: tests
>Affects Versions: 1.13.0
>Reporter: Kirk Lund
>Assignee: Kirk Lund
>Priority: Major
> Labels: flaky
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Multiple tests in JMXMBeanReconnectDUnitTest may fail an await due to one of
> the locators missing the LockServiceMXBean for the cluster config service.
> {noformat}
> but could not find:
>
> <[GemFire:service=LockService,name=__CLUSTER_CONFIG_LS,type=Member,member=locator1]>
> {noformat}
> These test failures are caused by *GEODE-7739*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (GEODE-8510) Write Testing.md to Illustrate How to Contribute New Tests
Michael Martell created GEODE-8510: -- Summary: Write Testing.md to Illustrate How to Contribute New Tests Key: GEODE-8510 URL: https://issues.apache.org/jira/browse/GEODE-8510 Project: Geode Issue Type: Improvement Components: native client Reporter: Michael Martell Testing for geode-native consists of a lot of different testing environments including NUnit, GoogleTest, XUnit, as well as legacy tests that use our own built-in test frameworks. We need a document describing the methodology for writing new tests using our new frameworks for C++ and .NET. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198547#comment-17198547 ] ASF subversion and git services commented on GEODE-8506: Commit a11b9c076a72609ff00802c010b6e32262228776 in geode's branch refs/heads/develop from Bruce Schuchardt [ https://gitbox.apache.org/repos/asf?p=geode.git;h=a11b9c0 ] GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525) * GEODE-8506: BufferPool returns byte buffers that may be much larger than requested Create a "slice" of the acquired buffer to return to the caller instead of returning a buffer larger than what was requested. On return we fish out the parent buffer and put it back in the pool. * cache reflection method, remove magic numbers in test and BufferPool The one dunit failure seems to be GEODE-7710 > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-7710) JMXMBeanReconnectDUnitTest fails intermittently because one locator is missing the LockServiceMXBean
[
https://issues.apache.org/jira/browse/GEODE-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198549#comment-17198549
]
ASF subversion and git services commented on GEODE-7710:
Commit a11b9c076a72609ff00802c010b6e32262228776 in geode's branch
refs/heads/develop from Bruce Schuchardt
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=a11b9c0 ]
GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525)
* GEODE-8506: BufferPool returns byte buffers that may be much larger than
requested
Create a "slice" of the acquired buffer to return to the caller instead
of returning a buffer larger than what was requested. On return we fish
out the parent buffer and put it back in the pool.
* cache reflection method, remove magic numbers in test and BufferPool
The one dunit failure seems to be GEODE-7710
> JMXMBeanReconnectDUnitTest fails intermittently because one locator is
> missing the LockServiceMXBean
>
>
> Key: GEODE-7710
> URL: https://issues.apache.org/jira/browse/GEODE-7710
> Project: Geode
> Issue Type: Bug
> Components: tests
>Affects Versions: 1.13.0
>Reporter: Kirk Lund
>Assignee: Kirk Lund
>Priority: Major
> Labels: flaky
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Multiple tests in JMXMBeanReconnectDUnitTest may fail an await due to one of
> the locators missing the LockServiceMXBean for the cluster config service.
> {noformat}
> but could not find:
>
> <[GemFire:service=LockService,name=__CLUSTER_CONFIG_LS,type=Member,member=locator1]>
> {noformat}
> These test failures are caused by *GEODE-7739*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198548#comment-17198548 ] ASF subversion and git services commented on GEODE-8506: Commit a11b9c076a72609ff00802c010b6e32262228776 in geode's branch refs/heads/develop from Bruce Schuchardt [ https://gitbox.apache.org/repos/asf?p=geode.git;h=a11b9c0 ] GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525) * GEODE-8506: BufferPool returns byte buffers that may be much larger than requested Create a "slice" of the acquired buffer to return to the caller instead of returning a buffer larger than what was requested. On return we fish out the parent buffer and put it back in the pool. * cache reflection method, remove magic numbers in test and BufferPool The one dunit failure seems to be GEODE-7710 > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198568#comment-17198568 ] ASF subversion and git services commented on GEODE-8506: Commit 709fe7b8c4c61eb610a73fcf71005fe93a4d6e1c in geode's branch refs/heads/support/1.13 from Bruce Schuchardt [ https://gitbox.apache.org/repos/asf?p=geode.git;h=709fe7b ] GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525) * GEODE-8506: BufferPool returns byte buffers that may be much larger than requested Create a "slice" of the acquired buffer to return to the caller instead of returning a buffer larger than what was requested. On return we fish out the parent buffer and put it back in the pool. * cache reflection method, remove magic numbers in test and BufferPool (cherry picked from commit a11b9c076a72609ff00802c010b6e32262228776) > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198569#comment-17198569 ] ASF subversion and git services commented on GEODE-8506: Commit 709fe7b8c4c61eb610a73fcf71005fe93a4d6e1c in geode's branch refs/heads/support/1.13 from Bruce Schuchardt [ https://gitbox.apache.org/repos/asf?p=geode.git;h=709fe7b ] GEODE-8506: BufferPool returns byte buffers that may be much larger t… (#5525) * GEODE-8506: BufferPool returns byte buffers that may be much larger than requested Create a "slice" of the acquired buffer to return to the caller instead of returning a buffer larger than what was requested. On return we fish out the parent buffer and put it back in the pool. * cache reflection method, remove magic numbers in test and BufferPool (cherry picked from commit a11b9c076a72609ff00802c010b6e32262228776) > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8508) Update .lgtm.yml to latest (1.13) Geode release
[ https://issues.apache.org/jira/browse/GEODE-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198586#comment-17198586 ] ASF subversion and git services commented on GEODE-8508: Commit 983036fc86b5254ca1acc3ec10af9a2a82cff693 in geode-native's branch refs/heads/develop from Alberto Bustamante Reyes [ https://gitbox.apache.org/repos/asf?p=geode-native.git;h=983036f ] GEODE-8508: Update LGTM to use Geode 1.13 (#652) > Update .lgtm.yml to latest (1.13) Geode release > --- > > Key: GEODE-8508 > URL: https://issues.apache.org/jira/browse/GEODE-8508 > Project: Geode > Issue Type: Task > Components: native client >Reporter: Alberto Bustamante Reyes >Assignee: Alberto Bustamante Reyes >Priority: Major > Labels: pull-request-available > > As done in GEODE-8018 for Geode 1.12, now that Geode 1.13 is out it is > needed to update LGTM config file to use the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8508) Update .lgtm.yml to latest (1.13) Geode release
[ https://issues.apache.org/jira/browse/GEODE-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198587#comment-17198587 ] ASF GitHub Bot commented on GEODE-8508: --- moleske merged pull request #652: URL: https://github.com/apache/geode-native/pull/652 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Update .lgtm.yml to latest (1.13) Geode release > --- > > Key: GEODE-8508 > URL: https://issues.apache.org/jira/browse/GEODE-8508 > Project: Geode > Issue Type: Task > Components: native client >Reporter: Alberto Bustamante Reyes >Assignee: Alberto Bustamante Reyes >Priority: Major > Labels: pull-request-available > > As done in GEODE-8018 for Geode 1.12, now that Geode 1.13 is out it is > needed to update LGTM config file to use the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (GEODE-8511) GeodeRedisServer exception tests should not require port 40404
Raymond Ingles created GEODE-8511: - Summary: GeodeRedisServer exception tests should not require port 40404 Key: GEODE-8511 URL: https://issues.apache.org/jira/browse/GEODE-8511 Project: Geode Issue Type: Test Components: redis Affects Versions: 1.14.0, 1.13.1 Reporter: Raymond Ingles In GeodeRedisServerStartUpAcceptanceTest, these three tests failed in the pipeline due to some other process holding port 40404. However, this port isn't required and in the test setup we should set the server-port to 0. Failing tests: shouldReturnErrorMessage_givenInvalidBindAddress shouldReturnErrorMessage_givenSamePortAndAddress shouldReturnErrorMessage_givenSamePortAndAllAddresses See this run: https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/AcceptanceTestOpenJDK8/builds/443 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (GEODE-8511) GeodeRedisServer exception tests should not require port 40404
[ https://issues.apache.org/jira/browse/GEODE-8511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Ingles reassigned GEODE-8511: - Assignee: Raymond Ingles > GeodeRedisServer exception tests should not require port 40404 > -- > > Key: GEODE-8511 > URL: https://issues.apache.org/jira/browse/GEODE-8511 > Project: Geode > Issue Type: Test > Components: redis >Affects Versions: 1.14.0, 1.13.1 >Reporter: Raymond Ingles >Assignee: Raymond Ingles >Priority: Major > > In GeodeRedisServerStartUpAcceptanceTest, these three tests failed in the > pipeline due to some other process holding port 40404. However, this port > isn't required and in the test setup we should set the server-port to 0. > > Failing tests: > shouldReturnErrorMessage_givenInvalidBindAddress > shouldReturnErrorMessage_givenSamePortAndAddress > shouldReturnErrorMessage_givenSamePortAndAllAddresses > > See this run: > https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/AcceptanceTestOpenJDK8/builds/443 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (GEODE-8512) Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory
Mario Salazar de Torres created GEODE-8512: -- Summary: Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory Key: GEODE-8512 URL: https://issues.apache.org/jira/browse/GEODE-8512 Project: Geode Issue Type: Improvement Components: native client Affects Versions: 1.13.0 Reporter: Mario Salazar de Torres Within Java client, you can specify if the PdxInstance you want to create is a Java domain class or not, but this is not the case for geode-native C++ client. Thing is that in order to implement JsonFormatter in C++ client, having this change is mandatory, otherwise creating PdxType's with __GEMFIRE_JSON classname will present some issues while using cquery's. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GEODE-8512) Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory
[ https://issues.apache.org/jira/browse/GEODE-8512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Salazar de Torres updated GEODE-8512: --- Priority: Minor (was: Major) > Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory > > > Key: GEODE-8512 > URL: https://issues.apache.org/jira/browse/GEODE-8512 > Project: Geode > Issue Type: Improvement > Components: native client >Affects Versions: 1.13.0 >Reporter: Mario Salazar de Torres >Priority: Minor > > Within Java client, you can specify if the PdxInstance you want to create is > a Java domain class or not, but this is not the case for geode-native C++ > client. > Thing is that in order to implement JsonFormatter in C++ client, having this > change is mandatory, otherwise creating PdxType's with __GEMFIRE_JSON > classname will present some issues while using cquery's. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GEODE-8512) Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory
[ https://issues.apache.org/jira/browse/GEODE-8512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Salazar de Torres updated GEODE-8512: --- Labels: JsonFormatter (was: ) > Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory > > > Key: GEODE-8512 > URL: https://issues.apache.org/jira/browse/GEODE-8512 > Project: Geode > Issue Type: Improvement > Components: native client >Affects Versions: 1.13.0 >Reporter: Mario Salazar de Torres >Priority: Minor > Labels: JsonFormatter > > Within Java client, you can specify if the PdxInstance you want to create is > a Java domain class or not, but this is not the case for geode-native C++ > client. > Thing is that in order to implement JsonFormatter in C++ client, having this > change is mandatory, otherwise creating PdxType's with __GEMFIRE_JSON > classname will present some issues while using cquery's. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GEODE-8512) Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory
[ https://issues.apache.org/jira/browse/GEODE-8512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated GEODE-8512: -- Labels: JsonFormatter pull-request-available (was: JsonFormatter) > Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory > > > Key: GEODE-8512 > URL: https://issues.apache.org/jira/browse/GEODE-8512 > Project: Geode > Issue Type: Improvement > Components: native client >Affects Versions: 1.13.0 >Reporter: Mario Salazar de Torres >Priority: Minor > Labels: JsonFormatter, pull-request-available > > Within Java client, you can specify if the PdxInstance you want to create is > a Java domain class or not, but this is not the case for geode-native C++ > client. > Thing is that in order to implement JsonFormatter in C++ client, having this > change is mandatory, otherwise creating PdxType's with __GEMFIRE_JSON > classname will present some issues while using cquery's. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-8512) Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory
[ https://issues.apache.org/jira/browse/GEODE-8512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198613#comment-17198613 ] ASF GitHub Bot commented on GEODE-8512: --- gaussianrecurrence opened a new pull request #653: URL: https://github.com/apache/geode-native/pull/653 Add Cache::createPdxInstanceFactory overload with expectDomainClass parameter This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Be able to specify 'expectDomainClass' when calling createPdxInstanceFactory > > > Key: GEODE-8512 > URL: https://issues.apache.org/jira/browse/GEODE-8512 > Project: Geode > Issue Type: Improvement > Components: native client >Affects Versions: 1.13.0 >Reporter: Mario Salazar de Torres >Priority: Minor > Labels: JsonFormatter > > Within Java client, you can specify if the PdxInstance you want to create is > a Java domain class or not, but this is not the case for geode-native C++ > client. > Thing is that in order to implement JsonFormatter in C++ client, having this > change is mandatory, otherwise creating PdxType's with __GEMFIRE_JSON > classname will present some issues while using cquery's. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GEODE-8506) BufferPool returns byte buffers that may be much larger than requested
[ https://issues.apache.org/jira/browse/GEODE-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen Nichols updated GEODE-8506: Fix Version/s: 1.13.1 > BufferPool returns byte buffers that may be much larger than requested > -- > > Key: GEODE-8506 > URL: https://issues.apache.org/jira/browse/GEODE-8506 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0, 1.13.1 > > > BufferPool manages several pools of direct-memory ByteBuffers. When asked > for a ByteBuffer of size X you may receive a buffer that is any size greater > than or equal to X. For users of this pool this is unexpected behavior and > is causing some trouble. > MessageStreamer, for instance, performs message "chunking" based on the size > of a socket's buffer size. It requests a byte buffer of that size and then > fills it over and over again with message chunks to be written to the socket. > But it does this based on the buffer's capacity, which may be much larger > than the expected buffer size. This results in incorrect chunking and > requires larger buffers in the receiver of these message chunks. > BufferPool should always return a buffer that has exactly the requested > capacity. It could be a _slice_ of a pooled buffer, for instance. That > would let it hand out a larger buffer while not confusing the code that > requested the buffer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
