[jira] [Created] (GEODE-7838) Info about num of servers during rebalance command
Mario Kevo created GEODE-7838: - Summary: Info about num of servers during rebalance command Key: GEODE-7838 URL: https://issues.apache.org/jira/browse/GEODE-7838 Project: Geode Issue Type: Improvement Components: gfsh Reporter: Mario Kevo In case during rebalance some members joined or departed rebalance will automatically restarted. But from the user side it can not see on how many members rebalance was executed. We have a situation when have 3 servers and during rebalance one is departed and rebalance is executed on other 2 servers. But after rebalance is finished server is back and user have a wrong perception on how many servers rebalance is executed. It will be good to have another row in result table to print this message. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (GEODE-7838) Info about num of servers during rebalance command
[ https://issues.apache.org/jira/browse/GEODE-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mario Kevo reassigned GEODE-7838: - Assignee: Mario Kevo > Info about num of servers during rebalance command > -- > > Key: GEODE-7838 > URL: https://issues.apache.org/jira/browse/GEODE-7838 > Project: Geode > Issue Type: Improvement > Components: gfsh >Reporter: Mario Kevo >Assignee: Mario Kevo >Priority: Major > > In case during rebalance some members joined or departed rebalance will > automatically restarted. > But from the user side it can not see on how many members rebalance was > executed. > We have a situation when have 3 servers and during rebalance one is departed > and rebalance is executed on other 2 servers. But after rebalance is finished > server is back and user have a wrong perception on how many servers rebalance > is executed. > It will be good to have another row in result table to print this message. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GEODE-7832) Remove connection "send semaphores" from DirectChannel
[ https://issues.apache.org/jira/browse/GEODE-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Ramos updated GEODE-7832: -- Labels: GeodeCommons (was: ) > Remove connection "send semaphores" from DirectChannel > -- > > Key: GEODE-7832 > URL: https://issues.apache.org/jira/browse/GEODE-7832 > Project: Geode > Issue Type: Improvement > Components: membership >Reporter: Bruce J Schuchardt >Assignee: Juan Ramos >Priority: Major > Labels: GeodeCommons > Time Spent: 10m > Remaining Estimate: 0h > > Remove the acquiredSendSemaphore/acquireGroupSendSemaphore and related > methods from DirectChannel/TCPConduit code. Connection.isReaderThread() and > associated makeReaderThread() methods can also be deleted because they're > only used by the semaphore code. > These semaphores only constrain messaging if you set a couple of undocumented > system properties, p2p.maxGroupSenders and p2p.maxConnectionSenders to some > small value. These are (MAX_INT / 2) by default. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-7808) standardize on use of LocatorAddress/HostAddress for connection formation
[ https://issues.apache.org/jira/browse/GEODE-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050373#comment-17050373 ] ASF subversion and git services commented on GEODE-7808: Commit 3ace639daa243efd28f4ca8bb7b275666d514fce in geode's branch refs/heads/feature/GEODE-7808c from Bruce Schuchardt [ https://gitbox.apache.org/repos/asf?p=geode.git;h=3ace639 ] Revert "GEODE-7808: standardize on use of HostAndPort to form client-side connections (#4743)" This reverts commit 0af626462642c6352840cd6e81a5265c74045c7f. That commit seems to have caused a severe performance drop in several Benchmark tests: org.apache.geode.benchmark.tests.PartitionedGetBenchmark average ops/second Baseline:981794.46 Test: 41239.82 Difference: -95.8% org.apache.geode.benchmark.tests.ReplicatedGetBenchmark average ops/second Baseline:972769.18 Test: 41299.96 Difference: -95.8% org.apache.geode.benchmark.tests.PartitionedNonIndexedQueryBenchmark average ops/second Baseline:90.05 Test:70.52 Difference: -21.7% > standardize on use of LocatorAddress/HostAddress for connection formation > - > > Key: GEODE-7808 > URL: https://issues.apache.org/jira/browse/GEODE-7808 > Project: Geode > Issue Type: Improvement > Components: membership, messaging >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Fix For: 1.13.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > We currently use InetAddress and InetSocketAddress in many places to identify > locators, servers and peers. Some work has been done in the past couple of > years to reduce the use of these in order to accommodate changes in IP > addresses due to various causes. The class LocatorAddress was created to > help with this and it is able to hold a host name without resolving it until > that resolution is needed to form a tcp/ip connection. > These days we are seeing more and more movement into cloud computing and the > need to accommodate IP address changes is becoming a bigger issue. To that > end we would like to change our primary client/server and WAN communication > interfaces to stop taking InetAddresses and InetSocketAddresses as arguments > and, instead, take something like a LocatorAddress that can hold an > unresolved hostname that our communication implementations will resolve when > needed. > To that end we should also remove the hostname->inetaddress cache in > SocketCreator and rely on the operating system's DNS cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (GEODE-7839) add gfsh support for connecting with a oauth token instead of password
[ https://issues.apache.org/jira/browse/GEODE-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darrel Schneider reassigned GEODE-7839: --- Assignee: Darrel Schneider > add gfsh support for connecting with a oauth token instead of password > -- > > Key: GEODE-7839 > URL: https://issues.apache.org/jira/browse/GEODE-7839 > Project: Geode > Issue Type: Improvement > Components: gfsh >Reporter: Darrel Schneider >Assignee: Darrel Schneider >Priority: Major > > Currently gfsh connect only has support for making a secure connection given > a username and password. > If you already have obtained an oauth token then it would be nice to have a > way to give that to gfsh and have it pass it on to the server side > SecurityManager. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (GEODE-7839) add gfsh support for connecting with a oauth token instead of password
Darrel Schneider created GEODE-7839: --- Summary: add gfsh support for connecting with a oauth token instead of password Key: GEODE-7839 URL: https://issues.apache.org/jira/browse/GEODE-7839 Project: Geode Issue Type: Improvement Components: gfsh Reporter: Darrel Schneider Currently gfsh connect only has support for making a secure connection given a username and password. If you already have obtained an oauth token then it would be nice to have a way to give that to gfsh and have it pass it on to the server side SecurityManager. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-7839) add gfsh support for connecting with a oauth token instead of password
[ https://issues.apache.org/jira/browse/GEODE-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050386#comment-17050386 ] ASF subversion and git services commented on GEODE-7839: Commit 3ad4f1711562f928891e9ceb0e99532403ba2124 in geode's branch refs/heads/develop from Darrel Schneider [ https://gitbox.apache.org/repos/asf?p=geode.git;h=3ad4f17 ] GEODE-7839: add gfsh oauth token support (#4736) The gfsh connect command now has a --token option. Co-authored-by: Joris Melchior Co-authored-by: Darrel Schneider > add gfsh support for connecting with a oauth token instead of password > -- > > Key: GEODE-7839 > URL: https://issues.apache.org/jira/browse/GEODE-7839 > Project: Geode > Issue Type: Improvement > Components: gfsh >Reporter: Darrel Schneider >Assignee: Darrel Schneider >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently gfsh connect only has support for making a secure connection given > a username and password. > If you already have obtained an oauth token then it would be nice to have a > way to give that to gfsh and have it pass it on to the server side > SecurityManager. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GEODE-7839) add gfsh support for connecting with a oauth token instead of password
[ https://issues.apache.org/jira/browse/GEODE-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darrel Schneider resolved GEODE-7839. - Fix Version/s: 1.13.0 Resolution: Fixed > add gfsh support for connecting with a oauth token instead of password > -- > > Key: GEODE-7839 > URL: https://issues.apache.org/jira/browse/GEODE-7839 > Project: Geode > Issue Type: Improvement > Components: gfsh >Reporter: Darrel Schneider >Assignee: Darrel Schneider >Priority: Major > Fix For: 1.13.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Currently gfsh connect only has support for making a secure connection given > a username and password. > If you already have obtained an oauth token then it would be nice to have a > way to give that to gfsh and have it pass it on to the server side > SecurityManager. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-7819) Add debugging support to SerializableTemporaryFolder
[ https://issues.apache.org/jira/browse/GEODE-7819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050398#comment-17050398 ] ASF subversion and git services commented on GEODE-7819: Commit bcd3655205d9a2341e27b2c554796fe78fab12c8 in geode's branch refs/heads/develop from Kirk Lund [ https://gitbox.apache.org/repos/asf?p=geode.git;h=bcd3655 ] GEODE-7819: Add debugging support to SerializableTemporaryFolder (#4740) Specifying delete(false) will prevent deletion of the temporary folder tree during tear down. Specifying copyTo(directory) with optional when(When) will result in copying the contents of the temporary folder tree to the destination directory during tear down. > Add debugging support to SerializableTemporaryFolder > > > Key: GEODE-7819 > URL: https://issues.apache.org/jira/browse/GEODE-7819 > Project: Geode > Issue Type: Wish > Components: tests >Reporter: Kirk Lund >Assignee: Kirk Lund >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When we are debugging a distributed test that uses > SerializableTemporaryFolder for Server and Locator files, we need a way to > save off the files. This is especially important for debugging remote > failures in the cloud. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (GEODE-7840) Redis pub/sub does not handle binary messages correctly
Jens Deppe created GEODE-7840: - Summary: Redis pub/sub does not handle binary messages correctly Key: GEODE-7840 URL: https://issues.apache.org/jira/browse/GEODE-7840 Project: Geode Issue Type: Bug Components: redis Reporter: Jens Deppe The new pub/sub functionality currently only considers published messages to be actual strings and does not account for them being binary data. This results in messages being corrupted since they end up being UTF-8 encoded. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (GEODE-7840) Redis pub/sub does not handle binary messages correctly
[ https://issues.apache.org/jira/browse/GEODE-7840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jens Deppe reassigned GEODE-7840: - Assignee: Jens Deppe > Redis pub/sub does not handle binary messages correctly > --- > > Key: GEODE-7840 > URL: https://issues.apache.org/jira/browse/GEODE-7840 > Project: Geode > Issue Type: Bug > Components: redis >Reporter: Jens Deppe >Assignee: Jens Deppe >Priority: Major > > The new pub/sub functionality currently only considers published messages to > be actual strings and does not account for them being binary data. This > results in messages being corrupted since they end up being UTF-8 encoded. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-7421) Ability: can deploy jar by REST API for Management
[ https://issues.apache.org/jira/browse/GEODE-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050474#comment-17050474 ] ASF subversion and git services commented on GEODE-7421: Commit a61f9518f3497afafaf0d89144fe85f4baa91318 in geode's branch refs/heads/develop from Joris Melchior [ https://gitbox.apache.org/repos/asf?p=geode.git;h=a61f951 ] GEODE-7421: deployment should use PUT instead of POST in REST request Co-authored-by: Joris Melchior Co-authored-by: Jinmei Liao > Ability: can deploy jar by REST API for Management > -- > > Key: GEODE-7421 > URL: https://issues.apache.org/jira/browse/GEODE-7421 > Project: Geode > Issue Type: New Feature > Components: management, rest (admin) >Reporter: Gang Yan >Priority: Major > Time Spent: 4h 40m > Remaining Estimate: 0h > > # WHAT > 1. can deploy jar file by REST API > 2. from feature point , it will cover current 'gfsh deploy' > 3. if two files have same file name, the later wins > 4. can recognize version pattern. "filename-version[-label].jar" . > filename=[a-zA-Z/-]+, not single "-", not end with "-" > version=[0-9/.]*, not single ".", not end with "." > label=[a-zA-Z0-9]* > such as: > -is a later version of > , will deploy. > -is a same version of > , the later wins > -is a same version of > , the later wins > -is an earlier version of > , will block it. > 5. if there is a version part in the file name, we will deploy them > without append "#1" part to the file name > 6. can accept "group" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GEODE-7421) Ability: can deploy jar by REST API for Management
[ https://issues.apache.org/jira/browse/GEODE-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinmei Liao resolved GEODE-7421. Fix Version/s: 1.13.0 Resolution: Fixed > Ability: can deploy jar by REST API for Management > -- > > Key: GEODE-7421 > URL: https://issues.apache.org/jira/browse/GEODE-7421 > Project: Geode > Issue Type: New Feature > Components: management, rest (admin) >Reporter: Gang Yan >Priority: Major > Fix For: 1.13.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > # WHAT > 1. can deploy jar file by REST API > 2. from feature point , it will cover current 'gfsh deploy' > 3. if two files have same file name, the later wins > 4. can recognize version pattern. "filename-version[-label].jar" . > filename=[a-zA-Z/-]+, not single "-", not end with "-" > version=[0-9/.]*, not single ".", not end with "." > label=[a-zA-Z0-9]* > such as: > -is a later version of > , will deploy. > -is a same version of > , the later wins > -is a same version of > , the later wins > -is an earlier version of > , will block it. > 5. if there is a version part in the file name, we will deploy them > without append "#1" part to the file name > 6. can accept "group" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-7808) standardize on use of LocatorAddress/HostAddress for connection formation
[ https://issues.apache.org/jira/browse/GEODE-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050492#comment-17050492 ] ASF subversion and git services commented on GEODE-7808: Commit 4b06a7ae04e3eb3e32d8a6a96c7eb5e5d3269df0 in geode's branch refs/heads/develop from Bruce Schuchardt [ https://gitbox.apache.org/repos/asf?p=geode.git;h=4b06a7a ] Revert "GEODE-7808: standardize on use of HostAndPort to form client-side connections (#4743)" (#4761) This reverts commit 0af626462642c6352840cd6e81a5265c74045c7f. That commit seems to have caused a severe performance drop in several Benchmark tests: org.apache.geode.benchmark.tests.PartitionedGetBenchmark average ops/second Baseline:981794.46 Test: 41239.82 Difference: -95.8% org.apache.geode.benchmark.tests.ReplicatedGetBenchmark average ops/second Baseline:972769.18 Test: 41299.96 Difference: -95.8% org.apache.geode.benchmark.tests.PartitionedNonIndexedQueryBenchmark average ops/second Baseline:90.05 Test:70.52 Difference: -21.7% > standardize on use of LocatorAddress/HostAddress for connection formation > - > > Key: GEODE-7808 > URL: https://issues.apache.org/jira/browse/GEODE-7808 > Project: Geode > Issue Type: Improvement > Components: membership, messaging >Reporter: Bruce J Schuchardt >Assignee: Bruce J Schuchardt >Priority: Major > Fix For: 1.13.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > We currently use InetAddress and InetSocketAddress in many places to identify > locators, servers and peers. Some work has been done in the past couple of > years to reduce the use of these in order to accommodate changes in IP > addresses due to various causes. The class LocatorAddress was created to > help with this and it is able to hold a host name without resolving it until > that resolution is needed to form a tcp/ip connection. > These days we are seeing more and more movement into cloud computing and the > need to accommodate IP address changes is becoming a bigger issue. To that > end we would like to change our primary client/server and WAN communication > interfaces to stop taking InetAddresses and InetSocketAddresses as arguments > and, instead, take something like a LocatorAddress that can hold an > unresolved hostname that our communication implementations will resolve when > needed. > To that end we should also remove the hostname->inetaddress cache in > SocketCreator and rely on the operating system's DNS cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (GEODE-7841) Add benchmarking scripts for Geode-Redis module
Raymond Ingles created GEODE-7841: - Summary: Add benchmarking scripts for Geode-Redis module Key: GEODE-7841 URL: https://issues.apache.org/jira/browse/GEODE-7841 Project: Geode Issue Type: New Feature Components: benchmarks, redis Reporter: Raymond Ingles Fix For: 1.12.0 Add the following scripts to the Geode-Redis module, in the "performanceTest" directory: * aggregator.sh runs the actual tests * benchmark.sh handles validating the redis server (e.g. starting and stopping Geode Redis if needed), then runs aggregator.sh * shacompare.sh takes two git SHAs as arguments and runs benchmark.sh on each of them, generating separate output files for each. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GEODE-7841) Add benchmarking scripts for Geode-Redis module
[ https://issues.apache.org/jira/browse/GEODE-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Ingles updated GEODE-7841: -- Fix Version/s: (was: 1.12.0) > Add benchmarking scripts for Geode-Redis module > --- > > Key: GEODE-7841 > URL: https://issues.apache.org/jira/browse/GEODE-7841 > Project: Geode > Issue Type: New Feature > Components: benchmarks, redis >Reporter: Raymond Ingles >Priority: Major > > Add the following scripts to the Geode-Redis module, in the "performanceTest" > directory: > * aggregator.sh runs the actual tests > * benchmark.sh handles validating the redis server (e.g. starting and > stopping Geode Redis if needed), then runs aggregator.sh > * shacompare.sh takes two git SHAs as arguments and runs benchmark.sh on > each of them, generating separate output files for each. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (GEODE-7841) Add benchmarking scripts for Geode-Redis module
[ https://issues.apache.org/jira/browse/GEODE-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jens Deppe reassigned GEODE-7841: - Assignee: Jens Deppe > Add benchmarking scripts for Geode-Redis module > --- > > Key: GEODE-7841 > URL: https://issues.apache.org/jira/browse/GEODE-7841 > Project: Geode > Issue Type: New Feature > Components: benchmarks, redis >Reporter: Raymond Ingles >Assignee: Jens Deppe >Priority: Major > > Add the following scripts to the Geode-Redis module, in the "performanceTest" > directory: > * aggregator.sh runs the actual tests > * benchmark.sh handles validating the redis server (e.g. starting and > stopping Geode Redis if needed), then runs aggregator.sh > * shacompare.sh takes two git SHAs as arguments and runs benchmark.sh on > each of them, generating separate output files for each. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GEODE-7763) Apache Geode 1.11 severely and negatively impacts performance and resource utilization
[
https://issues.apache.org/jira/browse/GEODE-7763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050590#comment-17050590
]
Barrett Oglesby commented on GEODE-7763:
[~jblum]: Sorry about the confusion regarding this statement:
{noformat}
The second during the put in the SessionEventHandlerCacheListenerAdapter here:
{noformat}
I didn't mean the put in the adapter (which is what I said :)). I mean the put
operation that caused the adapter to be invoked. I wasn't blaming the adapter
in any way.
Sorry for the confusion.
> Apache Geode 1.11 severely and negatively impacts performance and resource
> utilization
> --
>
> Key: GEODE-7763
> URL: https://issues.apache.org/jira/browse/GEODE-7763
> Project: Geode
> Issue Type: Bug
> Components: client/server
>Affects Versions: 1.10.0, 1.11.0
>Reporter: John Blum
>Priority: Critical
> Labels: performance
> Attachments: 1.11-client-stats.gfs, 1.11-server-stats.gfs,
> 1.11_thread_dumps.rtf, 1.9-client-stats.gfs, 1.9-server-stats.gfs, 1.9.log,
> apache-geode-1.10-client-server-interaction-output.txt,
> apache-geode-1.10-client-server-startup-output.txt,
> apache-geode-1.11-client-server-interaction-output.txt,
> apache-geode-1.11-client-server-startup-output.txt,
> geode-7763-geode-changes.diff, geode-7763-ssdg-changes.diff
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This problem was first observed in Apache Geode 1.11.0. The problem was not
> present in Apache Geode 1.9.2. This problem is an issue for Apache Geode
> 1.10 as well!
> After upgrading _Spring Session for Apache Geode_ (SSDG) 2.3 to _Spring Data
> for Apache Geode_ (SDG) Neumann/2.3, which is based on Apache Geode 1.11,
> this problem with SSDG's test suite started occurring.
> _Spring Session for Apache Geode_ (SSDG) 2.2, which is based on _Spring Data
> for Apache Geode_ (SDG) Moore/2.2, pulls in Apache Geode 1.9.2. This problem
> did not occur in SSDG 2.2. with Apache Geode 1.9.2.
> Out of curiosity, I wondered whether this problem affects (i.e. was actually
> introduced in) Apache Geode 1.10.0. So, I configured SSDG 2.3 to pull in SDG
> Moore/2.2 but run with Apache Geode 1.10. The problem occurred with Apache
> Geode 1.10 as well!
> The SSDG test class in question, affected by Geode's deficiencies, is the
> [MultiThreadedHighlyConcurrentClientServerSessionOperationsIntegrationTests|https://github.com/spring-projects/spring-session-data-geode/blob/2.2.2.RELEASE/spring-session-data-geode/src/integration-test/java/org/springframework/session/data/gemfire/MultiThreadedHighlyConcurrentClientServerHttpSessionAccessIntegrationTests.java].
> The test class was modeled after a customer UC, who were using Spring Session
> and Apache Geode/Pivotal GemFire as the HTTP Session state management
> provider, therefore it simulates their highly concurrent environment.
> The test class has 2 primary parameters: [Thread
> Count|https://github.com/spring-projects/spring-session-data-geode/blob/2.2.2.RELEASE/spring-session-data-geode/src/integration-test/java/org/springframework/session/data/gemfire/MultiThreadedHighlyConcurrentClientServerHttpSessionAccessIntegrationTests.java#L90]
> and the [Workload
> Size|https://github.com/spring-projects/spring-session-data-geode/blob/2.2.2.RELEASE/spring-session-data-geode/src/integration-test/java/org/springframework/session/data/gemfire/MultiThreadedHighlyConcurrentClientServerHttpSessionAccessIntegrationTests.java#L91].
> The "_Workload Size_" should not be confused with the "_Payload Size_" of the
> individual objects passed to the Geode data access operations (i.e. {{gets}},
> {{puts}}, {{removes}}). The "_Workload Size_" merely determines the number
> of {{get}}, {{put}} or {{remove}} operations performed on the (Session)
> Region over the duration of the test run. Certain operations are "favored"
> over others, therefore the number of {{gets}}, {{puts}} and {{removes}} is
> weighted.
> The "_Payload_" in this case is a (HTTP) {{Session}} object and the "size" is
> directly proportional to the number of Session attributes stored in the
> Session.
> As you can see from the [test class
> configuration|https://github.com/spring-projects/spring-session-data-geode/blob/2.2.2.RELEASE/spring-session-data-geode/src/integration-test/java/org/springframework/session/data/gemfire/MultiThreadedHighlyConcurrentClientServerHttpSessionAccessIntegrationTests.java#L90-L91]
> in *SSDG* {{2.2}}, the *Thread Count* was set to *180* and the *Workload
> Size* (or number of Region operations) was set to *10,000*.
> This had to be significantly adjusted in SSDG 2.3 using Apache Geode 1.11
> (and, as it turns out, Apache Geode 1.10 as well), as can be seen in t
[jira] [Comment Edited] (GEODE-7763) Apache Geode 1.11 severely and negatively impacts performance and resource utilization
[
https://issues.apache.org/jira/browse/GEODE-7763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050590#comment-17050590
]
Barrett Oglesby edited comment on GEODE-7763 at 3/3/20 9:45 PM:
[~jblum]: Sorry about the confusion regarding this statement:
{noformat}
The second during the put in the SessionEventHandlerCacheListenerAdapter here:
{noformat}
I didn't mean the put in the adapter (which is what I said :)). I meant the put
operation that caused the adapter to be invoked. I wasn't blaming the adapter
in any way.
Sorry for the confusion.
was (Author: barry.oglesby):
[~jblum]: Sorry about the confusion regarding this statement:
{noformat}
The second during the put in the SessionEventHandlerCacheListenerAdapter here:
{noformat}
I didn't mean the put in the adapter (which is what I said :)). I mean the put
operation that caused the adapter to be invoked. I wasn't blaming the adapter
in any way.
Sorry for the confusion.
> Apache Geode 1.11 severely and negatively impacts performance and resource
> utilization
> --
>
> Key: GEODE-7763
> URL: https://issues.apache.org/jira/browse/GEODE-7763
> Project: Geode
> Issue Type: Bug
> Components: client/server
>Affects Versions: 1.10.0, 1.11.0
>Reporter: John Blum
>Priority: Critical
> Labels: performance
> Attachments: 1.11-client-stats.gfs, 1.11-server-stats.gfs,
> 1.11_thread_dumps.rtf, 1.9-client-stats.gfs, 1.9-server-stats.gfs, 1.9.log,
> apache-geode-1.10-client-server-interaction-output.txt,
> apache-geode-1.10-client-server-startup-output.txt,
> apache-geode-1.11-client-server-interaction-output.txt,
> apache-geode-1.11-client-server-startup-output.txt,
> geode-7763-geode-changes.diff, geode-7763-ssdg-changes.diff
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This problem was first observed in Apache Geode 1.11.0. The problem was not
> present in Apache Geode 1.9.2. This problem is an issue for Apache Geode
> 1.10 as well!
> After upgrading _Spring Session for Apache Geode_ (SSDG) 2.3 to _Spring Data
> for Apache Geode_ (SDG) Neumann/2.3, which is based on Apache Geode 1.11,
> this problem with SSDG's test suite started occurring.
> _Spring Session for Apache Geode_ (SSDG) 2.2, which is based on _Spring Data
> for Apache Geode_ (SDG) Moore/2.2, pulls in Apache Geode 1.9.2. This problem
> did not occur in SSDG 2.2. with Apache Geode 1.9.2.
> Out of curiosity, I wondered whether this problem affects (i.e. was actually
> introduced in) Apache Geode 1.10.0. So, I configured SSDG 2.3 to pull in SDG
> Moore/2.2 but run with Apache Geode 1.10. The problem occurred with Apache
> Geode 1.10 as well!
> The SSDG test class in question, affected by Geode's deficiencies, is the
> [MultiThreadedHighlyConcurrentClientServerSessionOperationsIntegrationTests|https://github.com/spring-projects/spring-session-data-geode/blob/2.2.2.RELEASE/spring-session-data-geode/src/integration-test/java/org/springframework/session/data/gemfire/MultiThreadedHighlyConcurrentClientServerHttpSessionAccessIntegrationTests.java].
> The test class was modeled after a customer UC, who were using Spring Session
> and Apache Geode/Pivotal GemFire as the HTTP Session state management
> provider, therefore it simulates their highly concurrent environment.
> The test class has 2 primary parameters: [Thread
> Count|https://github.com/spring-projects/spring-session-data-geode/blob/2.2.2.RELEASE/spring-session-data-geode/src/integration-test/java/org/springframework/session/data/gemfire/MultiThreadedHighlyConcurrentClientServerHttpSessionAccessIntegrationTests.java#L90]
> and the [Workload
> Size|https://github.com/spring-projects/spring-session-data-geode/blob/2.2.2.RELEASE/spring-session-data-geode/src/integration-test/java/org/springframework/session/data/gemfire/MultiThreadedHighlyConcurrentClientServerHttpSessionAccessIntegrationTests.java#L91].
> The "_Workload Size_" should not be confused with the "_Payload Size_" of the
> individual objects passed to the Geode data access operations (i.e. {{gets}},
> {{puts}}, {{removes}}). The "_Workload Size_" merely determines the number
> of {{get}}, {{put}} or {{remove}} operations performed on the (Session)
> Region over the duration of the test run. Certain operations are "favored"
> over others, therefore the number of {{gets}}, {{puts}} and {{removes}} is
> weighted.
> The "_Payload_" in this case is a (HTTP) {{Session}} object and the "size" is
> directly proportional to the number of Session attributes stored in the
> Session.
> As you can see from the [test class
> configuration|https://github.com/spring-projects/spring-session-data-geode/blob/2.2.2.RELEASE/spring-session-data-geode/src/i
[jira] [Created] (GEODE-7842) Improve concurrent region creation
Jens Deppe created GEODE-7842:
-
Summary: Improve concurrent region creation
Key: GEODE-7842
URL: https://issues.apache.org/jira/browse/GEODE-7842
Project: Geode
Issue Type: Bug
Components: gfsh
Reporter: Jens Deppe
The Redis adapter has some tests that result in regions being attempted to be
created concurrently. The region create call uses {{ifNotExists=true}} but
occasionally I still see {{RegionCreationException}}s as multiple threads are
in the {{RegionCreateFunction}} concurrently trying to create the same region.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (GEODE-7842) Improve concurrent region creation
[
https://issues.apache.org/jira/browse/GEODE-7842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jens Deppe reassigned GEODE-7842:
-
Assignee: Jens Deppe
> Improve concurrent region creation
> --
>
> Key: GEODE-7842
> URL: https://issues.apache.org/jira/browse/GEODE-7842
> Project: Geode
> Issue Type: Bug
> Components: gfsh
>Reporter: Jens Deppe
>Assignee: Jens Deppe
>Priority: Major
>
> The Redis adapter has some tests that result in regions being attempted to be
> created concurrently. The region create call uses {{ifNotExists=true}} but
> occasionally I still see {{RegionCreationException}}s as multiple threads are
> in the {{RegionCreateFunction}} concurrently trying to create the same region.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-7828) Convert backing store for Redis Hashes and Sets to single regions
[
https://issues.apache.org/jira/browse/GEODE-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050622#comment-17050622
]
ASF subversion and git services commented on GEODE-7828:
Commit 2f6bf013368df5a4b5efe68162a4953f9a88bbf2 in geode's branch
refs/heads/develop from Jens Deppe
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=2f6bf01 ]
GEODE-7828: Convert backing store for Redis Hashes and Sets to single regions
(#4745)
* GEODE-7828: Convert backing store for Redis Hashes and Sets to single regions
This PR builds on the work originally submitted by Greg Green in
https://github.com/apache/geode/pull/404. Also acknowledgements to Galen
O'Sullivan for addressing locking issues in that PR.
This commit changes the storage model of Redis hashes and sets from one
region per each Redis key to a single hash region and single set region.
The Redis key is now also the region key and the data is stored in a Map
and Set respectively in the region. Currently, the backing values do not
implement Delta changes, however this will be a future optimization.
This also fixes the inability of Redis keys to contain other characters
commonly used, such as colons (':').
- Add `RedisLockService` which manages a lock per key within a single
JVM. Locks are held in a WeakHasMap to allow for automatic cleanup
(prior PR work, using a pure ConcurrentHashMap, ended up leaking
memory since there is no straight-forward way to clean up unused
keys/locks).
- Add new tests including concurrency tests for hashes and sets
Co-authored-by: Greg Green
Co-authored-by: Ray Ingles
Co-authored-by: Sarah Abbey
Co-authored-by: John Hutchison
Co-authored-by: Murtuza Boxwala
Co-authored-by: Prasath Durairaj
Co-authored-by: Jens Deppe
> Convert backing store for Redis Hashes and Sets to single regions
> -
>
> Key: GEODE-7828
> URL: https://issues.apache.org/jira/browse/GEODE-7828
> Project: Geode
> Issue Type: Improvement
> Components: redis
>Reporter: Jens Deppe
>Assignee: Jens Deppe
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently the Redis adapter creates separate regions for each new hash or
> set. This can be a significant performance impact when many hashes are being
> created. In addition, hashes are conventionally often named with colons (':')
> in their name - for example {{user:1000}}. This does not work as region names
> cannot contain colons.
> The work here will create a single region for hashes and a single region for
> sets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (GEODE-7828) Convert backing store for Redis Hashes and Sets to single regions
[
https://issues.apache.org/jira/browse/GEODE-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050621#comment-17050621
]
ASF subversion and git services commented on GEODE-7828:
Commit 2f6bf013368df5a4b5efe68162a4953f9a88bbf2 in geode's branch
refs/heads/develop from Jens Deppe
[ https://gitbox.apache.org/repos/asf?p=geode.git;h=2f6bf01 ]
GEODE-7828: Convert backing store for Redis Hashes and Sets to single regions
(#4745)
* GEODE-7828: Convert backing store for Redis Hashes and Sets to single regions
This PR builds on the work originally submitted by Greg Green in
https://github.com/apache/geode/pull/404. Also acknowledgements to Galen
O'Sullivan for addressing locking issues in that PR.
This commit changes the storage model of Redis hashes and sets from one
region per each Redis key to a single hash region and single set region.
The Redis key is now also the region key and the data is stored in a Map
and Set respectively in the region. Currently, the backing values do not
implement Delta changes, however this will be a future optimization.
This also fixes the inability of Redis keys to contain other characters
commonly used, such as colons (':').
- Add `RedisLockService` which manages a lock per key within a single
JVM. Locks are held in a WeakHasMap to allow for automatic cleanup
(prior PR work, using a pure ConcurrentHashMap, ended up leaking
memory since there is no straight-forward way to clean up unused
keys/locks).
- Add new tests including concurrency tests for hashes and sets
Co-authored-by: Greg Green
Co-authored-by: Ray Ingles
Co-authored-by: Sarah Abbey
Co-authored-by: John Hutchison
Co-authored-by: Murtuza Boxwala
Co-authored-by: Prasath Durairaj
Co-authored-by: Jens Deppe
> Convert backing store for Redis Hashes and Sets to single regions
> -
>
> Key: GEODE-7828
> URL: https://issues.apache.org/jira/browse/GEODE-7828
> Project: Geode
> Issue Type: Improvement
> Components: redis
>Reporter: Jens Deppe
>Assignee: Jens Deppe
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently the Redis adapter creates separate regions for each new hash or
> set. This can be a significant performance impact when many hashes are being
> created. In addition, hashes are conventionally often named with colons (':')
> in their name - for example {{user:1000}}. This does not work as region names
> cannot contain colons.
> The work here will create a single region for hashes and a single region for
> sets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (GEODE-7828) Convert backing store for Redis Hashes and Sets to single regions
[
https://issues.apache.org/jira/browse/GEODE-7828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jens Deppe resolved GEODE-7828.
---
Fix Version/s: 1.13.0
Resolution: Fixed
> Convert backing store for Redis Hashes and Sets to single regions
> -
>
> Key: GEODE-7828
> URL: https://issues.apache.org/jira/browse/GEODE-7828
> Project: Geode
> Issue Type: Improvement
> Components: redis
>Reporter: Jens Deppe
>Assignee: Jens Deppe
>Priority: Major
> Fix For: 1.13.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently the Redis adapter creates separate regions for each new hash or
> set. This can be a significant performance impact when many hashes are being
> created. In addition, hashes are conventionally often named with colons (':')
> in their name - for example {{user:1000}}. This does not work as region names
> cannot contain colons.
> The work here will create a single region for hashes and a single region for
> sets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
