[jira] [Commented] (GEODE-10371) C++ Native client: Improve dispersion on connections expiration
[
https://issues.apache.org/jira/browse/GEODE-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556943#comment-17556943
]
ASF GitHub Bot commented on GEODE-10371:
gaussianrecurrence commented on code in PR #974:
URL: https://github.com/apache/geode-native/pull/974#discussion_r902712371
##
cppcache/src/TcrConnection.cpp:
##
@@ -63,23 +63,10 @@ bool useReplyTimeout(const
apache::geode::client::TcrMessage& request) {
}
int expiryTimeVariancePercentage() {
- auto nowTimePoint = std::chrono::steady_clock::now().time_since_epoch();
- auto now_ms =
- std::chrono::duration_cast(nowTimePoint)
- .count();
- auto now_s =
- std::chrono::duration_cast(nowTimePoint).count();
-
- srand(static_cast((now_s * 1000) + (now_ms / 1000)));
-
- const int numbers = 21;
- // NOLINTNEXTLINE(clang-analyzer-security.insecureAPI.rand): TODO replace
- int random = rand() % numbers + 1;
-
- if (random > 10) {
-random = random - numbers;
- }
- return random;
+ std::random_device rd;
+ std::default_random_engine generator(rd());
+ std::uniform_int_distribution distribution(-9, 9);
Review Comment:
It'd be good to add a couple of constants to add some semantic meaning to
the distribution limits
> C++ Native client: Improve dispersion on connections expiration
> ---
>
> Key: GEODE-10371
> URL: https://issues.apache.org/jira/browse/GEODE-10371
> Project: Geode
> Issue Type: Improvement
> Components: native client
>Reporter: Alberto Gomez
>Assignee: Alberto Gomez
>Priority: Major
>
> The dispersion on connections expirations in the C++ native client works in
> such a way that it adds a dispersion (variance) between -9% and 9% over the
> time for a connection to expire due to load-conditioning so that, in the
> event of having many connections being created at the same, they do not
> expire at the right exact time.
> Nevertheless, the current implementation has two problems:
> - The randomness of the variance depends on the current time in seconds. As a
> result, for connections created in the same second, the variance will be the
> same and, therefore, the expiration time too.
> - The randomness is created using the C standard's library "rand()" function
> which is considered not secure.
> It is recommended to change the library used to generate the random variance
> to a secure one and also to make sure that for the time in seconds it does
> not return the same variance.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (GEODE-10371) C++ Native client: Improve dispersion on connections expiration
[ https://issues.apache.org/jira/browse/GEODE-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated GEODE-10371: --- Labels: pull-request-available (was: ) > C++ Native client: Improve dispersion on connections expiration > --- > > Key: GEODE-10371 > URL: https://issues.apache.org/jira/browse/GEODE-10371 > Project: Geode > Issue Type: Improvement > Components: native client >Reporter: Alberto Gomez >Assignee: Alberto Gomez >Priority: Major > Labels: pull-request-available > > The dispersion on connections expirations in the C++ native client works in > such a way that it adds a dispersion (variance) between -9% and 9% over the > time for a connection to expire due to load-conditioning so that, in the > event of having many connections being created at the same, they do not > expire at the right exact time. > Nevertheless, the current implementation has two problems: > - The randomness of the variance depends on the current time in seconds. As a > result, for connections created in the same second, the variance will be the > same and, therefore, the expiration time too. > - The randomness is created using the C standard's library "rand()" function > which is considered not secure. > It is recommended to change the library used to generate the random variance > to a secure one and also to make sure that for the time in seconds it does > not return the same variance. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (GEODE-10371) C++ Native client: Improve dispersion on connections expiration
[
https://issues.apache.org/jira/browse/GEODE-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556950#comment-17556950
]
ASF GitHub Bot commented on GEODE-10371:
gaussianrecurrence commented on code in PR #974:
URL: https://github.com/apache/geode-native/pull/974#discussion_r902718227
##
cppcache/test/TcrConnectionTest.cpp:
##
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include
+#include
+#include
+
+#include
+
+namespace {
+
+using apache::geode::client::CacheImpl;
+using apache::geode::client::TcrConnection;
+using apache::geode::client::TcrConnectionManager;
+
+class TcrConnectionTest : public TcrConnection {
+ public:
+ explicit TcrConnectionTest(const TcrConnectionManager& manager)
+ : TcrConnection(manager) {}
+
+ int getExpiryTimeVariancePercentage() {
+return expiryTimeVariancePercentage_;
+ }
+};
+
+TEST(TcrConnectionTest,
+ getExpiryTimeVariancePercentageReturnsRandomBetweenMinusNineAndNine) {
+ CacheImpl* cache = nullptr;
+
+ // Create several connections at the same time
+ const int connections = 100;
+ std::unique_ptr tcrConnections[connections];
+ for (int i = 0; i < connections; i++) {
+tcrConnections[i] =
+std::unique_ptr(new TcrConnectionTest(
+static_cast(nullptr)));
+ }
+
Review Comment:
This is a suggestion for a change to this case or maybe for another case,
but given the goal of this change is to ensure the variance is evenly
distributed along the connections created within a close period of time, I'd
suggest instead of comparing with the first value, I'd rather sample the
distribution and check no bucket in the sample has more than x times (i.e
double) the expected amount.
> C++ Native client: Improve dispersion on connections expiration
> ---
>
> Key: GEODE-10371
> URL: https://issues.apache.org/jira/browse/GEODE-10371
> Project: Geode
> Issue Type: Improvement
> Components: native client
>Reporter: Alberto Gomez
>Assignee: Alberto Gomez
>Priority: Major
> Labels: pull-request-available
>
> The dispersion on connections expirations in the C++ native client works in
> such a way that it adds a dispersion (variance) between -9% and 9% over the
> time for a connection to expire due to load-conditioning so that, in the
> event of having many connections being created at the same, they do not
> expire at the right exact time.
> Nevertheless, the current implementation has two problems:
> - The randomness of the variance depends on the current time in seconds. As a
> result, for connections created in the same second, the variance will be the
> same and, therefore, the expiration time too.
> - The randomness is created using the C standard's library "rand()" function
> which is considered not secure.
> It is recommended to change the library used to generate the random variance
> to a secure one and also to make sure that for the time in seconds it does
> not return the same variance.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (GEODE-10363) Add geode-deployment-legacy.jar to list of jar files to copy
[ https://issues.apache.org/jira/browse/GEODE-10363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated GEODE-10363: --- Labels: pull-request-available (was: ) > Add geode-deployment-legacy.jar to list of jar files to copy > > > Key: GEODE-10363 > URL: https://issues.apache.org/jira/browse/GEODE-10363 > Project: Geode > Issue Type: Improvement >Affects Versions: 1.15.0 >Reporter: Max Hufnagel >Assignee: Max Hufnagel >Priority: Major > Labels: pull-request-available > > Add the geode-deployment-legacy.jar to the list of jars on these pages in > 1.15: > * > [https://geode.apache.org/docs/guide/114/tools_modules/http_session_mgmt/tomcat_installing_the_module.html] > * > [https://geode.apache.org/docs/guide/114/tools_modules/http_session_mgmt/weblogic_setting_up_the_module.html] -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (GEODE-10391) Region Operation During Primary Change in P2P-only Configuration Results in Spurious Entry{NotFound|Exists}Exception
[
https://issues.apache.org/jira/browse/GEODE-10391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated GEODE-10391:
---
Labels: pull-request-available (was: )
> Region Operation During Primary Change in P2P-only Configuration Results in
> Spurious Entry{NotFound|Exists}Exception
>
>
> Key: GEODE-10391
> URL: https://issues.apache.org/jira/browse/GEODE-10391
> Project: Geode
> Issue Type: Bug
> Components: regions
>Affects Versions: 1.16.0
>Reporter: Bill Burcham
>Assignee: Bill Burcham
>Priority: Major
> Labels: pull-request-available
>
> When a primary moves while a region operation, e.g. create, is in-flight,
> i.e. started but not yet acknowledged, the operation will be retried
> automatically, until the operation succeeds or fails.
> When a member notices another member has crashed, the surviving member
> requests (from the remaining members) data for which the crashed member had
> been primary (delta-GII/sync). This sync is necessary to regain consistency
> in case the (retrying) requester fails before it can re-issue the request to
> the new primary.
> In GEODE-5055 we learned that we needed to delay that sync request long
> enough for the new primary to be chosen and for the original requester to
> make a new request against the new primary. If we didn't delay the sync, the
> primary could end up with the entry in the new state (as if the operation had
> completed) but without the corresponding event tracker data needed to
> conflate the retried event.
> The fix for GEODE-5055 introduced a delay, but only for configurations where
> clients were present. If only peers were present there would be no delay.
> This ticket pertains to the P2P-only case.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (GEODE-9175) Clean up the terminal output and log progress for benchmarks
[ https://issues.apache.org/jira/browse/GEODE-9175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17557135#comment-17557135 ] ASF GitHub Bot commented on GEODE-9175: --- upthewaterspout closed pull request #147: GEODE-9175: Logging progress of benchmarks as they are running URL: https://github.com/apache/geode-benchmarks/pull/147 > Clean up the terminal output and log progress for benchmarks > > > Key: GEODE-9175 > URL: https://issues.apache.org/jira/browse/GEODE-9175 > Project: Geode > Issue Type: Improvement > Components: benchmarks >Reporter: Dan Smith >Assignee: Dan Smith >Priority: Major > Labels: pull-request-available > > When developing a new benchmark with geode-benchmarks or testing out code > changes interactively, it would be nice to for the geode-benchmarks to log > the current throughput and latency numbers as the test is running, similar to > the way YCSB does. > This lets someone writing a new benchmark quickly eyeball if the performance > has changed, or if their warm up time is too short or too long because the > throughput fluctuates. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Resolved] (GEODE-10366) Using GfshRule default constructor leaves test directories around
[ https://issues.apache.org/jira/browse/GEODE-10366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk Lund resolved GEODE-10366. --- Fix Version/s: 1.15.0 1.16.0 Resolution: Fixed > Using GfshRule default constructor leaves test directories around > - > > Key: GEODE-10366 > URL: https://issues.apache.org/jira/browse/GEODE-10366 > Project: Geode > Issue Type: Bug > Components: tests >Affects Versions: 1.15.0, 1.16.0 >Reporter: Kirk Lund >Assignee: Kirk Lund >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0, 1.16.0 > > > Using GfshRule default constructor leaves test directories around. It's > supposed to automatically delete the test directories after the test > completes. This is not a blocker for 1.15 but the fix should be merged to > develop and possibly support/1.15 unless it's frozen for release. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Updated] (GEODE-10394) [CI Failure] : ConnectionManagerJUnitTest > testExclusiveConnectionAccess FAILED
[
https://issues.apache.org/jira/browse/GEODE-10394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nabarun Nag updated GEODE-10394:
Issue Type: Bug (was: Improvement)
> [CI Failure] : ConnectionManagerJUnitTest > testExclusiveConnectionAccess
> FAILED
>
>
> Key: GEODE-10394
> URL: https://issues.apache.org/jira/browse/GEODE-10394
> Project: Geode
> Issue Type: Bug
>Affects Versions: 1.16.0
>Reporter: Nabarun Nag
>Priority: Major
>
> {noformat}
> > Task :geode-core:integrationTest
> ConnectionManagerJUnitTest > testExclusiveConnectionAccess FAILED
> java.util.concurrent.ExecutionException: java.lang.RuntimeException:
> org.apache.geode.cache.client.AllConnectionsInUseException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:206)
> at
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest.testExclusiveConnectionAccess(ConnectionManagerJUnitTest.java:539)
> Caused by:
> java.lang.RuntimeException:
> org.apache.geode.cache.client.AllConnectionsInUseException
> at
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.borrow(ConnectionManagerJUnitTest.java:813)
> at
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.run(ConnectionManagerJUnitTest.java:795)
> at
> org.apache.geode.test.junit.rules.ExecutorServiceRule.lambda$submit$3(ExecutorServiceRule.java:287)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Caused by:
> org.apache.geode.cache.client.AllConnectionsInUseException
> at
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl.borrowConnection(ConnectionManagerImpl.java:310)
> at
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.lambda$doBorrow$2(ConnectionManagerJUnitTest.java:821)
> at
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$Timer.measure(ConnectionManagerJUnitTest.java:769)
> at
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.doBorrow(ConnectionManagerJUnitTest.java:820)
> at
> org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.borrow(ConnectionManagerJUnitTest.java:811)
> ... 8 more{noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (GEODE-10394) [CI Failure] : ConnectionManagerJUnitTest > testExclusiveConnectionAccess FAILED
Nabarun Nag created GEODE-10394:
---
Summary: [CI Failure] : ConnectionManagerJUnitTest >
testExclusiveConnectionAccess FAILED
Key: GEODE-10394
URL: https://issues.apache.org/jira/browse/GEODE-10394
Project: Geode
Issue Type: Improvement
Affects Versions: 1.16.0
Reporter: Nabarun Nag
{noformat}
> Task :geode-core:integrationTest
ConnectionManagerJUnitTest > testExclusiveConnectionAccess FAILED
java.util.concurrent.ExecutionException: java.lang.RuntimeException:
org.apache.geode.cache.client.AllConnectionsInUseException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at
org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest.testExclusiveConnectionAccess(ConnectionManagerJUnitTest.java:539)
Caused by:
java.lang.RuntimeException:
org.apache.geode.cache.client.AllConnectionsInUseException
at
org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.borrow(ConnectionManagerJUnitTest.java:813)
at
org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.run(ConnectionManagerJUnitTest.java:795)
at
org.apache.geode.test.junit.rules.ExecutorServiceRule.lambda$submit$3(ExecutorServiceRule.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by:
org.apache.geode.cache.client.AllConnectionsInUseException
at
org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl.borrowConnection(ConnectionManagerImpl.java:310)
at
org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.lambda$doBorrow$2(ConnectionManagerJUnitTest.java:821)
at
org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$Timer.measure(ConnectionManagerJUnitTest.java:769)
at
org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.doBorrow(ConnectionManagerJUnitTest.java:820)
at
org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$UpdaterThread.borrow(ConnectionManagerJUnitTest.java:811)
... 8 more{noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
