Re: Hints handoff is memory intensive
Major changes in 3.0, but the big ones involve storage - parent ticket is https://issues.apache.org/jira/browse/CASSANDRA-9427 64G cms heap with 4G new gen is somewhat atypical - would expect g1 at that size, or at the very least a larger new gen to try to avoid promotion which will bring you that parnew pause. -- Jeff Jirsa > On Sep 19, 2016, at 11:12 PM, Dikang Gu wrote: > > In our 2.1 cluster, I find that hints handoff is using a lot of memory on > our proxy nodes, when delivering hints to a data node that was dead for 3+ > hours (our hints window is 3 hours). It makes the young gen GC time as long > as 2 secs. > > I'm using 64G max heap size, and 4G young gen size. I'm considering > increasing the young gen size, I'm wondering any improvements to hints > handoff in newer version? > > some logs: > 2016-09-20_05:04:30.67928 INFO 05:04:30 [Service Thread]: ParNew GC in > 2200ms. CMS Old Gen: 49112623080 -> 50052787792; Par Eden Space: > 2863398912 -> 0; > > mem usage: > 2016-09-20T01:25:23.522+ Process summary > process cpu=0.00% > application cpu=140.36% (user=132.28% sys=8.08%) > other: cpu=-140.36% > heap allocation rate 1500mb/s > [001343] user=90.42% sys= 0.01% alloc= 1439mb/s - HintedHandoff:3 > [290234] user=18.85% sys= 5.14% alloc= 40mb/s - CompactionExecutor:176226 > [064445] user=17.57% sys= 1.09% alloc= 10mb/s - RMI TCP > Connection(148376)-127.0.0.1 > [290439] user= 3.20% sys= 1.10% alloc= 4607kb/s - RMI TCP > Connection(148380)-127.0.0.1 > [16] user= 0.75% sys=-0.03% alloc= 2321kb/s - ScheduledTasks:1 > [000149] user= 0.32% sys=-0.04% alloc= 1516kb/s - GossipStage:1 > [001370] user= 0.11% sys= 0.04% alloc= 420kb/s - GossipTasks:1 > > Thanks > Dikang.
R: Re: Hints handoff is memory intensive
+1 for G1 with such a heap size. And don't set heap new size, let g1 decide. Inviato da Yahoo Mail su Android Il mar, 20 set, 2016 alle 9:01, Jeff Jirsa ha scritto: Major changes in 3.0, but the big ones involve storage - parent ticket is https://issues.apache.org/jira/browse/CASSANDRA-9427 64G cms heap with 4G new gen is somewhat atypical - would expect g1 at that size, or at the very least a larger new gen to try to avoid promotion which will bring you that parnew pause. -- Jeff Jirsa > On Sep 19, 2016, at 11:12 PM, Dikang Gu wrote: > > In our 2.1 cluster, I find that hints handoff is using a lot of memory on > our proxy nodes, when delivering hints to a data node that was dead for 3+ > hours (our hints window is 3 hours). It makes the young gen GC time as long > as 2 secs. > > I'm using 64G max heap size, and 4G young gen size. I'm considering > increasing the young gen size, I'm wondering any improvements to hints > handoff in newer version? > > some logs: > 2016-09-20_05:04:30.67928 INFO 05:04:30 [Service Thread]: ParNew GC in > 2200ms. CMS Old Gen: 49112623080 -> 50052787792; Par Eden Space: > 2863398912 -> 0; > > mem usage: > 2016-09-20T01:25:23.522+ Process summary > process cpu=0.00% > application cpu=140.36% (user=132.28% sys=8.08%) > other: cpu=-140.36% > heap allocation rate 1500mb/s > [001343] user=90.42% sys= 0.01% alloc= 1439mb/s - HintedHandoff:3 > [290234] user=18.85% sys= 5.14% alloc= 40mb/s - CompactionExecutor:176226 > [064445] user=17.57% sys= 1.09% alloc= 10mb/s - RMI TCP > Connection(148376)-127.0.0.1 > [290439] user= 3.20% sys= 1.10% alloc= 4607kb/s - RMI TCP > Connection(148380)-127.0.0.1 > [16] user= 0.75% sys=-0.03% alloc= 2321kb/s - ScheduledTasks:1 > [000149] user= 0.32% sys=-0.04% alloc= 1516kb/s - GossipStage:1 > [001370] user= 0.11% sys= 0.04% alloc= 420kb/s - GossipTasks:1 > > Thanks > Dikang.
[RELEASE] Apache Cassandra 3.0.9 released
The Cassandra team is pleased to announce the release of Apache Cassandra version 3.0.9. Apache Cassandra is a fully distributed database. It is the right choice when you need scalability and high availability without compromising performance. http://cassandra.apache.org/ Downloads of source and binary distributions are listed in our download section: http://cassandra.apache.org/download/ This version is a bug fix release[1] on the 3.0 series. As always, please pay attention to the release notes[2] and Let us know[3] if you were to encounter any problem. Enjoy! [1]: https://goo.gl/YfvFn8 (CHANGES.txt) [2]: https://goo.gl/k9leqx (NEWS.txt) [3]: https://issues.apache.org/jira/browse/CASSANDRA
Re: Proposal - 3.5.1
@Sylvain - I see what you're saying now on the branches. I suppose a branching strategy like that does give some flexibility to have multiple things in the pipeline so it does give some additional flexibility there. On Mon, Sep 19, 2016 at 9:06 AM Eric Evans wrote: > On Fri, Sep 16, 2016 at 5:05 AM, Sylvain Lebresne > wrote: > > In light of all this, my suggesting for a release cycle woud be: > > - To have 3 branches: 'features', 'testing' and 'stable', with an X month > > rotation: 'features' becomes 'testing' after X months and then 'stable' > > after > > X more, before getting EOL X months later. > > - The feature branch gets everything. The testing branch only gets bug > > fixes. > > The stable branch only gets critical bug fixes. And imo, we should be > very > > strict on this (I acknowledge that there is sometimes a bit of > > subjectivity on > > whether something is a bug or an improvement, and if it's critical or > > not, but > > I think it's not that hard to get consensus if we're all reasonable > > (though it > > might worth agreeing on some rough but written guideline upfront)). > > - We release on a short and fixed cadence of Y month(s) for both the > > feature and > > testing branch. For the stable branch, given that it already had X > months > > of > > only bug fixes during the testing phase, one can hope critical fixes > will > > be > > fairly rare, less than 1 per Y period on average). Further, it's > supposed > > to > > be stable and fixes are supposed to be critical, so doing hot-fix > releases > > probably makes the most sense (though it probably only work if we're > > indeed > > strict on what is considered critical). > > This seems pretty close to what Mck suggested; I think this could work. > > > -- > Eric Evans > john.eric.ev...@gmail.com >
Failing tests 2016-09-20
Hi All, Joel asked me to take this over for him, and so here I am. cassandra-3.9: === testall: 5 failures org.apache.cassandra.cql3.ViewFilteringTest .testMVCreationSelectRestrictions org.apache.cassandra.cql3.ViewFilteringTest .testClusteringKeyFilteringRestrictions org.apache.cassandra.cql3.ViewTest .ttlExpirationTest org.apache.cassandra.cql3.validation.entities .UFTest.testDropKeyspaceContainingFunctionDropsPreparedStatementsWithDelayedValues org.apache.cassandra.cql3.validation.entities .UFTest.testEmptyString All five test failures are due to timeouts. === dtest: All passed! === upgrade: 1 failure upgrade_tests.cql_tests .TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_x.bug_5732_test New flaky failure. No JIRA created yet. === novnode: All passed! === trunk: === testall: 12 failures I'm going to leave out the 10 timeouts for my sake. org.apache.cassandra.config.DatabaseDescriptorRefTest .testDatabaseDescriptorRef org.apache.cassandra.config.DatabaseDescriptorRefTest .testDatabaseDescriptorRef-compression CASSANDRA-12677 for the two failures above. New failure === dtest: 1 failure cdc_test.TestCDC .test_cdc_data_available_in_cdc_raw CASSANDRA-11811. Known failure === upgrade: 1 failure upgrade_tests.cql_tests.TestCQLNodes3RF3_Upgrade_current_3_0_x_To_indev_3_x .whole_map_conditional_test New flaky failure. No jira filed yet. === novnode: 8 failures All failures are in paging test, and are due to CASSANDRA-12666 which is Patch Available.
