Now after adding one more node I have almost classic situation, but the https://github.com/basho/riak/blob/1.0.2-release/RELEASE-NOTES.org#ownership-handoff-stall solution does not work for it. Any ideas?
# riak-admin ring-status ================================== Claimant =================================== Claimant: 'riak@2' Status: up Ring Ready: true ============================== Ownership Handoff ============================== Owner: riak@2 Next Owner: riak@3 Index: 685078892498860742907977265335757665463718379520 Waiting on: [] Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] Index: 959110449498405040071168171470060731649205731328 Waiting on: [] Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] ------------------------------------------------------------------------------- Owner: riak@2 Next Owner: riak@4 Index: 1438665674247607560106752257205091097473808596992 Waiting on: [] Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] ------------------------------------------------------------------------------- Owner: riak@3 Next Owner: riak@2 Index: 1118962191081472546749696200048404186924073353216 Waiting on: [] Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] Index: 1392993748081016843912887106182707253109560705024 Waiting on: [] Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] ------------------------------------------------------------------------------- Owner: riak@3 Next Owner: riak@4 Index: 707914855582156101004909840846949587645842325504 Waiting on: [] Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] Index: 1255977969581244695331291653115555720016817029120 Waiting on: [] Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] ------------------------------------------------------------------------------- Owner: riak@4 Next Owner: riak@3 Index: 1415829711164312202009819681693899175291684651008 Waiting on: [] Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] ------------------------------------------------------------------------------- ============================== Unreachable Nodes ============================== All nodes are up and reachable 2015-09-21 18:55 GMT+03:00 Nico Revin <[email protected]>: > now after running > https://github.com/basho/riak/blob/1.0.2-release/RELEASE-NOTES.org#ownership-handoff-stall > things got slightly better but still... > # riak-admin ring-status > ================================== Claimant > =================================== > Claimant: 'riak@2' > Status: up > Ring Ready: true > > ============================== Ownership Handoff > ============================== > Owner: riak@2 > Next Owner: riak@1 > > Index: 1370157784997721485815954530671515330927436759040 > Waiting on: [riak_kv_vnode] > Complete: [riak_pipe_vnode] > > ------------------------------------------------------------------------------- > Owner: riak@2 > Next Owner: riak@3 > > Index: 685078892498860742907977265335757665463718379520 > Waiting on: [] > Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] > > Index: 959110449498405040071168171470060731649205731328 > Waiting on: [] > Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] > > ------------------------------------------------------------------------------- > Owner: riak@2 > Next Owner: riak@4 > > Index: 1438665674247607560106752257205091097473808596992 > Waiting on: [riak_kv_vnode] > Complete: [riak_pipe_vnode,riak_search_vnode] > > ------------------------------------------------------------------------------- > Owner: riak@3 > Next Owner: riak@2 > > Index: 1118962191081472546749696200048404186924073353216 > Waiting on: [] > Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] > > Index: 1392993748081016843912887106182707253109560705024 > Waiting on: [riak_kv_vnode] > Complete: [riak_pipe_vnode,riak_search_vnode] > > ------------------------------------------------------------------------------- > Owner: riak@3 > Next Owner: riak@4 > > Index: 707914855582156101004909840846949587645842325504 > Waiting on: [] > Complete: [riak_kv_vnode,riak_pipe_vnode,riak_search_vnode] > > Index: 1255977969581244695331291653115555720016817029120 > Waiting on: [riak_kv_vnode] > Complete: [riak_pipe_vnode,riak_search_vnode] > > ------------------------------------------------------------------------------- > Owner: riak@4 > Next Owner: riak@1 > > Index: 182687704666362864775460604089535377456991567872 > Waiting on: [riak_search_vnode] > Complete: [riak_kv_vnode,riak_pipe_vnode] > > Index: 456719261665907161938651510223838443642478919680 > Waiting on: [riak_search_vnode] > Complete: [riak_kv_vnode,riak_pipe_vnode] > > Index: 730750818665451459101842416358141509827966271488 > Waiting on: [riak_search_vnode] > Complete: [riak_kv_vnode,riak_pipe_vnode] > > Index: 1004782375664995756265033322492444576013453623296 > Waiting on: [riak_search_vnode] > Complete: [riak_kv_vnode,riak_pipe_vnode] > > Index: 1278813932664540053428224228626747642198940975104 > Waiting on: [riak_kv_vnode,riak_search_vnode] > Complete: [riak_pipe_vnode] > > ------------------------------------------------------------------------------- > Owner: riak@4 > Next Owner: riak@3 > > Index: 1415829711164312202009819681693899175291684651008 > Waiting on: [riak_kv_vnode] > Complete: [riak_pipe_vnode,riak_search_vnode] > > ------------------------------------------------------------------------------- > > ============================== Unreachable Nodes > ============================== > All nodes are up and reachable > > # riak-admin transfers > 'riak@4' waiting to handoff 7 partitions > 'riak@3' waiting to handoff 10 partitions > 'riak@2' waiting to handoff 10 partitions > 'riak@1' waiting to handoff 14 partitions > > Active Transfers: > > 2015-09-21 18:53 GMT+03:00 Nico Revin <[email protected]>: >> now after running riak_core_ring:handoff_complete >> >> 2015-09-21 17:42 GMT+03:00 Nico Revin <[email protected]>: >>> transfer-limit is 24 on all nodes >>> >>> 2015-09-21 17:41 GMT+03:00 Damien Krotkine <[email protected]>: >>>> >>>> Have you checked riak-admin transfer-limit ? if it's at zero handoffs will >>>> be blocked. >>>> >>>> >>>> >>>>> Le 21 sept. 2015 à 15:41, Nico Revin <[email protected]> a écrit : >>>>> >>>>> Hi! >>>>> >>>>> I have the following issue: >>>>> # riak-admin ringready >>>>> TRUE All nodes agree on the ring ['riak@1', >>>>> 'riak@2', >>>>> 'riak@3', >>>>> 'riak@4'] >>>>> >>>>> # riak-admin transfers >>>>> 'riak@4' waiting to handoff 8 partitions >>>>> 'riak@3' waiting to handoff 17 partitions >>>>> 'riak@2' waiting to handoff 17 partitions >>>>> 'riak@1' waiting to handoff 26 partitions >>>>> >>>>> Active Transfers: >>>>> >>>>> >>>>> # riak-admin member-status >>>>> ================================= Membership >>>>> ================================== >>>>> Status Ring Pending Node >>>>> ------------------------------------------------------------------------------- >>>>> valid 0.0% 25.0% 'riak@1' >>>>> valid 32.8% 25.0% 'riak@2' >>>>> valid 34.4% 25.0% 'riak@3' >>>>> valid 32.8% 25.0% 'riak@4' >>>>> ------------------------------------------------------------------------------- >>>>> Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 >>>>> >>>>> # riak-admin ring-status >>>>> ================================== Claimant >>>>> =================================== >>>>> Claimant: 'riak@2' >>>>> Status: up >>>>> Ring Ready: true >>>>> >>>>> ============================== Ownership Handoff >>>>> ============================== >>>>> Owner: riak@2 >>>>> Next Owner: riak@1 >>>>> >>>>> Index: 0 >>>>> Waiting on: [] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 274031556999544297163190906134303066185487351808 >>>>> Waiting on: [] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 548063113999088594326381812268606132370974703616 >>>>> Waiting on: [] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 822094670998632891489572718402909198556462055424 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode] >>>>> >>>>> Index: 1096126227998177188652763624537212264741949407232 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode] >>>>> >>>>> Index: 1370157784997721485815954530671515330927436759040 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode] >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Owner: riak@2 >>>>> Next Owner: riak@3 >>>>> >>>>> Index: 685078892498860742907977265335757665463718379520 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> >>>>> Index: 959110449498405040071168171470060731649205731328 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Owner: riak@2 >>>>> Next Owner: riak@4 >>>>> >>>>> Index: 1438665674247607560106752257205091097473808596992 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Owner: riak@3 >>>>> Next Owner: riak@1 >>>>> >>>>> Index: 91343852333181432387730302044767688728495783936 >>>>> Waiting on: [] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 365375409332725729550921208179070754913983135744 >>>>> Waiting on: [] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 639406966332270026714112114313373821099470487552 >>>>> Waiting on: [] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 913438523331814323877303020447676887284957839360 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode] >>>>> >>>>> Index: 1187470080331358621040493926581979953470445191168 >>>>> Waiting on: [] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Owner: riak@3 >>>>> Next Owner: riak@2 >>>>> >>>>> Index: 1118962191081472546749696200048404186924073353216 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> Index: 1392993748081016843912887106182707253109560705024 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> ------------------------------------------------------------------------------- >>>>> Owner: riak@3 >>>>> Next Owner: riak@4 >>>>> >>>>> Index: 707914855582156101004909840846949587645842325504 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> >>>>> Index: 1255977969581244695331291653115555720016817029120 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Owner: riak@4 >>>>> Next Owner: riak@1 >>>>> >>>>> Index: 182687704666362864775460604089535377456991567872 >>>>> Waiting on: [riak_search_vnode] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 456719261665907161938651510223838443642478919680 >>>>> Waiting on: [riak_search_vnode] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 730750818665451459101842416358141509827966271488 >>>>> Waiting on: [riak_search_vnode] >>>>> Complete: [riak_kv_vnode,riak_pipe_vnode] >>>>> >>>>> Index: 1004782375664995756265033322492444576013453623296 >>>>> Waiting on: [riak_kv_vnode,riak_search_vnode] >>>>> Complete: [riak_pipe_vnode] >>>>> >>>>> Index: 1278813932664540053428224228626747642198940975104 >>>>> Waiting on: [riak_kv_vnode,riak_search_vnode] >>>>> Complete: [riak_pipe_vnode] >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Owner: riak@4 >>>>> Next Owner: riak@2 >>>>> >>>>> Index: 662242929415565384811044689824565743281594433536 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> >>>>> Index: 936274486415109681974235595958868809467081785344 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> Owner: riak@4 >>>>> Next Owner: riak@3 >>>>> >>>>> Index: 1415829711164312202009819681693899175291684651008 >>>>> Waiting on: [riak_kv_vnode] >>>>> Complete: [riak_pipe_vnode,riak_search_vnode] >>>>> >>>>> ------------------------------------------------------------------------------- >>>>> >>>>> ============================== Unreachable Nodes >>>>> ============================== >>>>> All nodes are up and reachable >>>>> >>>>> # riak-admin diag >>>>> [warning] The following preflists do not satisfy the n_val: [[{0, >>>>> 'riak@2'}, >>>>> >>>>> {22835963083295358096932575511191922182123945984, >>>>> 'riak@2'}, >>>>> >>>>> {45671926166590716193865151022383844364247891968, >>>>> 'riak@3'}], >>>>> ... >>>>> ... >>>>> >>>>> [{1438665674247607560106752257205091097473808596992, >>>>> 'riak@2'}, >>>>> {0, >>>>> 'riak@2'}, >>>>> >>>>> {22835963083295358096932575511191922182123945984, >>>>> 'riak@2'}]] >>>>> [warning] Search is not enabled on all nodes: [{'riak@1', >>>>> false}, >>>>> {'riak@2', >>>>> false}, >>>>> {'riak@3', >>>>> false}, >>>>> {'riak@4', >>>>> true}] >>>>> >>>>> _______________________________________________ >>>>> riak-users mailing list >>>>> [email protected] >>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
