Hi all,
I'm kinda new to Pacemaker and have some questions I hope some of you could
help me with, any idea(s) would be highly appreciated, thank you.
I'am building the HA cluster with corosync2.3.3 + cluster-glue +
pacemaker-1.1.11-rc5. The problem is, when I try to remove a node from HA
cluster with "crm_node -f -R nodeID", it seems to remove the wrong node.
My setup is as follows(and it can be reproduced):
1. The HA is ok with 2 resource and 2 nodes, and with stonith-enabled="false":
root@h1:/opt/bin# crm status
Last updated: Sat Apr 12 16:00:51 2014
Last change: Sat Apr 12 16:00:43 2014 via cibadmin on h1
Stack: corosync
Current DC: h1 (1084752017) - partition with quorum
Version: 1.1.10-33f9d09
2 Nodes configured
2 Resources configured
Online: [ h0 h1 ]
VMdos-1 (ocf::heartbeat:VirtualDomain): Started h1
root@h1:/opt/bin#
root@h1:/opt/bin# crm_node --list
1084752017 h1
1084752016 h0
root@h1:/opt/bin#
2. remove node h0 from HA
(1). Stop pacemakerd and corosync service on node h0 (2). Run the follow cmd on
node h1 to Remove node info from cib:
cibadmin --delete --obj_type status --crm_xml "<node_state id=\"1084752016\"/>"
cibadmin --delete --obj_type nodes --crm_xml "<node id=\"1084752016\"/>"
at this time; the result of "crm status" shows that node " h0 " is offline and
there's no information of node h0 in CIB. But the "crm_node --list " still
include node" h0"
3. crm_node -f -R 1084752016
After this cmd, node " h1" is removed, and result of "crm status " is as
follow(with 0 node):
root@h1:/opt/bin# crm status
Last updated: Sat Apr 12 15:59:42 2014
Last change: Sat Apr 12 15:59:37 2014 via crm_node on h1
Stack: corosync
Current DC: NONE
0 Nodes configured
2 Resources configured
root@h1:/opt/bin
The corosync.conf is as follow:
.............
quorum {
provider: corosync_votequorum
expected_votes: 2
allow_downscale: 1
two_node: 1
}
...........
The corosync.log is as follow, it seems node " h1" is removed due to fencing!
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ringbuffer.c:299 Free'ing
ringbuffer: /dev/shm/qb-cmap-request-25004-14152-27-header
Apr 12 16:22:35 [25026] h1 cib: debug: activateCibXml:
Triggering CIB write for cib_delete op
Apr 12 16:22:35 [25031] h1 crmd: notice: crm_reap_dead_member:
Removing h0/1084752016 from the membership list
Apr 12 16:22:35 [25031] h1 crmd: notice: reap_crm_member: Purged
1 peers with id=1084752016 and/or uname=(null) from the membership cache
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: log_cib_diff: Config update:
Local-only Change: 0.12.1
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: - <cib
admin_epoch="0" epoch="11" num_updates="1">
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
<configuration>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
<nodes>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<node id="1084752017" uname="h1"/>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
</nodes>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
</configuration>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
</cib>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: ++ <cib
epoch="12" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2"
crm_feature_set="3.0.8" cib-last-written="Sat Apr 12 16:22:35 2014"
update-origin="h1" update-client="crm_node" have-quorum="1"
dc-uuid="1084752017"/>
Apr 12 16:22:35 [25026] h1 cib: notice: log_cib_diff: cib:diff:
Local-only Change: 0.12.1
Apr 12 16:22:35 [25026] h1 cib: notice: cib:diff: -- <node
id="1084752017" uname="h1"/>
Apr 12 16:22:35 [25026] h1 cib: notice: cib:diff: ++ <cib
epoch="12" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2"
crm_feature_set="3.0.8" cib-last-written="Sat Apr 12 16:22:35 2014"
update-origin="h1" update-client="crm_node" have-quorum="1"
dc-uuid="1084752017"/>
Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
Completed cib_delete operation for section nodes: OK (rc=0,
origin=local/crm_node/2, version=0.12.1)
Apr 12 16:22:35 [25031] h1 crmd: debug: te_update_diff:
Processing diff (cib_delete): 0.11.1 -> 0.12.1 (S_IDLE)
Apr 12 16:22:35 [25031] h1 crmd: info: abort_transition_graph:
te_update_diff:126 - Triggered transition abort (complete=1, node=, tag=diff,
id=(null), magic=NA, cib=0.12.1) : Non-status change
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <diff crm_feature_set="3.0.8" digest="3cccef06483ac4dfeadfb562f6f8293a">
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <diff-removed admin_epoch="0" epoch="11" num_updates="1">
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <cib admin_epoch="0" epoch="11" num_updates="1">
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <configuration>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <nodes>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <node id="1084752017" uname="h1"
__crm_diff_marker__="removed:top"/>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause </nodes>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause </configuration>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause </cib>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause </diff-removed>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <diff-added>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <cib epoch="12" num_updates="1" admin_epoch="0"
validate-with="pacemaker-1.2" crm_feature_set="3.0.8" cib-last-written="Sat Apr
12 16:22:35 2014" update-origin="h1" update-client="crm_node" have-quorum="1"
dc-uuid="1084752017"/>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause </diff-added>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause </diff>
Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_IDLE cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Apr 12 16:22:35 [25031] h1 crmd: notice: do_state_transition: State
transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL
origin=abort_transition_graph ]
Apr 12 16:22:35 [25031] h1 crmd: debug: do_state_transition: All 1
cluster nodes are eligible to run resources.
Apr 12 16:22:35 [25031] h1 crmd: debug: do_pe_invoke: Query 38:
Requesting the current CIB: S_POLICY_ENGINE
Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
Completed cib_query operation for section 'all': OK (rc=0,
origin=local/crmd/38, version=0.12.1)
Apr 12 16:22:35 [25031] h1 crmd: debug: do_pe_invoke_callback:
Invoking the PE: query=38, ref=pe_calc-dc-1397290955-21, seq=1024, quorate=1
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: Diff:
--- 0.12.1
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: Diff:
+++ 0.12.2 3d673c27c3c92939b41c7207edee9f46
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: - <cib
num_updates="1">
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
<status>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<node_state id="1084752017" uname="h1" in_ccm="true" crmd="online"
crm-debug-origin="post_cache_update" join="member" expected="member">
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<transient_attributes id="1084752017">
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<instance_attributes id="status-1084752017">
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<nvpair id="status-1084752017-shutdown" name="shutdown" value="0"/>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<nvpair id="status-1084752017-probe_complete" name="probe_complete"
value="true"/>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
</instance_attributes>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
</transient_attributes>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<lrm id="1084752017">
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<lrm_resources>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<lrm_resource id="VMdos-1" type="VirtualDomain" class="ocf"
provider="heartbeat">
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<lrm_rsc_op id="VMdos-1_last_failure_0"
operation_key="VMdos-1_monitor_0" operation="monitor"
crm-debug-origin="build_active_RAs" crm_feature_set="3.0.8"
transition-key="6:4:7:affc1dba-30eb-458d-8f86-c37d0268e52c"
transition-magic="0:0;6:4:7:affc1dba-30eb-458d-8f86-c37d0268e52c" call-id="5"
rc-code="0" op-status="0" interval="0" last-run="1397290607"
last-rc-change="1397290607" exec-time="92" queue-time="0" op-digest="
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<lrm_rsc_op id="VMdos-1_monitor_30000"
operation_key="VMdos-1_monitor_30000" operation="monitor"
crm-debug-origin="build_active_RAs" crm_feature_set="3.0.8"
transition-key="7:5:0:affc1dba-30eb-458d-8f86-c37d0268e52c"
transition-magic="0:0;7:5:0:affc1dba-30eb-458d-8f86-c37d0268e52c" call-id="6"
rc-code="0" op-status="0" interval="30000" last-rc-change="1397290607"
exec-time="67" queue-time="0" op-digest="0874c7ce5f61a12
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
</lrm_resource>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<lrm_resource id="VMdos-2" type="VirtualDomain" class="ocf"
provider="heartbeat">
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
<lrm_rsc_op id="VMdos-2_last_0" operation_key="VMdos-2_monitor_0"
operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.8"
transition-key="7:6:7:affc1dba-30eb-458d-8f86-c37d0268e52c"
transition-magic="0:7;7:6:7:affc1dba-30eb-458d-8f86-c37d0268e52c" call-id="10"
rc-code="7" op-status="0" interval="0" last-run="1397290608"
last-rc-change="1397290608" exec-time="61" queue-time="0" op-digest="c7d22be
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
</lrm_resource>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
</lrm_resources>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
</lrm>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: --
</node_state>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
</status>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: -
</cib>
Apr 12 16:22:35 [25027] h1 stonith-ng: debug: Config update: ++ <cib
epoch="12" num_updates="2" admin_epoch="0" validate-with="pacemaker-1.2"
crm_feature_set="3.0.8" cib-last-written="Sat Apr 12 16:22:35 2014"
update-origin="h1" update-client="crm_node" have-quorum="1"
dc-uuid="1084752017"/>
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: STONITH
timeout: 60000
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Stop
all active resources: false
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Default
stickiness: 0
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: On loss
of CCM Quorum: Stop ALL resources
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_domains:
Unpacking domains
Apr 12 16:22:35 [25030] h1 pengine: warning: unpack_status: Node h1
in status section no longer exists
Apr 12 16:22:35 [25030] h1 pengine: info: unpack_status: Node
1084752017 is unknown
Apr 12 16:22:35 [25030] h1 pengine: info: native_print: VMdos-1
(ocf::heartbeat:VirtualDomain): Stopped
Apr 12 16:22:35 [25030] h1 pengine: info: native_print: VMdos-2
(ocf::heartbeat:VirtualDomain): Stopped
Apr 12 16:22:35 [25030] h1 pengine: debug: native_assign_node: Could
not allocate a node for VMdos-1
Apr 12 16:22:35 [25030] h1 pengine: info: native_color: Resource
VMdos-1 cannot run anywhere
Apr 12 16:22:35 [25030] h1 pengine: debug: native_assign_node: Could
not allocate a node for VMdos-2
Apr 12 16:22:35 [25030] h1 pengine: info: native_color: Resource
VMdos-2 cannot run anywhere
Apr 12 16:22:35 [25030] h1 pengine: info: LogActions: Leave VMdos-1
(Stopped)
Apr 12 16:22:35 [25030] h1 pengine: info: LogActions: Leave VMdos-2
(Stopped)
Apr 12 16:22:35 [25030] h1 pengine: notice: process_pe_message:
Calculated Transition 4: /var/lib/pacemaker/pengine/pe-input-4.bz2
Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Apr 12 16:22:35 [25031] h1 crmd: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=handle_response ]
Apr 12 16:22:35 [25031] h1 crmd: debug: unpack_graph: Unpacked
transition 4: 0 actions in 0 synapses
Apr 12 16:22:35 [25031] h1 crmd: info: do_te_invoke: Processing
graph 4 (ref=pe_calc-dc-1397290955-21) derived from
/var/lib/pacemaker/pengine/pe-input-4.bz2
Apr 12 16:22:35 [25031] h1 crmd: debug: print_graph: Empty
transition graph
Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
Completed cib_delete operation for section status: OK (rc=0,
origin=local/crm_node/3, version=0.12.2)
Apr 12 16:22:35 [25031] h1 crmd: debug: te_update_diff:
Processing diff (cib_delete): 0.12.1 -> 0.12.2 (S_TRANSITION_ENGINE)
Apr 12 16:22:35 [25031] h1 crmd: info: abort_transition_graph:
te_update_diff:188 - Triggered transition abort (complete=0, node=h1,
tag=transient_attributes, id=1084752017, magic=NA, cib=0.12.2) : Transient
attribute: removal
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <transient_attributes id="1084752017">
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <instance_attributes id="status-1084752017">
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <nvpair id="status-1084752017-shutdown" name="shutdown" value="0"/>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause <nvpair id="status-1084752017-probe_complete" name="probe_complete"
value="true"/>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause </instance_attributes>
Apr 12 16:22:35 [25031] h1 crmd: debug: abort_transition_graph:
Cause </transient_attributes>
Apr 12 16:22:35 [25031] h1 crmd: debug: update_abort_priority:
Abort priority upgraded from 0 to 1000000
Apr 12 16:22:35 [25031] h1 crmd: debug: update_abort_priority:
Abort action done superceeded by restart
Apr 12 16:22:35 [25031] h1 crmd: notice: run_graph: Transition 4
(Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-4.bz2): Complete
Apr 12 16:22:35 [25031] h1 crmd: debug: print_graph: Empty
transition graph
Apr 12 16:22:35 [25031] h1 crmd: debug: te_graph_trigger:
Transition 4 is now complete
Apr 12 16:22:35 [25031] h1 crmd: debug: notify_crmd: Processing
transition completion in state S_TRANSITION_ENGINE
Apr 12 16:22:35 [25031] h1 crmd: debug: notify_crmd: Transition 4
status: restart - Transient attribute: removal
Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL origin=notify_crmd ]
Apr 12 16:22:35 [25031] h1 crmd: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC
cause=C_FSA_INTERNAL origin=notify_crmd ]
Apr 12 16:22:35 [25031] h1 crmd: debug: do_state_transition: All 1
cluster nodes are eligible to run resources.
Apr 12 16:22:35 [25031] h1 crmd: debug: do_pe_invoke: Query 39:
Requesting the current CIB: S_POLICY_ENGINE
Apr 12 16:22:35 [25024] h1 pacemakerd: info: crm_client_new:
Connecting 0x25b4ea0 for uid=0 gid=0 pid=14152
id=f3612e17-0806-4355-a3fc-2cf1feda1e6d
Apr 12 16:22:35 [25024] h1 pacemakerd: debug: handle_new_connection:
IPC credentials authenticated (25024-14152-10)
Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_ipcs_shm_connect:
connecting to client [14152]
Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
Completed cib_query operation for section 'all': OK (rc=0,
origin=local/crmd/39, version=0.12.2)
Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Apr 12 16:22:35 [25026] h1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (25026-14152-13)
Apr 12 16:22:35 [25026] h1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(25026-14152-13) state:2
Apr 12 16:22:35 [25026] h1 cib: info: crm_client_destroy:
Destroying 0 events
Apr 12 16:22:35 [25026] h1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-response-25026-14152-13-header
Apr 12 16:22:35 [25026] h1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-event-25026-14152-13-header
Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_rb_open_2: shm
size:131085; real_size:135168; rb->word_size:33792
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ipc_setup.c:478 IPC
credentials authenticated (25004-14152-27)
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ipc_shm.c:294 connecting
to client [14152]
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ringbuffer.c:236 shm
size:1048589; real_size:1052672; rb->word_size:263168
Apr 12 16:22:35 [25031] h1 crmd: debug: do_pe_invoke_callback:
Invoking the PE: query=39, ref=pe_calc-dc-1397290955-22, seq=1024, quorate=1
Apr 12 16:22:35 [25031] h1 crmd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (25031-14152-14)
Apr 12 16:22:35 [25031] h1 crmd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(25031-14152-14) state:2
Apr 12 16:22:35 [25031] h1 crmd: info: crm_client_destroy:
Destroying 0 events
Apr 12 16:22:35 [25031] h1 crmd: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-crmd-response-25031-14152-14-header
...
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: STONITH
timeout: 60000
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: STONITH
of failed nodes is disabled
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Stop
all active resources: false
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Cluster
is symmetric - resources can run anywhere by default
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Default
stickiness: 0
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: On loss
of CCM Quorum: Stop ALL resources
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_config: Node
scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Apr 12 16:22:35 [25030] h1 pengine: debug: unpack_domains:
Unpacking domains
Apr 12 16:22:35 [25030] h1 pengine: info: native_print: VMdos-1
(ocf::heartbeat:VirtualDomain): Stopped
Apr 12 16:22:35 [25030] h1 pengine: info: native_print: VMdos-2
(ocf::heartbeat:VirtualDomain): Stopped
Apr 12 16:22:35 [25030] h1 pengine: debug: native_assign_node: Could
not allocate a node for VMdos-1
Apr 12 16:22:35 [25030] h1 pengine: info: native_color: Resource
VMdos-1 cannot run anywhere
Apr 12 16:22:35 [25030] h1 pengine: debug: native_assign_node: Could
not allocate a node for VMdos-2
Apr 12 16:22:35 [25030] h1 pengine: info: native_color: Resource
VMdos-2 cannot run anywhere
Apr 12 16:22:35 [25030] h1 pengine: info: LogActions: Leave VMdos-1
(Stopped)
Apr 12 16:22:35 [25030] h1 pengine: info: LogActions: Leave VMdos-2
(Stopped)
Apr 12 16:22:35 [25030] h1 pengine: notice: process_pe_message:
Calculated Transition 5: /var/lib/pacemaker/pengine/pe-input-5.bz2
Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Apr 12 16:22:35 [25031] h1 crmd: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=handle_response ]
Apr 12 16:22:35 [25031] h1 crmd: debug: unpack_graph: Unpacked
transition 5: 0 actions in 0 synapses
Apr 12 16:22:35 [25031] h1 crmd: info: do_te_invoke: Processing
graph 5 (ref=pe_calc-dc-1397290955-22) derived from
/var/lib/pacemaker/pengine/pe-input-5.bz2
Apr 12 16:22:35 [25031] h1 crmd: debug: print_graph: Empty
transition graph
Apr 12 16:22:35 [25031] h1 crmd: notice: run_graph: Transition 5
(Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-5.bz2): Complete
Apr 12 16:22:35 [25031] h1 crmd: debug: print_graph: Empty
transition graph
Apr 12 16:22:35 [25031] h1 crmd: debug: te_graph_trigger:
Transition 5 is now complete
Apr 12 16:22:35 [25031] h1 crmd: debug: notify_crmd: Processing
transition completion in state S_TRANSITION_ENGINE
Apr 12 16:22:35 [25031] h1 crmd: debug: notify_crmd: Transition 5
status: done - <null>
Apr 12 16:22:35 [25031] h1 crmd: debug: s_crmd_fsa: Processing
I_TE_SUCCESS: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Apr 12 16:22:35 [25031] h1 crmd: info: do_log: FSA: Input
I_TE_SUCCESS from notify_crmd() received in state S_TRANSITION_ENGINE
Apr 12 16:22:35 [25031] h1 crmd: notice: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
cause=C_FSA_INTERNAL origin=notify_crmd ]
Apr 12 16:22:35 [25031] h1 crmd: debug: do_state_transition:
Starting PEngine Recheck Timer
Apr 12 16:22:35 [25031] h1 crmd: debug: crm_timer_start: Started
PEngine Recheck Timer (I_PE_CALC:900000ms), src=62
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ringbuffer.c:236 shm
size:1048589; real_size:1052672; rb->word_size:263168
Apr 12 16:22:35 [25026] h1 cib: info: write_cib_contents:
Archived previous version as /var/lib/pacemaker/cib/cib-9.raw
Apr 12 16:22:35 [25026] h1 cib: debug: write_cib_contents: Writing
CIB to disk
Apr 12 16:22:35 [25026] h1 cib: debug: qb_rb_close: Free'ing
ringbuffer: /dev/shm/qb-cib_rw-request-25026-14152-13-header
Apr 12 16:22:35 [25026] h1 cib: info: write_cib_contents: Wrote
version 0.12.0 of the CIB to disk (digest: bd7d26226d6aa75f28b9eb670a67e944)
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ringbuffer.c:236 shm
size:1048589; real_size:1052672; rb->word_size:263168
Apr 12 16:22:35 [25003] h1 corosync debug [MAIN ] ipc_glue.c:272 connection
created
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] cmap.c:306 lib_init_fn:
conn=0x7f32d0f4f4b0
Apr 12 16:22:35 [25024] h1 pacemakerd: notice: pcmk_ipc_dispatch:
Instructing peers to remove references to node (null)/1084752016
Apr 12 16:22:35 [25026] h1 cib: info: crm_client_new:
Connecting 0xa23770 for uid=0 gid=0 pid=14152
id=701dfa3c-d585-49d2-bd1e-f44636823e1b
Apr 12 16:22:35 [25026] h1 cib: debug: handle_new_connection:
IPC credentials authenticated (25026-14152-13)
Apr 12 16:22:35 [25026] h1 cib: debug: qb_ipcs_shm_connect:
connecting to client [14152]
Apr 12 16:22:35 [25026] h1 cib: debug: write_cib_contents: Wrote
digest bd7d26226d6aa75f28b9eb670a67e944 to disk
Apr 12 16:22:35 [25026] h1 cib: info: retrieveCib: Reading cluster
configuration from: /var/lib/pacemaker/cib/cib.y6wnAg (digest:
/var/lib/pacemaker/cib/cib.ncXFvs)
...
Apr 12 16:22:35 [25026] h1 cib: debug: write_cib_contents:
Activating /var/lib/pacemaker/cib/cib.y6wnAg
Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
Completed cib_delete operation for section nodes: OK (rc=0,
origin=local/crm_node/2, version=0.12.2)
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ipcs.c:757 HUP conn
(25004-14152-27)
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] ipcs.c:605
qb_ipcs_disconnect(25004-14152-27) state:2
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] loop_poll_epoll.c:117
epoll_ctl(del): Bad file descriptor (9)
Apr 12 16:22:35 [25003] h1 corosync debug [MAIN ] ipc_glue.c:417
cs_ipcs_connection_closed()
Apr 12 16:22:35 [25003] h1 corosync debug [QB ] cmap.c:325 exit_fn for
conn=0x7f32d0f4f4b0
Apr 12 16:22:35 [25003] h1 corosync debug [MAIN ] ipc_glue.c:390
cs_ipcs_connection_destroyed()
Apr 12 16:22:35 [25026] h1 cib: info: cib_process_request:
Completed cib_delete operation for section status: OK (rc=0,
origin=local/crm_node/3, version=0.12.2)
...
Apr 12 16:22:35 [25024] h1 pacemakerd: debug:
qb_ipcs_dispatch_connection_request: HUP conn (25024-14152-10)
Apr 12 16:22:35 [25024] h1 pacemakerd: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(25024-14152-10) state:2
Apr 12 16:22:35 [25024] h1 pacemakerd: info: crm_client_destroy:
Destroying 0 events
...
Apr 12 16:22:35 [25026] h1 cib: debug:
qb_ipcs_dispatch_connection_request: HUP conn (25026-14152-13)
Apr 12 16:22:35 [25026] h1 cib: debug: qb_ipcs_disconnect:
qb_ipcs_disconnect(25026-14152-13) state:2
Apr 12 16:22:35 [25026] h1 cib: info: crm_client_destroy:
Destroying 0 events
...
Apr 12 16:22:35 [25024] h1 pacemakerd: notice: crm_reap_dead_member:
Removing h0/1084752016 from the membership list
Apr 12 16:22:35 [25024] h1 pacemakerd: notice: reap_crm_member: Purged
1 peers with id=1084752016 and/or uname=(null) from the membership cache
-------------------------------------------------------------------------------------------------------------------------------------
本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址中列出
的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
邮件！
This e-mail and its attachments contain confidential information from H3C,
which is
intended only for the person or entity whose address is listed above. Any use
of the
information contained herein in any way (including, but not limited to, total
or partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify
the sender
by phone or email immediately and delete it!
_______________________________________________
Pacemaker mailing list: [email protected]
http://oss.clusterlabs.org/mailman/listinfo/pacemaker
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
