After watching a data management support group rename job (patch 9005) crash
threads on our 7.1.00.002 server several nights in a row during the search
phase, strangely enough the AREA thread 390695, I have gone back today to see
exactly what threads we have configured, and where we got the information to
configure them.
Sat Feb 21 00:14:54 2009 390695 : AR System server terminated when a
signal/exception was received by the server (ARNOTE 20)
Sat Feb 21 00:14:54 2009 0xc0000005
Sat Feb 21 00:14:54 2009 390695 : AR System server terminated -- fatal error
encountered (ARNOTE 21)
It is a documentation nightmare, in that contradictory information is scattered
throughout the guides for ARS, CMDB, ITSM, and various patch docs. Things like
the Plugin Loopback RPC Program Number on the Ports and Queues tab of Server
Information are virtually undocumented... in some cases it does not even appear
in the screen shots to look at (ITSM 7.0.03 Installation Guide), and the entry
in the ar.cfg table in the ARS 7.1.00 Confuring Guide leaves you uncertain
about how to use the setting.
Some of these settings have been clarified in the ITSM 7.0.03 Installation
Guide (and nowhere else), such as the settings for the Reconciliation Engine
[390698 or 699: 10 10] pg.20. The same guide shows setting the CAI Plugin
Registry form to use a custom private queue - we originally used [390630 2 4]
from the 7.0 guide pg.70, then changed it to [2 2] when we disabled the
Requester Console); the 7.0.03 guide says to use [2 6] on pg.59. All of the
screen shots fail to show the Plugin Loopback RPC Program Number field or its
value at all (helpful).
CMDB 2.1.00 Apparently sets the Plugin Loopback RPC Program Number to 390626
and creates a queue [390626 2 2] Install/Cfg pg.28, and every subsequent patch
to the CMDB recreates this line for the queue even if another already exists
with larger values.
The confusion really begins with the Data Management Tools 9003 and 9005, which
instruct you to use the existing Private Queue 390626, which is already
selected as the Plugin Loopback RPC Program Number, and modify it to [16 32]
and then specify it in the CAI Plugin Registry form (9003 pg.26, 9005 pg.34).
This then puts the CAI Plugin used my the data management tool, the Requester
Console (or SRM, I guess), and the CMDB on the same queue. Every time you
update the CMDB with a patch this queue will get an additional entry in the
ar.cfg for [2 2] instead of the [16 32] values, and the extra entry has to be
deleted.
At this point our queues and threads look like this (production and
development):
Type RPC # Min Max Ar.cfg Definition
Admin 390600
Alert 390601 1 1 Private-RPC-Socket: 390601 1 1
Full Text Index 390602
Escalation 390603 1 1 Private-RPC-Socket: 390603 1 1
Flashboards 390619
Fast 390620 16 32 Private-RPC-Socket: 390620 16 32
List 390635 24 24 Private-RPC-Socket: 390635 24 24
Plugin Loopback 390626 16 32 Private-RPC-Socket: 390626 16 32
Plugin-Loopback-RPC-Socket: 390626
CAI Plug-in Registry = 390626 16 Plugin-Filter-API-Threads: 8 40
Per Patch 9003 pg 26-28 and 9005 pg 34-36
(CMDB Install) 390626 2 2 RESETS VALUES TO ITS OWN DEFAULTS 2 2
CMDB Installation and Configuration Guide pg.28 - delete after install
(CAI Plug-in) 390630 2 2 Private-RPC-Socket: 390630 2 2
ITSM 7.0.03 Install Guide pg 59 (7.0 pg 70)
But CAI Plug-in Registry no longer uses this queue due to 9003/5
(Approval) 390624 1 1 Approval-RPC-Socket: 390624
Approval Server Loopback - Approval Server Guide pg 40-43
Private 390694 1 1 Private-RPC-Socket: 390694 1 1
AIE Data Exchanges use this thread - Inferred from AIE Docs
Ext Private 390698 10 10 Private-RPC-Socket: 390698 10 10
RE-RPC-Socket: 390698
ITSM 7.0.03 Install Guide pg 20 (Minimal details in CMDB Install Cfg pg 172)
External Authentication 390695 External-Authentication-RPC-Socket:
390695
QUESTIONS (Finally):
1. How is the Plugin Loopback RPC Program Number _supposed_ to be used, and by
which service or services?
2. Which queue is supposed to be reflected in the CAI Plug-in Registry form -
the same one specified in the Plugin Loopback RPC Program Number?
3. If the data management tool and CMDB are _both_ set to use the same queue
and CAI Plugin, is this why we were seeing problems when trying to run a data
management job after hours?
BTW, the job ran successfully this morning after 8 AM, without finding
something to contend with.
Christopher Strauss, Ph.D.
Call Tracking Administration Manager
University of North Texas Computing & IT Center
http://itsm.unt.edu/
[Also posted to BMCDN]
_______________________________________________________________________________
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
Platinum Sponsor: RMI Solutions ARSlist: "Where the Answers Are"
