We had a situation for months in which we would get server crashes several times a day for no reason. BMC Remedy support sent us a "debug server" that created a core dump when the server crashed. BMC analyzed this core dump and found it was due to a corrupted Active Link.
Other times we've had corrupted links or filters that caused crashes, but have been able to track them down by workflow logging. Not saying that this is what is your problem, but it might be worth checking into. Dwayne Martin James Madison University ---- Original message ---- >Date: Wed, 24 Dec 2008 16:08:05 +0000 >From: Ron Pierce <[email protected]> >Subject: Re: AR Server Crash - Not sure why... - ARS 7.1 patch 2 >To: [email protected] > > ** Tauf, I had a similarissue, about exact > actually, last week. It ended up being 1 User Tool > Client that was crashing when any workflow kicked > off and, somehow, it was crashing the ARS server as > well. Once the single User Tool instance was > identified and the User closed and restarted the UT > everything went back to normal. > > Might not be the exact same thing for you of course > but just to let you know things like this can > happen. > > Ron > > ------------------------------------------------ > > Date: Wed, 24 Dec 2008 10:22:20 -0500 > From: [email protected] > Subject: Re: AR Server Crash - Not sure why... - ARS > 7.1 patch 2 > To: [email protected] > > ** > > Joe, > > Thanks for the quick response. I think RPC is the > clue. The reason I think that is because after this > happened, when I tried using RDP to remote into the > ‘secondary’ AR server, I got an RPC error. RPC > Unavailable. > > So what that leads me to believe is that because the > 1 server could not execute RPC calls, it locked up > and crashed… which in turn took down the other AR > server as well. > > Does that sound logical? I will investigate with our > windows admins. > > > > Tauf Chowdhury | Forest Laboratories, Inc. > > Sr. Analyst > > Office: 631.858.7765 > > > > From: Action Request System discussion list(ARSList) > [mailto:[email protected]] On Behalf Of Joe > D'Souza > Sent: Wednesday, December 24, 2008 10:18 AM > To: [email protected] > Subject: Re: AR Server Crash - Not sure why... - ARS > 7.1 patch 2 > > > > ** > > Tauf, > > > > The RPC call GLEWF is a Get List Entry With Fields, > a call that seems to be failing while trying to > populate the form after a search with fields in the > current entry as well as populate the Results List.. > > > > I would begin with asking djones what he or she did > at the time when the error happened and then > investigate why that search was failing.. > > > > Cheers > > > > Joe > > -----Original Message----- > From: Action Request System discussion > list(ARSList) [mailto:[email protected]]on > Behalf Of Chowdhury, Tauf > Sent: Wednesday, December 24, 2008 10:11 AM > To: [email protected] > Subject: AR Server Crash - Not sure why... - ARS > 7.1 patch 2 > > ** > > All, > > OK, this is a new issue that started this week. > > Environment: > > AR: Server group, win2k3, 2 AR servers. 2 Mid > tier servers running exclusively on tomcat. Remote > Oracle 10g DB on Unix cluster. ITSM 7.0.3 p7 > > ARS 7.1 patch 2, mid tier on tomcat 5.5.20. > > Symptoms: > > 1. Users cannot log in. > > 2. I verify. Cannot log in with user tool on > either AR server. > > 3. AR System Service on both windows boxes > are in Started state. > > 4. Ar error log shows: > > Tue Dec 23 15:55:03 2008 390635 : AR System > server terminated when a signal/exception was > received by the server (ARNOTE 20) > > Tue Dec 23 15:55:03 2008 > > Timestamp: Tue Dec 23 2008 15:55:04.5320 > > Thread Id: 2420 > > Version: 7.1.00 Patch 002 200802011900 Feb 1 > 2008 20:12:40 > > ServerName: remedy-app > > Database: SQL -- Oracle > > Hardware: Intel Pentium > > OS: Windows NT 5.2 > > RPC Id: 365007 > > RPC Call: 73 (GLEWF) > > RPC Queue: 390635 > > Client: User djones from Mid-tier (protocol 13) > at IP address 172.24.19.113 > > Logging On: > > Code: c0000005 > > Operation: read > > Access Addr: 0x2 > > Stack Begin: > > Stack End > > > > Tue Dec 23 15:55:04 2008 390635 : AR System > server terminated when a signal/exception was > received by the server (ARNOTE 20) > > Tue Dec 23 15:55:04 2008 0xc0000005 > > Tue Dec 23 15:55:04 2008 390635 : AR System > server terminated -- fatal error encountered > (ARNOTE 21) > > > > 5. Had to restart services to regain > connectivity. > > 6. A few hours went by and I pretty much had > a repeat of Steps 1-5. Yes… step 4, the logs > look exactly the same. Same user. > > 7. The next day, Steps 1-5 again, different > time, same error but different user. Then again, > this new user later in the afternoon. > > 8. I asked the users if they had been using > the system all day. Answer was yes. I asked what > they might have been doing at the time, and they > stated just updating an incident. > > 9. SO I contacted BMC and I am basically > running every log under the sun for them until > this error happens again. BMC stated I should > upgrade to Patch 5 on ARS. > > 10. Has anyone experienced this and if so, what > should I look at??? > > > > > > > > Tauf Chowdhury | Forest Laboratories, Inc. > > Sr. Analyst > > Informatics > > Office: 631.858.7765 > > Mobile:646.483.2779 > > > > ---------------------------------------------------- > > This e-mail and its attachments may contain Forest > Laboratories, Inc. proprietary information that is > privileged, confidential or subject to copyright > belonging to Forest Laboratories, Inc. This e-mail > is intended solely for the use of the individual > or entity to which it is addressed. If you are not > the intended recipient of this e-mail, or the > employee or agent responsible for delivering this > e-mail to the intended recipient, you are hereby > notified that any dissemination, distribution, > copying or action taken in relation to the > contents of and attachments to this e-mail is > strictly prohibited and may be unlawful. If you > have received this e-mail in error, please notify > the sender immediately and permanently delete the > original and any copy of this e-mail and any > printout. > > __Platinum Sponsor: www.rmsportal.com ARSlist: > "Where the Answers Are" html___ > > ------------------------------------------------ > > This e-mail and its attachments may contain Forest > Laboratories, Inc. proprietary information that is > privileged, confidential or subject to copyright > belonging to Forest Laboratories, Inc. This e-mail > is intended solely for the use of the individual or > entity to which it is addressed. If you are not the > intended recipient of this e-mail, or the employee > or agent responsible for delivering this e-mail to > the intended recipient, you are hereby notified that > any dissemination, distribution, copying or action > taken in relation to the contents of and attachments > to this e-mail is strictly prohibited and may be > unlawful. If you have received this e-mail in error, > please notify the sender immediately and permanently > delete the original and any copy of this e-mail and > any printout. > __Platinum Sponsor: www.rmsportal.com ARSlist: > "Where the Answers Are" html___ > > ------------------------------------------------ > > Send e-mail anywhere. No map, no compass. Get your > Hotmail® account now. __Platinum Sponsor: > www.rmsportal.com ARSlist: "Where the Answers Are" > html___
