Brandon, just an idea, are you sure you use 10.8.2.2 jars also on client accessing this Derby network server? Once I switched everything to 10.8.2.2 I stopped seeing the errors on IBM J9.
On Wed, Apr 4, 2012 at 8:57 PM, Brandon L. Duncan <[email protected]> wrote: > Hi all, > > I am seeing intermittent failures on 10.8.2.2 running on J9. I switched to > lib-debug to add line numbers in the stack. Per your recommendation Myrna, > I'm putting together a test scenario that I can attach to a possible JIRA. > However, I was wondering in the mean time if the 10.8.2.2 stack with line > numbers indicated a possible area to isolate on in my attempt to recreate? > > Thanks > > > Wed Apr 04 14:46:14 EDT 2012 : Apache Derby Network Server - 10.8.2.2 - > (1181258) started and ready to accept connections on port 1555 > Wed Apr 04 14:46:21 EDT 2012 : Connection number: 1. > ---------------------------------------------------------------- > Wed Apr 04 14:46:21 EDT 2012: > Shutting down instance a816c00e-0136-7ead-b0af-ffffcab24f1a on database > directory /database with class loader > sun.misc.Launcher$AppClassLoader@376a376a > Wed Apr 04 14:46:21 EDT 2012 Thread[DRDAConnThread_11,10,main] Cleanup > action starting > java.lang.NullPointerException > at > org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop(BaseDataFileFactory.java:541) > at > org.apache.derby.impl.services.monitor.TopService.stop(TopService.java:443) > at > org.apache.derby.impl.services.monitor.TopService.shutdown(TopService.java:394) > at > org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1845) > at > org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(BaseMonitor.java:1682) > at > org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1560) > at > org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:979) > at > org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:550) > at > org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2697) > at > org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:385) > at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:70) > at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:248) > at org.apache.derby.jdbc.EmbeddedDriver.connect(EmbeddedDriver.java:126) > at org.apache.derby.impl.drda.Database.makeConnection(Database.java:257) > at > org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(DRDAConnThread.java:1447) > at > org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(DRDAConnThread.java:1397) > at > org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(DRDAConnThread.java:3248) > at > org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(DRDAConnThread.java:1177) > at > org.apache.derby.impl.drda.DRDAConnThread.processCommands(DRDAConnThread.java:982) > at org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:295) > Cleanup action completed > Wed Apr 04 14:46:21 EDT 2012 Thread[DRDAConnThread_11,10,main] Cleanup > action starting > java.lang.NullPointerException > at > org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop(BaseDataFileFactory.java:541) > at > org.apache.derby.impl.services.monitor.TopService.stop(TopService.java:443) > at > org.apache.derby.impl.services.monitor.TopService.shutdown(TopService.java:394) > at > org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1845) > at > org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(BaseMonitor.java:1682) > at > org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1560) > at > org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:979) > at > org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:550) > at > org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2697) > at > org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:385) > at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:70) > at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:248) > at org.apache.derby.jdbc.EmbeddedDriver.connect(EmbeddedDriver.java:126) > at org.apache.derby.impl.drda.Database.makeConnection(Database.java:257) > at > org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(DRDAConnThread.java:1447) > at > org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(DRDAConnThread.java:1397) > at > org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(DRDAConnThread.java:3248) > at > org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(DRDAConnThread.java:1177) > at > org.apache.derby.impl.drda.DRDAConnThread.processCommands(DRDAConnThread.java:982) > at org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:295) > Cleanup action completed > Wed Apr 04 14:46:21 EDT 2012 Thread[DRDAConnThread_11,10,main] (DATABASE = > /database), (DRDAID = {1}), Java exception: ': > java.lang.NullPointerException'. > Wed Apr 04 14:46:24 EDT 2012 : Connection number: 2. > ---------------------------------------------------------------- > Wed Apr 04 14:46:24 EDT 2012: Shutting down Derby engine > Wed Apr 04 14:46:24 EDT 2012 : Unexpected exception: > {0} > Wed Apr 04 14:46:24 EDT 2012 : null > java.lang.NullPointerException > at > org.apache.derby.impl.services.monitor.TopService.getService(TopService.java:128) > at > org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(BaseMonitor.java:199) > at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:241) > at org.apache.derby.jdbc.EmbeddedDriver.connect(EmbeddedDriver.java:126) > at > org.apache.derby.impl.drda.NetworkServerControlImpl.blockingStart(NetworkServerControlImpl.java:913) > at > org.apache.derby.impl.drda.NetworkServerControlImpl.executeWork(NetworkServerControlImpl.java:2243) > at > org.apache.derby.drda.NetworkServerControl.main(NetworkServerControl.java:320) > Wed Apr 04 14:46:24 EDT 2012 : Apache Derby Network Server - 10.8.2.2 - > (1181258) shutdown > > > On Tue, Mar 20, 2012 at 8:22 AM, Brandon L. Duncan > <[email protected]> wrote: >> >> Thanks Peter and Myrna for the feedback. Myrna, those Wiki links were >> helpful; I did not even know they existed. I'm going to get a J9 environment >> setup with 10.8.2.2 in the next day or two and see how that goes. >> >> Thank you both again, >> Brandon >> >> On Tue, Mar 20, 2012 at 3:47 AM, Peter Ondruška >> <[email protected]> wrote: >>> >>> Brandon, I run 10.8.2.2 with J9: >>> >>> $ java -version >>> java version "1.6.0" >>> Java(TM) SE Runtime Environment (build pap3260sr9fp2-20110627_03(SR9 >>> FP2)) >>> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc-32 >>> jvmap3260sr9-20110624_85526 (JIT enabled, AOT enabled) >>> J9VM - 20110624_085526 >>> JIT - r9_20101028_17488ifx17 >>> GC - 20101027_AA) >>> JCL - 20110530_01 >>> >>> at that works, also SR9 FP3 work fine (this is on AIX). >>> >>> On Mon, Mar 19, 2012 at 6:18 PM, Brandon L. Duncan >>> <[email protected]> wrote: >>> > Thanks Peter. >>> > >>> > Do you remember if you upgraded to 10.8.2.2 or 10.8.1.2? We do have a >>> > test >>> > environment with 10.8.1.2, and are seeing similar results. I do agree >>> > that >>> > it seems to be a strange coexistence with Derby and J9. The IBM Classic >>> > JVM >>> > doesn't seem to have this issue, although IBM seems to have eliminated >>> > it in >>> > V7R1M0. >>> > >>> > Mon Mar 19 12:46:40 EDT 2012 : Apache Derby Network Server - 10.8.1.2 - >>> > (1095077) started and ready to accept connections on port 11527 >>> > Mon Mar 19 12:46:44 EDT 2012 : Connection number: 1. >>> > ---------------------------------------------------------------- >>> > Mon Mar 19 12:46:45 EDT 2012: >>> > Shutting down instance a816c00e-0136-2bda-791f-ffffcab24f1a on database >>> > directory /database with class loader >>> > sun.misc.Launcher$AppClassLoader@376a376a >>> > Mon Mar 19 12:46:45 EDT 2012 Thread[DRDAConnThread_11,10,main] Cleanup >>> > action starting >>> > java.lang.NullPointerException >>> > at >>> > org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop(Unknown >>> > Source) >>> > at org.apache.derby.impl.services.monitor.TopService.stop(Unknown >>> > Source) >>> > at org.apache.derby.impl.services.monitor.TopService.shutdown(Unknown >>> > Source) >>> > at >>> > org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown >>> > Source) >>> > at >>> > >>> > org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown >>> > Source) >>> > at >>> > >>> > org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown >>> > Source) >>> > at >>> > >>> > org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown >>> > Source) >>> > at >>> > >>> > org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown >>> > Source) >>> > at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown >>> > Source) >>> > at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source) >>> > at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source) >>> > at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) >>> > at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) >>> > at org.apache.derby.impl.drda.Database.makeConnection(Unknown Source) >>> > at >>> > org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(Unknown >>> > Source) >>> > at >>> > org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(Unknown >>> > Source) >>> > at org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(Unknown >>> > Source) >>> > at >>> > org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown >>> > Source) >>> > at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown >>> > Source) >>> > at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source) >>> > Cleanup action completed >>> > Mon Mar 19 12:46:45 EDT 2012 Thread[DRDAConnThread_11,10,main] Cleanup >>> > action starting >>> > java.lang.NullPointerException >>> > at >>> > org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop(Unknown >>> > Source) >>> > at org.apache.derby.impl.services.monitor.TopService.stop(Unknown >>> > Source) >>> > at org.apache.derby.impl.services.monitor.TopService.shutdown(Unknown >>> > Source) >>> > at >>> > org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown >>> > Source) >>> > at >>> > >>> > org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown >>> > Source) >>> > at >>> > >>> > org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown >>> > Source) >>> > at >>> > >>> > org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown >>> > Source) >>> > at >>> > >>> > org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown >>> > Source) >>> > at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown >>> > Source) >>> > at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source) >>> > at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source) >>> > at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) >>> > at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) >>> > at org.apache.derby.impl.drda.Database.makeConnection(Unknown Source) >>> > at >>> > org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(Unknown >>> > Source) >>> > at >>> > org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(Unknown >>> > Source) >>> > at org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(Unknown >>> > Source) >>> > at >>> > org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown >>> > Source) >>> > at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown >>> > Source) >>> > at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source) >>> > Cleanup action completed >>> > Mon Mar 19 12:46:45 EDT 2012 Thread[DRDAConnThread_11,10,main] >>> > (DATABASE = >>> > /database), (DRDAID = {1}), Java exception: ': >>> > java.lang.NullPointerException'. >>> > Mon Mar 19 12:46:47 EDT 2012 : Connection number: 2. >>> > ---------------------------------------------------------------- >>> > Mon Mar 19 12:46:47 EDT 2012: Shutting down Derby engine >>> > Mon Mar 19 12:46:47 EDT 2012 : Unexpected exception: >>> > {0} >>> > Mon Mar 19 12:46:47 EDT 2012 : null >>> > java.lang.NullPointerException >>> > at org.apache.derby.impl.services.monitor.TopService.getService(Unknown >>> > Source) >>> > at org.apache.derby.impl.services.monitor.BaseMonitor.shutdown(Unknown >>> > Source) >>> > at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) >>> > at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) >>> > at >>> > org.apache.derby.impl.drda.NetworkServerControlImpl.blockingStart(Unknown >>> > Source) >>> > at >>> > org.apache.derby.impl.drda.NetworkServerControlImpl.executeWork(Unknown >>> > Source) >>> > at org.apache.derby.drda.NetworkServerControl.main(Unknown Source) >>> > Mon Mar 19 12:46:47 EDT 2012 : Apache Derby Network Server - 10.8.1.2 - >>> > (1095077) shutdown >>> > >>> > >>> > On Mon, Mar 19, 2012 at 12:12 PM, Peter Ondruška >>> > <[email protected]> wrote: >>> >> >>> >> I have seen the same problem and resolved by upgrading Derby to 10.8. >>> >> There must be something strange in J9 and Derby coexistence. This >>> >> happened very random. >>> >> >>> >> On Mon, Mar 19, 2012 at 4:55 PM, Brandon L. Duncan >>> >> <[email protected]> wrote: >>> >> > I was wondering if anyone came across this error before while >>> >> > attempting >>> >> > to >>> >> > establish a connection to Derby? The database seems to boot fine, >>> >> > but >>> >> > when a >>> >> > connection is attempted it just bombs out. It also is not >>> >> > always reproducible, as at times it will be fine, other times it >>> >> > errors >>> >> > with >>> >> > the exception below. The JVM is IBM's J9 implementation. I know >>> >> > Derby >>> >> > 10.4.2.0 is a wee bit old, but upgrading this instance would not be >>> >> > easy >>> >> > at >>> >> > this point. >>> >> > >>> >> > java version "1.6.0" >>> >> > >>> >> > Java(TM) SE Runtime Environment (build >>> >> > pap3260sr9ifix-20110211_02(SR9+IZ94423)) >>> >> > >>> >> > IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 OS/400 ppc-32 >>> >> > jvmap3260sr9-20101130 (JIT enabled, AOT enabled) >>> >> > >>> >> > J9VM - 20101124_069295 >>> >> > >>> >> > JIT - r9_20101028_17488ifx2 >>> >> > >>> >> > GC - 20101027_AA) >>> >> > >>> >> > JCL - 20101119_01 >>> >> > >>> >> > >>> >> > Derby Log: >>> >> > >>> >> > Apache Derby Network Server - 10.4.2.0 - (689064) started and ready >>> >> > to >>> >> > accept connections on port 1555 at 2012-03-15 13:27:29.860 GMT >>> >> > Connection number: 1. >>> >> > >>> >> > 2012-03-15 13:27:34.382 GMT: >>> >> > Shutting down instance a816c00e-0136-168a-b0de-ffffd934f54c >>> >> > ---------------------------------------------------------------- >>> >> > 2012-03-15 13:27:34.384 GMT Thread[DRDAConnThread_11,5,main] Cleanup >>> >> > action >>> >> > starting >>> >> > java.lang.NullPointerException >>> >> > at >>> >> > org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.services.monitor.TopService.stop(Unknown >>> >> > Source) >>> >> > at >>> >> > org.apache.derby.impl.services.monitor.TopService.shutdown(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > >>> >> > org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > >>> >> > org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > >>> >> > org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > >>> >> > org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source) >>> >> > at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown >>> >> > Source) >>> >> > at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) >>> >> > at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) >>> >> > at org.apache.derby.impl.drda.Database.makeConnection(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(Unknown >>> >> > Source) >>> >> > at >>> >> > org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source) >>> >> > Cleanup action completed >>> >> > 2012-03-15 13:27:34.385 GMT Thread[DRDAConnThread_11,5,main] Cleanup >>> >> > action >>> >> > starting >>> >> > java.lang.NullPointerException >>> >> > at >>> >> > org.apache.derby.impl.store.raw.data.BaseDataFileFactory.stop(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.services.monitor.TopService.stop(Unknown >>> >> > Source) >>> >> > at >>> >> > org.apache.derby.impl.services.monitor.TopService.shutdown(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > >>> >> > org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > >>> >> > org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > >>> >> > org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > >>> >> > org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source) >>> >> > at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown >>> >> > Source) >>> >> > at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) >>> >> > at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) >>> >> > at org.apache.derby.impl.drda.Database.makeConnection(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(Unknown >>> >> > Source) >>> >> > at >>> >> > >>> >> > org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(Unknown >>> >> > Source) >>> >> > at >>> >> > org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown >>> >> > Source) >>> >> > at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source) >>> >> > Cleanup action completed >>> >> > 2012-03-15 13:27:34.388 GMT Thread[DRDAConnThread_11,5,main] >>> >> > (DATABASE = >>> >> > ../../database), (DRDAID = {1}), Java exception: ': >>> >> > java.lang.NullPointerException'. >>> >> > Connection number: 2. >>> >> > >>> >> > Thanks for any feedback. >>> > >>> > >> >> >
