error.txt below
Java Platform Detected x64
Java Platform Detected -XX:MaxPermSize=64m -Xss256K -Xmx64m -Xms64m
-XX:+HeapDumpOnOutOfMemoryError -XX:+CreateMinidumpOnCrash
2014-04-04 15:49:43.341:INFO:oejs.Server:jetty-8.1.8.v20121106
2014-04-04 15:49:43.353:INFO:oejdp.ScanningAppProvider:Deployment monitor
D:\AbcData\V12\application server\server\indexer\example\contexts at
interval 0
2014-04-04 15:49:43.358:INFO:oejd.DeploymentManager:Deployable added:
D:\AbcData\V12\application
server\server\indexer\example\contexts\solr-jetty-context.xml
2014-04-04 15:49:43.989:INFO:oejw.StandardDescriptorProcessor:NO JSP
Support for /solr, did not find org.apache.jasper.servlet.JspServlet
Null identity service, trying login service: null
Finding identity service: null
2014-04-04 15:49:44.011:INFO:oejsh.ContextHandler:started
o.e.j.w.WebAppContext{/solr,file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr-webapp/webapp/},D:\AbcData\V12\application
server\server\indexer\example/webapps/solr.war
2014-04-04 15:49:44.012:INFO:oejsh.ContextHandler:started
o.e.j.w.WebAppContext{/solr,file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr-webapp/webapp/},D:\AbcData\V12\application
server\server\indexer\example/webapps/solr.war
Apr 04, 2014 3:49:44 PM org.apache.solr.servlet.SolrDispatchFilter init
INFO: SolrDispatchFilter.init()
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
locateSolrHome
INFO: JNDI not configured for solr (NoInitialContextEx)
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
locateSolrHome
INFO: solr home defaulted to 'solr/' (could not find system property or
JNDI)
Apr 04, 2014 3:49:44 PM org.apache.solr.core.CoreContainer$Initializer
initialize
INFO: looking for solr.xml: D:\AbcData\V12\application
server\server\indexer\example\solr\solr.xml
Apr 04, 2014 3:49:44 PM org.apache.solr.core.CoreContainer <init>
INFO: New CoreContainer 1879341237
Apr 04, 2014 3:49:44 PM org.apache.solr.core.CoreContainer load
INFO: Loading CoreContainer using Solr Home: 'solr/'
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader <init>
INFO: new SolrResourceLoader for directory: 'solr/'
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/apache-log4j-extras-1.2.17.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/jtds-1.2.5.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/log4j-1.2.17.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/msbase.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/mssqlserver.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/msutil.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/ojdbc6.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/slf4j-api-1.7.5.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/slf4j-nop-1.7.5.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/example/solr/lib/sqljdbc4.jar'
to classloader
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting socketTimeout to: 0
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting urlScheme to: http://
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting connTimeout to: 0
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting maxConnectionsPerHost to: 20
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting corePoolSize to: 0
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting maximumPoolSize to: 2147483647
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting maxThreadIdleTime to: 5
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting sizeOfQueue to: -1
Apr 04, 2014 3:49:44 PM
org.apache.solr.handler.component.HttpShardHandlerFactory getParameter
INFO: Setting fairnessPolicy to: false
Apr 04, 2014 3:49:44 PM org.apache.solr.client.solrj.impl.HttpClientUtil
createClient
INFO: Creating new http client,
config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
Apr 04, 2014 3:49:44 PM org.apache.solr.core.CoreContainer load
INFO: Registering Log Listener
Apr 04, 2014 3:49:44 PM org.apache.solr.core.CoreContainer create
INFO: Creating SolrCore 'core0' using instanceDir: solr\core0
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader <init>
INFO: new SolrResourceLoader for directory: 'solr\core0\'
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrConfig initLibs
INFO: Adding specified lib dirs to ClassLoader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/dist/solr-dataimporthandler-4.2.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/dist/solr-dataimporthandler-extras-4.2.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/apache-mime4j-core-0.7.2.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/apache-mime4j-dom-0.7.2.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/bcmail-jdk15-1.45.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/bcprov-jdk15-1.45.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/boilerpipe-1.1.0.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/commons-compress-1.4.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/dom4j-1.6.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/fontbox-1.7.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/icu4j-49.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/isoparser-1.0-RC-1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/jdom-1.0.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/jempbox-1.7.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/juniversalchardet-1.0.3.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/metadata-extractor-2.6.2.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/netcdf-4.2-min.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/pdfbox-1.7.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/poi-3.8.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/poi-ooxml-3.8.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/poi-ooxml-schemas-3.8.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/poi-scratchpad-3.8.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/rome-0.9.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/tagsoup-1.2.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/tika-core-1.3.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/tika-parsers-1.3.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/vorbis-java-core-0.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/vorbis-java-tika-0.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/xercesImpl-2.9.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/xmlbeans-2.3.0.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/extraction/lib/xz-1.0.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/clustering/lib/attributes-binder-1.0.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/clustering/lib/carrot2-mini-3.6.2.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/clustering/lib/hppc-0.4.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/clustering/lib/jackson-core-asl-1.7.4.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/clustering/lib/jackson-mapper-asl-1.7.4.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/clustering/lib/mahout-collections-1.0.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/clustering/lib/mahout-math-0.6.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/clustering/lib/simple-xml-2.6.4.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/langid/lib/jsonic-1.2.7.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/langid/lib/langdetect-1.1-20120112.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/velocity/lib/commons-beanutils-1.7.0.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/velocity/lib/commons-collections-3.2.1.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/velocity/lib/velocity-1.7.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/indexer/contrib/velocity/lib/velocity-tools-2.0.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/default/lib/common.jar' to
classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/default/lib/webapp.jar' to
classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrResourceLoader
replaceClassLoader
INFO: Adding
'file:/D:/AbcData/V12/applicationserver/server/default/lib/commons-pool-1.4.jar'
to classloader
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrConfig <init>
INFO: Using Lucene MatchVersion: LUCENE_42
Apr 04, 2014 3:49:44 PM org.apache.solr.core.SolrConfig <init>
INFO: Loaded SolrConfig: solrconfig.xml
Apr 04, 2014 3:49:44 PM org.apache.solr.schema.IndexSchema readSchema
INFO: Reading Solr Schema
Apr 04, 2014 3:49:44 PM org.apache.solr.schema.IndexSchema readSchema
INFO: Schema name=example
Apr 04, 2014 3:49:46 PM org.apache.solr.schema.IndexSchema readSchema
INFO: unique key field: id
Apr 04, 2014 3:49:47 PM org.apache.solr.schema.FileExchangeRateProvider
reload
INFO: Reloading exchange rates from file currency.xml
Apr 04, 2014 3:49:47 PM org.apache.solr.schema.FileExchangeRateProvider
reload
INFO: Reloading exchange rates from file currency.xml
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initDirectoryFactory
INFO: org.apache.solr.core.StandardDirectoryFactory
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore <init>
INFO: [core0] Opening new SolrCore at solr\core0\,
dataDir=D:\AbcData\V12\application
server\server\indexer\example\solr\core0\data\
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore <init>
INFO: JMX monitoring not detected for core: core0
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initListeners
INFO: [core0] Added SolrEventListener for newSearcher:
org.apache.solr.core.QuerySenderListener{queries=[]}
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initListeners
INFO: [core0] Added SolrEventListener for firstSearcher:
org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher
warming in solrconfig.xml}]}
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initDirectoryFactory
INFO: org.apache.solr.core.StandardDirectoryFactory
Apr 04, 2014 3:49:47 PM org.apache.solr.core.CachingDirectoryFactory get
INFO: return new directory for D:\AbcData\V12\application
server\server\indexer\example\solr\core0\data forceNew: false
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore getNewIndexDir
INFO: New index directory detected: old=null new=D:\AbcData\V12\application
server\server\indexer\example\solr\core0\data\index/
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initIndex
WARNING: [core0] Solr index directory 'D:\AbcData\V12\application
server\server\indexer\example\solr\core0\data\index' doesn't exist.
Creating new index...
Apr 04, 2014 3:49:47 PM org.apache.solr.core.CachingDirectoryFactory get
INFO: return new directory for D:\AbcData\V12\application
server\server\indexer\example\solr\core0\data\index forceNew: false
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=1
commit{dir=D:\AbcData\V12\application
server\server\indexer\example\solr\core0\data\index,segFN=segments_1,generation=1,filenames=[segments_1]
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1[segments_1]
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initWriters
INFO: created json: solr.JSONResponseWriter
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initWriters
INFO: adding lazy queryResponseWriter: solr.VelocityResponseWriter
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initWriters
INFO: created velocity: solr.VelocityResponseWriter
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore initWriters
INFO: created xslt: solr.XSLTResponseWriter
Apr 04, 2014 3:49:47 PM org.apache.solr.response.XSLTResponseWriter init
INFO: xsltCacheLifetimeSeconds=5
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /dataimport:
org.apache.solr.handler.dataimport.DataImportHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /select: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /query: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /browse: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /update: solr.XmlUpdateRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: adding lazy requestHandler: solr.extraction.ExtractingRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /update/extract: solr.extraction.ExtractingRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: adding lazy requestHandler: solr.FieldAnalysisRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /analysis/field: solr.FieldAnalysisRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: adding lazy requestHandler: solr.DocumentAnalysisRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /analysis/document: solr.DocumentAnalysisRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /admin/: solr.admin.AdminHandlers
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /admin/ping: solr.PingRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /debug/dump: solr.DumpRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: adding lazy requestHandler: solr.ReplicationHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /replication: solr.ReplicationHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: adding lazy requestHandler: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /spell: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: adding lazy requestHandler: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /tvrh: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: adding lazy requestHandler: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /terms: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: adding lazy requestHandler: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.core.RequestHandlers
initHandlersFromConfig
INFO: created /elevate: solr.SearchHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.handler.loader.XMLLoader init
INFO: xsltCacheLifetimeSeconds=60
Apr 04, 2014 3:49:47 PM org.apache.solr.handler.XmlUpdateRequestHandler init
WARNING: Using deprecated class: XmlUpdateRequestHandler -- replace with
UpdateRequestHandler
Apr 04, 2014 3:49:47 PM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@35f79e13 main
Apr 04, 2014 3:49:47 PM org.apache.solr.update.CommitTracker <init>
INFO: Hard AutoCommit: disabled
Apr 04, 2014 3:49:47 PM org.apache.solr.update.CommitTracker <init>
INFO: Soft AutoCommit: disabled
Apr 04, 2014 3:49:47 PM
org.apache.solr.handler.component.SpellCheckComponent inform
INFO: Initializing spell checkers
Apr 04, 2014 3:49:47 PM
org.apache.solr.handler.component.SpellCheckComponent inform
INFO: No queryConverter defined, using default converter
Apr 04, 2014 3:49:47 PM
org.apache.solr.handler.component.QueryElevationComponent inform
INFO: Loading QueryElevation from: D:\AbcData\V12\application
server\server\indexer\example\solr\core0\conf\elevate.xml
Apr 04, 2014 3:49:47 PM org.apache.solr.core.CoreContainer registerCore
INFO: registering core: core0
Apr 04, 2014 3:49:47 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener sending requests to
Searcher@35f79e13main{StandardDirectoryReader(segments_1:1)}
Apr 04, 2014 3:49:47 PM org.apache.solr.servlet.SolrDispatchFilter init
INFO: user.dir=D:\AbcData\V12\application server\server\indexer\example
Apr 04, 2014 3:49:47 PM org.apache.solr.servlet.SolrDispatchFilter init
INFO: SolrDispatchFilter.init() done
2014-04-04 15:49:47.722:INFO:oejs.AbstractConnector:Started
SocketConnector@0.0.0.0:8983
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore execute
INFO: [core0] webapp=null path=null
params={event=firstSearcher&q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false}
hits=0 status=0 QTime=28
Apr 04, 2014 3:49:47 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Apr 04, 2014 3:49:47 PM org.apache.solr.core.SolrCore registerSearcher
INFO: [core0] Registered new searcher
Searcher@35f79e13main{StandardDirectoryReader(segments_1:1)}
Apr 04, 2014 3:50:00 PM org.apache.solr.handler.dataimport.DataImporter
maybeReloadConfiguration
INFO: Loading DIH Configuration: database.xml
Apr 04, 2014 3:50:00 PM org.apache.solr.handler.dataimport.DataImporter
loadDataConfig
INFO: Data Configuration loaded successfully
Apr 04, 2014 3:50:00 PM org.apache.solr.handler.dataimport.DataImporter
doFullImport
INFO: Starting Full Import
Apr 04, 2014 3:50:00 PM org.apache.solr.core.SolrCore execute
INFO: [core0] webapp=/solr path=/dataimport
params={commit=true&command=full-import} status=0 QTime=12
Apr 04, 2014 3:50:00 PM
org.apache.solr.handler.dataimport.SimplePropertiesWriter
readIndexerProperties
INFO: Read dataimport.properties
Apr 04, 2014 3:50:00 PM org.apache.solr.update.DirectUpdateHandler2
deleteAll
INFO: [core0] REMOVING ALL DOCUMENTS FROM INDEX
Apr 04, 2014 3:50:00 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=D:\AbcData\V12\application
server\server\indexer\example\solr\core0\data\index,segFN=segments_1,generation=1,filenames=[segments_1]
Apr 04, 2014 3:50:00 PM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1[segments_1]
Apr 04, 2014 3:50:00 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Creating a connection for entity full-index with URL:
jdbc:oracle:thin:@test.abcdata.com:1521:ORCL
Apr 04, 2014 3:50:00 PM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Time taken for getConnection(): 702
MDFilter.class:::Message inserted for testing synch between compilation
and running application (Roberto Corona: 05/31/2013 4:02pm
parsepool.getNumActive(): 0
Parsepool.getNumIdle(): 0
creating new mdParse object
Apr 04, 2014 3:50:17 PM org.apache.solr.core.SolrCore execute
INFO: [core0] webapp=/solr path=/dataimport
params={commit=true&command=full-import} status=0 QTime=0
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid97000.hprof ...
Heap dump file created [62540489 bytes in 0.169 secs]
Apr 04, 2014 3:50:26 PM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: [core0] webapp=/solr path=/dataimport
params={commit=true&command=full-import} status=0 QTime=12
{deleteByQuery=*:*,add=[1-5S, 1-8G, 1+DH+740, 1+DH+741, 1+DH+744, 1+DH+745,
1+DH+747, 1+DH+749, 1+DH+752, 1+DH+753, ... (4514 adds)]} 0 12
Apr 04, 2014 3:50:26 PM org.apache.solr.common.SolrException log
SEVERE: Full Import failed:java.lang.RuntimeException:
java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:266)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:422)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468)
Caused by: java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:406)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:319)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:227)
... 3 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:535)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:404)
... 5 more
Caused by: java.lang.OutOfMemoryError: Java heap space
at
org.apache.lucene.util.ByteBlockPool$DirectTrackingAllocator.getByteBlock(ByteBlockPool.java:103)
at
org.apache.lucene.util.ByteBlockPool.nextBuffer(ByteBlockPool.java:202)
at
org.apache.lucene.util.ByteBlockPool.allocSlice(ByteBlockPool.java:257)
at
org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:252)
at
org.apache.lucene.index.TermsHashPerField.writeVInt(TermsHashPerField.java:273)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.writeProx(FreqProxTermsWriterPerField.java:145)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTermsWriterPerField.java:227)
at
org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:235)
at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:165)
at
org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:254)
at
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:256)
at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:376)
at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1473)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:201)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:477)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:346)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:70)
at
org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:235)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:500)
... 6 more
Apr 04, 2014 3:50:26 PM org.apache.solr.update.DirectUpdateHandler2 rollback
INFO: start rollback{}
Apr 04, 2014 3:50:26 PM org.apache.solr.update.DefaultSolrCoreState
newIndexWriter
INFO: Creating new IndexWriter...
Apr 04, 2014 3:50:26 PM org.apache.solr.update.DefaultSolrCoreState
newIndexWriter
INFO: Waiting until IndexWriter is unused... core=core0
Apr 04, 2014 3:50:26 PM org.apache.solr.update.DefaultSolrCoreState
newIndexWriter
INFO: Rollback old IndexWriter... core=core0
Apr 04, 2014 3:50:26 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=D:\AbcData\V12\application
server\server\indexer\example\solr\core0\data\index,segFN=segments_1,generation=1,filenames=[segments_1]
Apr 04, 2014 3:50:26 PM org.apache.solr.core.SolrDeletionPolicy
updateCommits
INFO: newest commit = 1[segments_1]
Apr 04, 2014 3:50:26 PM org.apache.solr.update.DefaultSolrCoreState
newIndexWriter
INFO: New IndexWriter is ready to be used.
Apr 04, 2014 3:50:26 PM org.apache.solr.update.DirectUpdateHandler2 rollback
INFO: end_rollback
On Fri, Apr 4, 2014 at 4:59 PM, Candygram For Mongo <
candygram.for.mo...@gmail.com> wrote:
> Guessing that the attachments won't work, I am pasting one file in each of
> four separate emails.
>
> database.xml
>
>
> <dataConfig>
> <dataSource
> name="org_only"
> type="JdbcDataSource"
> driver="oracle.jdbc.OracleDriver"
> url="jdbc:oracle:thin:@test.abcdata.com:1521:ORCL"
> user="admin"
> password="admin"
> readOnly="false"
> />
> <document>
>
>
> <entity name="full-index" query="
> select
>
> NVL(cast(ORACLE.ADDRESS_ALL.RECORD_ID as varchar2(100)), 'null')
> as SOLR_ID,
>
> 'ORACLE.ADDRESS_ALL'
> as SOLR_CATEGORY,
>
> NVL(cast(ORACLE.ADDRESS_ALL.RECORD_ID as varchar2(255)), ' ') as
> ADDRESSALLROWID,
> NVL(cast(ORACLE.ADDRESS_ALL.ADDR_TYPE_CD as varchar2(255)), ' ') as
> ADDRESSALLADDRTYPECD,
> NVL(cast(ORACLE.ADDRESS_ALL.LONGITUDE as varchar2(255)), ' ') as
> ADDRESSALLLONGITUDE,
> NVL(cast(ORACLE.ADDRESS_ALL.LATITUDE as varchar2(255)), ' ') as
> ADDRESSALLLATITUDE,
> NVL(cast(ORACLE.ADDRESS_ALL.ADDR_NAME as varchar2(255)), ' ') as
> ADDRESSALLADDRNAME,
> NVL(cast(ORACLE.ADDRESS_ALL.CITY as varchar2(255)), ' ') as ADDRESSALLCITY,
> NVL(cast(ORACLE.ADDRESS_ALL.STATE as varchar2(255)), ' ') as
> ADDRESSALLSTATE,
> NVL(cast(ORACLE.ADDRESS_ALL.EMAIL_ADDR as varchar2(255)), ' ') as
> ADDRESSALLEMAILADDR
>
> from ORACLE.ADDRESS_ALL
> " >
>
> <field column="SOLR_ID" name="id" />
> <field column="SOLR_CATEGORY" name="category" />
> <field column="ADDRESSALLROWID" name="ADDRESS_ALL.RECORD_ID_abc" />
> <field column="ADDRESSALLADDRTYPECD" name="ADDRESS_ALL.ADDR_TYPE_CD_abc" />
> <field column="ADDRESSALLLONGITUDE" name="ADDRESS_ALL.LONGITUDE_abc" />
> <field column="ADDRESSALLLATITUDE" name="ADDRESS_ALL.LATITUDE_abc" />
> <field column="ADDRESSALLADDRNAME" name="ADDRESS_ALL.ADDR_NAME_abc" />
> <field column="ADDRESSALLCITY" name="ADDRESS_ALL.CITY_abc" />
> <field column="ADDRESSALLSTATE" name="ADDRESS_ALL.STATE_abc" />
> <field column="ADDRESSALLEMAILADDR" name="ADDRESS_ALL.EMAIL_ADDR_abc" />
>
> </entity>
>
>
>
> <!-- Varaibles -->
> <!-- '${dataimporter.last_index_time}' -->
> </document>
> </dataConfig>
>
>
>
> On Fri, Apr 4, 2014 at 4:57 PM, Candygram For Mongo <
> candygram.for.mo...@gmail.com> wrote:
>
>> Does this user list allow attachments? I have four files attached
>> (database.xml, error.txt, schema.xml, solrconfig.xml). We just ran the
>> process again using the parameters you suggested, but not to a csv file.
>> It errored out quickly. We are working on the csv file run.
>>
>> Removed both <autoCommit> and <autoSoftCommit> parts/definitions from
>> solrconfig.xml
>>
>> Disabled tlog by removing
>>
>> <updateLog>
>> <str name="dir">${solr.ulog.dir:}</str>
>> </updateLog>
>>
>> from solrconfig.xml
>>
>> Used commit=true parameter. ?commit=true&command=full-import
>>
>>
>> On Fri, Apr 4, 2014 at 3:29 PM, Ahmet Arslan <iori...@yahoo.com> wrote:
>>
>>> Hi,
>>>
>>> This may not solve your problem but generally it is recommended to
>>> disable auto commit and transaction logs for bulk indexing.
>>> And issue one commit at the very end. Do you tlogs enabled? I see
>>> "commit failed" in the error message thats why I am offering this.
>>>
>>> And regarding comma separated values, with this approach you focus on
>>> just solr importing process. You separate data acquisition phrase. And it
>>> is very fast load even big csv files
>>> http://wiki.apache.org/solr/UpdateCSV
>>> I have never experienced OOM during indexing, I suspect data acquisition
>>> has role in it.
>>>
>>> Ahmet
>>>
>>> On Saturday, April 5, 2014 1:18 AM, Candygram For Mongo <
>>> candygram.for.mo...@gmail.com> wrote:
>>>
>>> We would be happy to try that. That sounds counter intuitive for the
>>> high volume of records we have. Can you help me understand how that might
>>> solve our problem?
>>>
>>>
>>>
>>>
>>> On Fri, Apr 4, 2014 at 2:34 PM, Ahmet Arslan <iori...@yahoo.com> wrote:
>>>
>>> Hi,
>>> >
>>> >Can you remove auto commit for bulk import. Commit at the very end?
>>> >
>>> >Ahmet
>>> >
>>> >
>>> >
>>> >
>>> >On Saturday, April 5, 2014 12:16 AM, Candygram For Mongo <
>>> candygram.for.mo...@gmail.com> wrote:
>>> >In case the attached database.xml file didn't show up, I have pasted in
>>> the
>>> >contents below:
>>> >
>>> ><dataConfig>
>>> ><dataSource
>>> >name="org_only"
>>> >type="JdbcDataSource"
>>> >driver="oracle.jdbc.OracleDriver"
>>> >url="jdbc:oracle:thin:@test2.abc.com:1521:ORCL"
>>> >user="admin"
>>> >password="admin"
>>> >readOnly="false"
>>> >batchSize="100"
>>> >/>
>>> ><document>
>>> >
>>> >
>>> ><entity name="full-index" query="
>>> >select
>>> >
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.RECORD_ID as varchar2(100)), 'null')
>>> >as SOLR_ID,
>>> >
>>> >'ORCL.ADDRESS_ACCT_ALL'
>>> >as SOLR_CATEGORY,
>>> >
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.RECORD_ID as varchar2(255)), ' ') as
>>> >ADDRESSALLROWID,
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.ADDR_TYPE_CD as varchar2(255)), ' ') as
>>> >ADDRESSALLADDRTYPECD,
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.LONGITUDE as varchar2(255)), ' ') as
>>> >ADDRESSALLLONGITUDE,
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.LATITUDE as varchar2(255)), ' ') as
>>> >ADDRESSALLLATITUDE,
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.ADDR_NAME as varchar2(255)), ' ') as
>>> >ADDRESSALLADDRNAME,
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.CITY as varchar2(255)), ' ') as
>>> >ADDRESSALLCITY,
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.STATE as varchar2(255)), ' ') as
>>> >ADDRESSALLSTATE,
>>> >NVL(cast(ORCL.ADDRESS_ACCT_ALL.EMAIL_ADDR as varchar2(255)), ' ') as
>>> >ADDRESSALLEMAILADDR
>>> >
>>> >from ORCL.ADDRESS_ACCT_ALL
>>> >" >
>>> >
>>> ><field column="SOLR_ID" name="id" />
>>> ><field column="SOLR_CATEGORY" name="category" />
>>> ><field column="ADDRESSALLROWID" name="ADDRESS_ACCT_ALL.RECORD_ID_abc" />
>>> ><field column="ADDRESSALLADDRTYPECD"
>>> >name="ADDRESS_ACCT_ALL.ADDR_TYPE_CD_abc" />
>>> ><field column="ADDRESSALLLONGITUDE"
>>> name="ADDRESS_ACCT_ALL.LONGITUDE_abc" />
>>> ><field column="ADDRESSALLLATITUDE" name="ADDRESS_ACCT_ALL.LATITUDE_abc"
>>> />
>>> ><field column="ADDRESSALLADDRNAME"
>>> name="ADDRESS_ACCT_ALL.ADDR_NAME_abc" />
>>> ><field column="ADDRESSALLCITY" name="ADDRESS_ACCT_ALL.CITY_abc" />
>>> ><field column="ADDRESSALLSTATE" name="ADDRESS_ACCT_ALL.STATE_abc" />
>>> ><field column="ADDRESSALLEMAILADDR"
>>> name="ADDRESS_ACCT_ALL.EMAIL_ADDR_abc"
>>> >/>
>>> >
>>> ></entity>
>>> >
>>> >
>>> >
>>> ><!-- Varaibles -->
>>> ><!-- '${dataimporter.last_index_time}' -->
>>> ></document>
>>> ></dataConfig>
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >On Fri, Apr 4, 2014 at 11:55 AM, Candygram For Mongo <
>>> >candygram.for.mo...@gmail.com> wrote:
>>> >
>>> >> In this case we are indexing an Oracle database.
>>> >>
>>> >> We do not include the data-config.xml in our distribution. We store
>>> the
>>> >> database information in the database.xml file. I have attached the
>>> >> database.xml file.
>>> >>
>>> >> When we use the default merge policy settings, we get the same
>>> results.
>>> >>
>>> >>
>>> >>
>>> >> We have not tried to dump the table to a comma separated file. We
>>> think
>>> >> that dumping this size table to disk will introduce other memory
>>> problems
>>> >> with big file management. We have not tested that case.
>>> >>
>>> >>
>>> >> On Fri, Apr 4, 2014 at 7:25 AM, Ahmet Arslan <iori...@yahoo.com>
>>> wrote:
>>> >>
>>> >>> Hi,
>>> >>>
>>> >>> Which database are you using? Can you send us data-config.xml?
>>> >>>
>>> >>> What happens when you use default merge policy settings?
>>> >>>
>>> >>> What happens when you dump your table to Comma Separated File and fed
>>> >>> that file to solr?
>>> >>>
>>> >>> Ahmet
>>> >>>
>>> >>> On Friday, April 4, 2014 5:10 PM, Candygram For Mongo <
>>> >>> candygram.for.mo...@gmail.com> wrote:
>>> >>>
>>> >>> The ramBufferSizeMB was set to 6MB only on the test system to make
>>> the
>>> >>> system crash sooner. In production that tag is commented out which
>>> >>> I believe forces the default value to be used.
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Thu, Apr 3, 2014 at 5:46 PM, Ahmet Arslan <iori...@yahoo.com>
>>> wrote:
>>> >>>
>>> >>> Hi,
>>> >>> >
>>> >>> >out of curiosity, why did you set ramBufferSizeMB to 6?
>>> >>> >
>>> >>> >Ahmet
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >On Friday, April 4, 2014 3:27 AM, Candygram For Mongo <
>>> >>> candygram.for.mo...@gmail.com> wrote:
>>> >>> >*Main issue: Full Indexing is Causing a Java Heap Out of Memory
>>> Exception
>>> >>> >
>>> >>> >*SOLR/Lucene version: *4.2.1*
>>> >>> >
>>> >>> >
>>> >>> >*JVM version:
>>> >>> >
>>> >>> >Java(TM) SE Runtime Environment (build 1.7.0_07-b11)
>>> >>> >
>>> >>> >Java HotSpot(TM) 64-Bit Server VM (build 23.3-b01, mixed mode)
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >*Indexer startup command:
>>> >>> >
>>> >>> >set JVMARGS=-XX:MaxPermSize=364m -Xss256K -Xmx6144m -Xms6144m
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >java " %JVMARGS% ^
>>> >>> >
>>> >>> >-Dcom.sun.management.jmxremote.port=1092 ^
>>> >>> >
>>> >>> >-Dcom.sun.management.jmxremote.ssl=false ^
>>> >>> >
>>> >>> >-Dcom.sun.management.jmxremote.authenticate=false ^
>>> >>> >
>>> >>> >-jar start.jar
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >*SOLR indexing HTTP parameters request:
>>> >>> >
>>> >>> >webapp=/solr path=/dataimport
>>> >>> >params={clean=false&command=full-import&wt=javabin&version=2}
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >We are getting a Java heap OOM exception when indexing (updating) 27
>>> >>> >million records. If we increase the Java heap memory settings the
>>> >>> problem
>>> >>> >goes away but we believe the problem has not been fixed and that we
>>> will
>>> >>> >eventually get the same OOM exception. We have other processes on
>>> the
>>> >>> >server that also require resources so we cannot continually
>>> increase the
>>> >>> >memory settings to resolve the OOM issue. We are trying to find a
>>> way to
>>> >>> >configure the SOLR instance to reduce or preferably eliminate the
>>> >>> >possibility of an OOM exception.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >We can reproduce the problem on a test machine. We set the Java
>>> heap
>>> >>> >memory size to 64MB to accelerate the exception. If we increase
>>> this
>>> >>> >setting the same problems occurs, just hours later. In the test
>>> >>> >environment, we are using the following parameters:
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >JVMARGS=-XX:MaxPermSize=64m -Xss256K -Xmx64m -Xms64m
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >Normally we use the default solrconfig.xml file with only the
>>> following
>>> >>> jar
>>> >>> >file references added:
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> ><lib path="../../../../default/lib/common.jar" />
>>> >>> >
>>> >>> ><lib path="../../../../default/lib/webapp.jar" />
>>> >>> >
>>> >>> ><lib path="../../../../default/lib/commons-pool-1.4.jar" />
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >Using these values and trying to index 6 million records from the
>>> >>> database,
>>> >>> >the Java Heap Out of Memory exception is thrown very quickly.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >We were able to complete a successful indexing by further modifying
>>> the
>>> >>> >solrconfig.xml and removing all or all but one <copyfield> tags
>>> from the
>>> >>> >schema.xml file.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >The following solrconfig.xml values were modified:
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> ><ramBufferSizeMB>6</ramBufferSizeMB>
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> ><mergePolicy class="org.apache.lucene.index.TieredMergePolicy">
>>> >>> >
>>> >>> ><int name="maxMergeAtOnce">2</int>
>>> >>> >
>>> >>> ><int name="maxMergeAtOnceExplicit">2</int>
>>> >>> >
>>> >>> ><int name="segmentsPerTier">10</int>
>>> >>> >
>>> >>> ><int name="maxMergedSegmentMB">150</int>
>>> >>> >
>>> >>> ></mergePolicy>
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> ><autoCommit>
>>> >>> >
>>> >>> ><maxDocs>15000</maxDocs> <!-- This tag was maxTime, before
>>> this -- >
>>> >>> >
>>> >>> ><openSearcher>false</openSearcher>
>>> >>> >
>>> >>> ></autoCommit>
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >Using our customized schema.xml file with two or more <copyfield>
>>> tags,
>>> >>> the
>>> >>> >OOM exception is always thrown. Based on the errors, the problem
>>> occurs
>>> >>> >when the process was trying to do the merge. The error is provided
>>> >>> below:
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >Exception in thread "Lucene Merge Thread #156"
>>> >>> >org.apache.lucene.index.MergePolicy$MergeException:
>>> >>> >java.lang.OutOfMemoryError: Java heap space
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:541)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:514)
>>> >>> >
>>> >>> >Caused by: java.lang.OutOfMemoryError: Java heap space
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.codecs.lucene42.Lucene42DocValuesProducer.loadNumeric(Lucene42DocValuesProducer.java:180)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.codecs.lucene42.Lucene42DocValuesProducer.getNumeric(Lucene42DocValuesProducer.java:146)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.index.SegmentCoreReaders.getNormValues(SegmentCoreReaders.java:301)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.index.SegmentReader.getNormValues(SegmentReader.java:259)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:233)
>>> >>> >
>>> >>> > at
>>> >>> >org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:137)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3693)
>>> >>> >
>>> >>> > at
>>> >>> >org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3296)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:401)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:478)
>>> >>> >
>>> >>> >Mar 12, 2014 12:17:40 AM org.apache.solr.common.SolrException log
>>> >>> >
>>> >>> >SEVERE: auto commit error...:java.lang.IllegalStateException: this
>>> writer
>>> >>> >hit an OutOfMemoryError; cannot commit
>>> >>> >
>>> >>> > at
>>> >>>
>>> >org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:3971)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2744)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2827)
>>> >>> >
>>> >>> > at
>>> >>> >org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2807)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:536)
>>> >>> >
>>> >>> > at
>>> >>> >org.apache.solr.update.CommitTracker.run(CommitTracker.java:216)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> >>> >
>>> >>> > at
>>> >>> >java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>>> >>> >
>>> >>> > at
>>> >>> java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>> >>> >
>>> >>> > at
>>> >>>
>>> >>>
>>> >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>> >>> >
>>> >>> > at java.lang.Thread.run(Thread.java:722)
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >We think but are not 100% sure that the problem is related to the
>>> merge.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >Normally our schema.xml contains a lot of field specifications
>>> (like the
>>> >>> >ones seen in the file fragment below):
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> ><copyField source="ADDRESS.RECORD_ID_abc"
>>> >>> dest="ADDRESS.RECORD_ID.case_abc"
>>> >>> >/>
>>> >>> >
>>> >>> ><copyField source="ADDRESS.RECORD_ID_abc"
>>> >>> >dest="ADDRESS.RECORD_ID.case.soundex_abc" />
>>> >>> >
>>> >>> ><copyField source="ADDRESS.RECORD_ID_abc"
>>> >>> >dest="ADDRESS.RECORD_ID.case_nvl_abc" />
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >In tests using the default file schema.xml and no <copyfield> tags,
>>> >>> >indexing completed successfully. 6 million records produced a 900
>>> MB
>>> >>> data
>>> >>> >directory.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >When I included just one <copyfield> tag, indexing completed
>>> >>> successfully. 6
>>> >>> >million records produced a 990 MB data directory (90 MB bigger).
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >When I included just two <copyfield> tags, the index crashed with
>>> an OOM
>>> >>> >exception.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >Changing parameters like maxMergedSegmentMB or maxDocs, only
>>> postponed
>>> >>> the
>>> >>> >crash.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >The net of our test results I as follows:
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >*solrconfig.xml*
>>> >>> >
>>> >>> >*schema.xml*
>>> >>> >
>>> >>> >*result*
>>> >>> >
>>> >>> >
>>> >>> >default plus only jar references
>>> >>> >
>>> >>> >default (no copyfield tags)
>>> >>> >
>>> >>> >success
>>> >>> >
>>> >>> >default plus only jar references
>>> >>> >
>>> >>> >modified with one copyfield tag
>>> >>> >
>>> >>> >success
>>> >>> >
>>> >>> >default plus only jar references
>>> >>> >
>>> >>> >modified with two copyfield tags
>>> >>> >
>>> >>> >crash
>>> >>> >
>>> >>> >additional modified settings
>>> >>> >
>>> >>> >default (no copyfield tags)
>>> >>> >
>>> >>> >success
>>> >>> >
>>> >>> >additional modified settings
>>> >>> >
>>> >>> >modified with one copyfield tag
>>> >>> >
>>> >>> >success
>>> >>> >
>>> >>> >additional modified settings
>>> >>> >
>>> >>> >modified with two copyfield tags
>>> >>> >
>>> >>> >crash
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> >Our question is, what can we do to eliminate these OOM exceptions?
>>> >>> >
>>> >>> >
>>> >>>
>>> >>
>>> >>
>>> >
>>> >
>>>
>>
>>
>
