Hi, I have a beefy box with 24Gb RAM (12GB for Tomcat7 which houses SOLR3.6) 2 Processors Intel Xeon 64 bit Server, 30TB HDD. JDK 1.7.0_03 x64 bit
Data Index Dir Size: 400GB Metadata of files is stored in it. I have around 15 schema fields. Total number of items:150million approx. I have a scenario which I will try to explain to the best of my knowledge here: Let us consider the fields I am interested in Url: Entire path of a file in windows file system including the filename. ex:C:\Documents\A.txt mtm: Modified Time of the file Jid:JOb ID conv_sort is string field type where the filename is stored. I run a job where the following gets inserted Total Items:2 Url:C:\personal\A1.txt mtm:08/14/2012 12:00:00 Jid:1 Conv_sort:A1.txt ----------------------------------- Url:C:\personal\B1.txt mtm:08/14/2012 12:01:00 Jid:1 Conv_sort:B1.txt In the second run only one item changes: Url:C:\personal\A1.txt mtm:08/15/2012 1:00:00 Jid:2 Conv_sort=A1.txt When queried I would like to return the latest A1.txt and B1.txt back to the end user. I am trying to use grouping with no luck. It keeps throwing OOM… can someone please help… as it is critical for my project The query I am trying is under a folder there are 1000 files and I putting a filtered query param too asking it to group by filenames or url and none of them work…what am I doing wrong here http://172.19.108.78:8080/solr/select/?q=*:*&version=2.2&start=0&rows=10&indent=on&group.query=filefolder:"E\:\\pd_dst\\646c6907-a948-4b83-ac1d-d44742bb0307<smb://pd_dst//646c6907-a948-4b83-ac1d-d44742bb0307>"&group=true&group.limit=1&group.field=conv_sort&group.ngroup=true The stack trace: SEVERE: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOfRange(Unknown Source) at java.lang.String.<init>(Unknown Source) at org.apache.lucene.index.TermBuffer.toTerm(TermBuffer.java:122) at org.apache.lucene.index.SegmentTermEnum.term(SegmentTermEnum.java:184 ) at org.apache.lucene.search.FieldCacheImpl$StringIndexCache.createValue( FieldCacheImpl.java:882) at org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java :233) at org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl .java:856) at org.apache.lucene.search.grouping.TermFirstPassGroupingCollector.setN extReader(TermFirstPassGroupingCollector.java:74) at org.apache.lucene.search.MultiCollector.setNextReader(MultiCollector. java:113) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:576) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:364) at org.apache.solr.search.Grouping.searchWithTimeLimiter(Grouping.java:3 76) at org.apache.solr.search.Grouping.execute(Grouping.java:298) at org.apache.solr.handler.component.QueryComponent.process(QueryCompone nt.java:372) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(Sea rchHandler.java:186) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl erBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter .java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte r.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl icationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF ilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperV alve.java:225) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextV alve.java:123) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authentica torBase.java:472) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.j ava:168) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.j ava:98) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java: 927) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineVal ve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.jav a:407) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp 11Processor.java:1001) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process( AbstractProtocol.java:585) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoin t.java:1770) ******************Legal Disclaimer*************************** "This communication may contain confidential and privileged material for the sole use of the intended recipient. Any unauthorized review, use or distribution by others is strictly prohibited. If you have received the message in error, please advise the sender by reply email and delete the message. Thank you." *********************************************************
