Shawn, Please find below the resulset size of each query,
select objectuid as uid, objectid, objecttype, objectname, repositoryname, a.lastupdateddate from MetaModel.POC.Object a, MetaModel.POC.Repository b where a.repositoryid = b.repositoryid ---> 300000 rows ---> Query 1 select ObjectUID,ObjectPropertyName as name, ObjectPropertyValue as value from MetaModel.POC.ObjectProperty --> 600000 rows --> Query 2 select OBJECT1uid,Object2name as rname,Object2type as rtype,relationshiptype as rship, b.RepositoryName as rrepname from MetaModel.POC.BinaryRelationShip a, MetaModel.POC.Repository b where a.Object2RepositoryId=b.repositoryId" --> 600 rows --> Query 3 I want my second and third query to run for every row returned by my first query without hitting the DB for every loop hence started using SqlCachedEntityProcessor. Please find below RAM stats of my server running SOLR / MetaMatrix DB SOLR - 100,176 K MetaMatrix DB - 1106048 K almost 1080.125 MB Thanks, Barani Shawn Heisey-4 wrote: > > Do keep looking into the batchSize, but I think I might have found the > issue. If I understand things correctly, you will need to add > processor="CachedSqlEntityProcessor" to your first entity. It's only > specified on the other two. Assuming you have enough RAM and heap space > available in your JVM to load the results of all three queries, that > ought to make it work very quickly. > > If I'm right, basically what it's doing is issuing a real SQL query > against your first table for every entry it has read for the other two > tables. > > Shawn > > On 3/6/2010 11:58 AM, JavaGuy84 wrote: >> Shawn, >> >> Thanks a lot for your response, >> >> Yes, still the DB connection is active.. It is still fetching the data >> from >> the DB. >> >> I am using Redhat MetaMatrix DB as backend and I am trying to find out >> the >> parameter for setting the JDBC fetch size.. >> >> Do you think that this problem will be mostly due to fetch size? >> >> Thanks, >> Barani >> > > > -- View this message in context: http://old.nabble.com/SOLR-takes-more-than-9-hours-to-index-300000-rows-tp27805403p27806552.html Sent from the Solr - User mailing list archive at Nabble.com.
