Hi,

I am trying to load a big table into Solr using DataImportHandler and Mysql. 
I am getting OutOfMemory error because Solr is trying to load the full
table. I have been reading different posts and tried batchSize="-1". 
https://wiki.apache.org/solr/DataImportHandlerFaq

Do you have any idea what could be the issue?
Completely lost here.

Solr.6.4.1
mysql-connector-java-5.1.41-bin.jar

data-config 

<dataSource type="JdbcDataSource" 
            driver="com.mysql.jdbc.Driver"
            url="jdbc:mysql://188.68.190.85:3306/jobsdb" 
            user="suer" 
            password="passowrd"/>
<document>
  <entity name="jobsearch"  
    pk="id"
        batchSize="-1"
    query="select * from job"
    deltaImportQuery="SELECT * from job WHERE id='${dih.delta.id}'"
    deltaQuery="SELECT id FROM job  WHERE updated_at >
'${dih.last_index_time}'"
    >
     <field column="job_id" name="JobID"/>
     <field column="position" name="Position"/>    
     <field column="employment_type" name="EmploymentType"/>    
         <field column="description" name="Description"/>
     <field column="category" name="Category"/>  
         <field column="apply_url" name="ApplyUrl"/>
     <field column="description_url" name="DescriptionUrl"/>    
         <field column="company" name="Company"/>       
         <field column="city" name="City"/>     
         <field column="country_subdivision1" name="CountrySubdivision1"/>      
         <field column="country_subdivision2" name="CountrySubdivision2"/>      
         <field column="country" name="Country"/>       
     <field column="source" name="Source"/>
         <field column="created_at" name="CreatedAt"/>
         <field column="updated_at" name="UpdatedAt"/>    
  </entity>
</document>
</dataConfig>

Thanks
Sergio



--
View this message in context: 
http://lucene.472066.n3.nabble.com/DataImportHandler-OutOfMemory-Mysql-tp4327982.html
Sent from the Solr - User mailing list archive at Nabble.com.

Reply via email to