Can you give it a shot on Solr 1.4 instead? DIH has had numerous
enhancements/fixes since 1.3.
Erik
On Feb 1, 2010, at 8:42 AM, Vijayant Kumar wrote:
Hi,
I am trying to index some large set of data in solr using
dataimporthandler.
It is working fine for small set but when I am trying to index on
large
set it produces error.
I am using solr version 1.3 and mysql version Ver 14.7 Distrib
4.1.20,
for
redhat-linux-gnu (i686)
Earlier I was facing java heap space error
Exception in thread "Thread-16" java.lang.OutOfMemoryError: Java heap
space
Get it solve by allocating java -Xmx512M -Xms512M -jar start.jar
But Now it is giving mysql communication link failure error
com.mysql.jdbc.CommunicationsException: Communications link failure
due to
underlying exception:
** BEGIN NESTED EXCEPTION **
java.io.EOFException
STACKTRACE:
java.io.EOFException
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1913)
I had added transactionIsolation="TRANSACTION_READ_COMMITTED"
holdability="CLOSE_CURSORS_AT_COMMIT" in this schema as suggested
by solr
wiki page
but it is still not working.
Here is my data-config.xml file
<?xml version="1.0" encoding="UTF-8" ?>
<dataConfig>
<dataSource name="jdbc" driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/databasename"
user="xxx" password="****" batchSize="-1" readOnly="true"
autoCommit="false" transactionIsolation="TRANSACTION_READ_COMMITTED"
holdability="CLOSE_CURSORS_AT_COMMIT"/>
<document name="feature">
<entity name="posts" dataSource="jdbc" pk="pid" query="
SELECT uid, pid, message FROM xxx"
transformer="TemplateTransformer">
<add>
<doc>
<field column = "pid" name="pid"/>
<field column = "uid" name="uid"/>
<field column = "message" name="message"/>
</doc>
</add>
</entity>
</document>
</dataConfig>
My table contain around 1.5 core of data on local machine On
production it
is
containing around 4 times of data as on local.
Thank you,
Vijayant Kumar
Software Engineer
Website Toolbox Inc.
http://www.websitetoolbox.com
1-800-921-7803 x211
--
Thank you,
Vijayant Kumar
Software Engineer
Website Toolbox Inc.
http://www.websitetoolbox.com
1-800-921-7803 x211
