In my datasource config file:
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://127.0.0.1:3306/MYSOLR?useUnicode=true&characterEncoding=UTF-8"
user="root" password="qwertyuiop" batchSize="500" />
i have done it,set batchSize="500"
On Fri, Jun 1, 2012 at 10:38 PM, Dyer, James <james.d...@ingrambook.com> wrote:
> I see this in your stacktrace: java.sql.SQLException: Illegal value for
> setFetchSize().
>
> It must be that your JDBC driver doesn't like the default value (300) that is
> used. In your datasource tag, try adding a "batchSize" attribute of either 0
> or -1 (if using -1, DIH automatically changes it to "Integer.MIN_VALUE".
> According to the wiki this is to fix this error.) The value of "batchSize"
> is used on the java.sql.Statement objects with "setFetchSize(batchSize)".
>
> example:
> <dataSource ... batchSize="0" />
>
> see: http://wiki.apache.org/solr/DataImportHandler#Configuring_JdbcDataSource
>
> James Dyer
> E-Commerce Systems
> Ingram Content Group
> (615) 213-4311
>
>
> -----Original Message-----
> From: wangjing [mailto:ppm10...@gmail.com]
> Sent: Friday, June 01, 2012 9:18 AM
> To: solr-user@lucene.apache.org
> Subject: why DIH works in normal mode,error in debug mode
>
> why DIH works in normal mode,use
> http://localhost:8080/apache-solr-3.6.0/cn/admin/
> query *:* it can find all
>
> 0 35 0 2.2 *:* on 10 测试类型1
> 台灯是人们生活中用来照明的一种家用电器。它一般分为两种，一种是立柱式的，一种是有夹子的。它的工作原理主要是把灯光集中在一小块区域内，便于工作和学习。一般台灯用的灯泡是白炽灯或者节能灯泡。
> 有的台灯还有应急功能，用于停电时无电照明已用来应急。 包含灯泡啥的 1 2012-06-01T06:15:58Z 台灯 法国台灯A234 3
> 100.2 100.2,USD 包含灯泡啥的法国台灯A234台灯
> 台灯是人们生活中用来照明的一种家用电器。它一般分为两种，一种是立柱式的，一种是有夹子的。它的工作原理主要是把灯光集中在一小块区域内，便于工作和学习。一般台灯用的灯泡是白炽灯或者节能灯泡。
> 有的台灯还有应急功能，用于停电时无电照明已用来应急。 10.0
>
>
> BUT ERROR in debug mode,it boring me few days :_(
>
> the detail exception statck is
>
> 2012-6-1 21:51:51 org.apache.solr.common.SolrException log
> ????: Full Import failed:java.lang.RuntimeException:
> java.lang.RuntimeException:
> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable
> to execute query: select * from ITEM; Processing Document # 1
> at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:264)
> at
> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)
> at
> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)
> at
> org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:205)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:999)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.RuntimeException:
> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable
> to execute query: select * from ITEM; Processing Document # 1
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:621)
> at
> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:327)
> at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:225)
> ... 23 more
> Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException:
> Unable to execute query: select * from ITEM; Processing Document # 1
> at
> org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:72)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:253)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:210)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:39)
> at
> org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:188)
> at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
> at
> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
> at
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.pullRow(EntityProcessorWrapper.java:330)
> at
> org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:296)
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:683)
> at
> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:619)
> ... 25 more
> Caused by: java.sql.SQLException: Illegal value for setFetchSize().
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
> at com.mysql.jdbc.StatementImpl.setFetchSize(StatementImpl.java:2461)
> at
> org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:242)
> ... 34 more
>
> 2012-6-1 21:51:51 org.apache.solr.update.DirectUpdateHandler2 rollback
> ??Ϣ: start rollback
> 2012-6-1 21:51:51 org.apache.solr.update.DirectUpdateHandler2 rollback
> ??Ϣ: end_rollback
> 2012-6-1 21:51:51 org.apache.solr.core.SolrCore execute
> ??Ϣ: [cn] webapp=/apache-solr-3.6.0 path=/dataimport
> params={start=0&dataConfig=<dataConfig>%0d%0a%09<dataSource+type%3D"JdbcDataSource"+driver%3D"com.mysql.jdbc.Driver"%0d%0a%09%09url%3D"jdbc:mysql://127.0.0.1:3306/MYSOLR?useUnicode%3Dtrue%26amp;characterEncoding%3DUTF-8"%0d%0a%09%09user%3D"root"+password%3D"qwertyuiop"+batchSize%3D"500"+/>%0d%0a%0d%0a%09<document+datasource%3D"JdbcDataSource"+name%3D"item_document">%0d%0a%09%09<entity++name%3D"item"+pk%3D"ITEM_ID"+query%3D"select+*+from+ITEM;"%0d%0a%09%09%09deltaQuery%3D"select+ITEM_ID+from+ITEM+where+LAST_MODIFIED+>+'${dataimporter.last_index_time}';">%0d%0a%09%09%09<field+column%3D"ITEM_ID"+name%3D"item_id"+/>%0d%0a%09%09%09<field+column%3D"NAME"+name%3D"name"+/>%0d%0a%09%09%09<field+column%3D"NAME"+name%3D"nameSort"+/>%0d%0a%09%09%09<field+column%3D"NAME"+name%3D"alphaNameSort"+/>%0d%0a%0d%0a%09%09%09<entity+name%3D"feature"%0d%0a%09%09%09%09query%3D"select+*+from+FEATURE+where+ITEM_ID%3D'${item.ITEM_ID}';"%0d%0a%09%09%09%09deltaQuery%3D"select+ITEM_ID+from+FEATURE+where+LAST_MODIFIED+>+'${dataimporter.last_index_time}';"%0d%0a%09%09%09%09parentDeltaQuery%3D"select+ITEM_ID+from+ITEM+where+ITEM_ID%3D'${feature.ITEM_ID}';"+pk%3D"FEATURE_ID">%0d%0a%09%09%09%09<field+name%3D"features"+column%3D"DESCRIPTION"+/>%0d%0a%09%09%09</entity>%0d%0a%0d%0a%09%09%09<entity+name%3D"item_category"%0d%0a%09%09%09%09query%3D"select+CATEGORY_ID+from+ITEM_CATEGORY+where+ITEM_ID%3D'${item.ITEM_ID}';"%0d%0a%09%09%09%09deltaQuery%3D"select+ITEM_ID,+CATEGORY_ID+from+ITEM_CATEGORY+where+LAST_MODIFIED+>+'${dataimporter.last_index_time}';"%0d%0a%09%09%09%09parentDeltaQuery%3D"select+ITEM_ID+from+ITEM+where+ITEM_ID%3D$'{item_category.ITEM_ID}';"+pk%3D"ITEM_CATEGORY_ID">%0d%0a%09%09%09%09<entity+name%3D"category"%0d%0a%09%09%09%09%09query%3D"select+DESCRIPTION+from+CATEGORY+where+CATEGORY_ID+%3D+'${item_category.CATEGORY_ID}';"%0d%0a%09%09%09%09%09deltaQuery%3D"select+CATEGORY_ID+from+CATEGORY+where+LAST_MODIFIED+>+'${dataimporter.last_index_time}';"%0d%0a%09%09%09%09%09parentDeltaQuery%3D"select+ITEM_ID,+CATEGORY_ID+from+ITEM_CATEGORY+where+CATEGORY_ID%3D'${category.CATEGORY_ID}';"+pk%3D"CATEGORY_ID">%0d%0a%09%09%09%09%09<field+column%3D"description"+name%3D"cat"+/>%0d%0a%09%09%09%09</entity>%0d%0a%09%09%09</entity>%0d%0a%09%09</entity>%0d%0a%09</document>%0d%0a</dataConfig>&verbose=on&command=full-import&debug=on&rows=10}
> status=0 QTime=18
>
>
> /*
> Navicat MySQL Data Transfer
>
> Source Server : 172.16.197.2
> Source Server Version : 50162
> Source Host : 172.16.197.2:3306
> Source Database : MYSOLR
>
> Target Server Type : MYSQL
> Target Server Version : 50162
> File Encoding : 65001
>
> Date: 2012-06-01 14:18:03
> */
>
> SET FOREIGN_KEY_CHECKS=0;
> -- ----------------------------
> -- Table structure for `CATEGORY`
> -- ----------------------------
> DROP TABLE IF EXISTS `CATEGORY`;
> CREATE TABLE `CATEGORY` (
> `CATEGORY_ID` int(11) NOT NULL,
> `DESCRIPTION` varchar(100) DEFAULT NULL,
> `LAST_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON
> UPDATE CURRENT_TIMESTAMP,
> PRIMARY KEY (`CATEGORY_ID`)
> ) ENGINE=MyISAM DEFAULT CHARSET=utf8;
>
> -- ----------------------------
> -- Records of CATEGORY
> -- ----------------------------
> INSERT INTO CATEGORY VALUES ('1', '测试类型1', '2012-06-01 14:14:33');
>
> -- ----------------------------
> -- Table structure for `FEATURE`
> -- ----------------------------
> DROP TABLE IF EXISTS `FEATURE`;
> CREATE TABLE `FEATURE` (
> `FEATURE_ID` char(8) NOT NULL,
> `ITEM_ID` char(8) DEFAULT NULL,
> `DESCRIPTION` varchar(500) DEFAULT NULL,
> `LAST_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON
> UPDATE CURRENT_TIMESTAMP,
> PRIMARY KEY (`FEATURE_ID`),
> KEY `FK_Reference_1` (`ITEM_ID`)
> ) ENGINE=MyISAM DEFAULT CHARSET=utf8;
>
> -- ----------------------------
> -- Records of FEATURE
> -- ----------------------------
> INSERT INTO FEATURE VALUES ('1', '1',
> '台灯是人们生活中用来照明的一种家用电器。它一般分为两种，一种是立柱式的，一种是有夹子的。它的工作原理主要是把灯光集中在一小块区域内，便于工作和学习。一般台灯用的灯泡是白炽灯或者节能灯泡。
> 有的台灯还有应急功能，用于停电时无电照明已用来应急。 ', '2012-06-01 14:17:54');
>
> -- ----------------------------
> -- Table structure for `ITEM`
> -- ----------------------------
> DROP TABLE IF EXISTS `ITEM`;
> CREATE TABLE `ITEM` (
> `ITEM_ID` char(8) NOT NULL,
> `NAME` varchar(100) DEFAULT NULL,
> `MANU` varchar(100) DEFAULT NULL,
> `WEIGHT` double DEFAULT NULL,
> `PRICE` double DEFAULT NULL,
> `POPULARITY` int(11) DEFAULT NULL,
> `INCLUDES` varchar(100) DEFAULT NULL,
> `LAST_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON
> UPDATE CURRENT_TIMESTAMP,
> PRIMARY KEY (`ITEM_ID`)
> ) ENGINE=MyISAM DEFAULT CHARSET=utf8;
>
> -- ----------------------------
> -- Records of ITEM
> -- ----------------------------
> INSERT INTO ITEM VALUES ('1', '法国台灯A234', '台灯', '10', '100.2', '3',
> '包含灯泡啥的', '2012-06-01 14:15:58');
>
> -- ----------------------------
> -- Table structure for `ITEM_CATEGORY`
> -- ----------------------------
> DROP TABLE IF EXISTS `ITEM_CATEGORY`;
> CREATE TABLE `ITEM_CATEGORY` (
> `ITEM_CATEGORY_ID` char(8) DEFAULT NULL,
> `ITEM_ID` char(8) DEFAULT NULL,
> `CATEGORY_ID` int(11) DEFAULT NULL,
> `LAST_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON
> UPDATE CURRENT_TIMESTAMP,
> KEY `FK_Reference_2` (`ITEM_ID`),
> KEY `FK_Reference_3` (`CATEGORY_ID`)
> ) ENGINE=MyISAM DEFAULT CHARSET=utf8;
>
> -- ----------------------------
> -- Records of ITEM_CATEGORY
> -- ----------------------------
> INSERT INTO ITEM_CATEGORY VALUES ('1', '1', '1', '2012-06-01 14:16:08');
>
>
> <dataConfig>
> <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
>
> url="jdbc:mysql://127.0.0.1:3306/MYSOLR?useUnicode=true&characterEncoding=UTF-8"
> user="root" password="qwertyuiop" batchSize="500" />
>
> <document datasource="JdbcDataSource" name="item_document">
> <entity name="item" pk="ITEM_ID" query="select * from ITEM;"
> deltaQuery="select ITEM_ID from ITEM where
> LAST_MODIFIED >
> '${dataimporter.last_index_time}';">
> <field column="ITEM_ID" name="item_id" />
> <field column="NAME" name="name" />
> <field column="NAME" name="nameSort" />
> <field column="NAME" name="alphaNameSort" />
>
> <entity name="feature"
> query="select * from FEATURE where
> ITEM_ID='${item.ITEM_ID}';"
> deltaQuery="select ITEM_ID from FEATURE where
> LAST_MODIFIED >
> '${dataimporter.last_index_time}';"
> parentDeltaQuery="select ITEM_ID from ITEM
> where
> ITEM_ID='${feature.ITEM_ID}';" pk="FEATURE_ID">
> <field name="features" column="DESCRIPTION" />
> </entity>
>
> <entity name="item_category"
> query="select CATEGORY_ID from ITEM_CATEGORY
> where
> ITEM_ID='${item.ITEM_ID}';"
> deltaQuery="select ITEM_ID, CATEGORY_ID from
> ITEM_CATEGORY where
> LAST_MODIFIED > '${dataimporter.last_index_time}';"
> parentDeltaQuery="select ITEM_ID from ITEM
> where
> ITEM_ID=$'{item_category.ITEM_ID}';" pk="ITEM_CATEGORY_ID">
> <entity name="category"
> query="select DESCRIPTION from
> CATEGORY where CATEGORY_ID =
> '${item_category.CATEGORY_ID}';"
> deltaQuery="select CATEGORY_ID from
> CATEGORY where LAST_MODIFIED
>> '${dataimporter.last_index_time}';"
> parentDeltaQuery="select ITEM_ID,
> CATEGORY_ID from ITEM_CATEGORY
> where CATEGORY_ID='${category.CATEGORY_ID}';" pk="CATEGORY_ID">
> <field column="description" name="cat"
> />
> </entity>
> </entity>
> </entity>
> </document>
> </dataConfig>
>
>
> <requestHandler name="/dataimport"
> class="org.apache.solr.handler.dataimport.DataImportHandler">
> <lst name="defaults">
> <str
> name="config">/Volumes/Date/apache-tomcat-7.0.27/webapps/apache-solr-3.6.0/home/cores/cn/conf/mysql-data-config.xml</str>
> </lst>
> </requestHandler>
>
> <fieldType name="text_cn" class="solr.TextField">
> <analyzer type="index">
> <tokenizer
> class="org.wltea.analyzer.solr.IKTokenizerFactory"
> useSmart="false" />
> </analyzer>
> <analyzer type="query">
> <tokenizer
> class="org.wltea.analyzer.solr.IKTokenizerFactory"
> useSmart="false" />
> </analyzer>
> </fieldType>
>
> <field name="text_cn" type="text_cn" indexed="true" stored="true"
> multiValued="true"/>
>
> <field name="item_id" type="string" indexed="true" stored="true"
> required="true" />
> <field name="sku" type="text_en_splitting_tight" indexed="true"
> stored="true" omitNorms="true"/>
> <field name="name" type="text_cn" indexed="true" stored="true"/>
> <field name="alphaNameSort" type="alphaOnlySort" indexed="true"
> stored="false"/>
> <field name="manu" type="text_cn" indexed="true" stored="true"
> omitNorms="true"/>
> <field name="cat" type="string" indexed="true" stored="true"
> multiValued="true"/>
> <field name="features" type="text_cn" indexed="true" stored="true"
> multiValued="true" />
> <field name="includes" type="text_cn" indexed="true" stored="true"
> termVectors="true" termPositions="true" termOffsets="true" />
>
> <field name="weight" type="float" indexed="true" stored="true"/>
> <field name="price" type="float" indexed="true" stored="true"/>
> <field name="popularity" type="int" indexed="true" stored="true" />
> <field name="inStock" type="boolean" indexed="true" stored="true" />
> ......
