FileListEntityProcessor and LineEntityProcessor

[wojtekpia]

Hi,

I'm trying to import data from a list of files using the
FileListEntityProcessor. Here is my import configuration:

  <dataSource type="FileDataSource" name="fileDataSource"/>
  <document name="dict-entries">
    <entity name="f" processor="FileListEntityProcessor"
baseDir="d:\my\directory\" fileName=".*WRK" recursive="false"
rootEntity="false">
      <entity name="jc"
        processor="LineEntityProcessor"
        url="${f.fileAbsolutePath}"
        dataSource="fileDataSource"
        transformer="myTransformer">
      </entity>
    </entity>
  </document>

If I have only one file in d:\my\directory\ then everything works correctly.
If I have multiple files then I get the following exception: 

Sep 16, 2009 9:48:46 AM org.apache.solr.handler.dataimport.DocBuilder
buildDocum
ent
SEVERE: Exception while processing: f document : null
org.apache.solr.handler.dataimport.DataImportHandlerException: Problem
reading f
rom input Processing Document # 53812
        at
org.apache.solr.handler.dataimport.LineEntityProcessor.nextRow(LineEn
tityProcessor.java:112)
        at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(Ent
ityProcessorWrapper.java:237)
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
r.java:348)
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
r.java:376)
        at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j
ava:224)
        at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java
:167)
        at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo
rter.java:316)
        at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j
ava:376)
        at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja
va:355)
Caused by: java.io.IOException: Stream closed
        at java.io.BufferedReader.ensureOpen(Unknown Source)
        at java.io.BufferedReader.readLine(Unknown Source)
        at java.io.BufferedReader.readLine(Unknown Source)
        at
org.apache.solr.handler.dataimport.LineEntityProcessor.nextRow(LineEn
tityProcessor.java:109)
        ... 8 more
Sep 16, 2009 9:48:46 AM org.apache.solr.handler.dataimport.DataImporter
doFullIm
port
SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException: Problem
reading f
rom input Processing Document # 53812
        at
org.apache.solr.handler.dataimport.LineEntityProcessor.nextRow(LineEn
tityProcessor.java:112)
        at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(Ent
ityProcessorWrapper.java:237)
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
r.java:348)
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde
r.java:376)
        at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j
ava:224)
        at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java
:167)
        at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo
rter.java:316)
        at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j
ava:376)
        at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja
va:355)
Caused by: java.io.IOException: Stream closed
        at java.io.BufferedReader.ensureOpen(Unknown Source)
        at java.io.BufferedReader.readLine(Unknown Source)
        at java.io.BufferedReader.readLine(Unknown Source)
        at
org.apache.solr.handler.dataimport.LineEntityProcessor.nextRow(LineEn
tityProcessor.java:109)
        ... 8 more



Note that my input files have 53812 lines, which is the same as the document
number that I'm choking on. Does anyone know what I'm doing wrong?

Thanks,

Wojtek
-- 
View this message in context: 
http://www.nabble.com/FileListEntityProcessor-and-LineEntityProcessor-tp25476443p25476443.html
Sent from the Solr - User mailing list archive at Nabble.com.