RE: [SOLR-2549] DIH LineEntityProcessor support for delimited & fixed-width files

[Dyer, James]

Zakaria,

You might want to post your data-config.xml, or at least the part that uses 
SOLR-2549.  If its throwing an NPE, it certaintly has a bug (if you're doing 
something wrong, it would at least give you a sensible error message).  Also, 
unless you need to use DIH for some other reason, you might want to consider 
the csv request handler to do your imports, which is a mature feature of Solr 
for importing whole documents from delimited (not just csv) files.  See 
http://wiki.apache.org/solr/UpdateCSV

Here is an example that loads a fixed-width file using DIH and SOLR-2549 
(actually it uses code that SOLR-2549 was based on.  I haven't tried this with 
the exact code in SOLR-2549):

<dataConfig>
        <dataSource name="URL" baseUrl="${dataimporter.request.fileBasepath}" 
type="URLDataSource" />
        <document name="FixedWidthCounts">
                <entity
                        name="Counts"
                        
processor="org.apache.solr.handler.dataimport.LineEntityProcessor"
                        dataSource="URL"
                        url="incoming/COUNTS.txt"
                        colDef1="ID,0,9,BIGDECIMAL,0,LEFT"
                        colDef2="COUNT,9,19,INTEGER,0,LEFT"
                />
        </document>
</dataConfig>


James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311


-----Original Message-----
From: zakaria benzidalmal [mailto:zakib...@gmail.com] 
Sent: Wednesday, November 07, 2012 9:08 AM
To: solr-user@lucene.apache.org
Subject: [SOLR-2549] DIH LineEntityProcessor support for delimited & 
fixed-width files

Hi all,

Could some one provide a clear exemple using this Processor
(data-config.xml exemple)?

I run into this problem after patching and building my code:

GRAVE: Full Import failed:java.lang.RuntimeException:
java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.NullPointerException
        at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:273)
        at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:382)
        at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:448)
        at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:429)
Caused by: java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.NullPointerException
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:413)
        at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:326)
        at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:234)
        ... 3 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.NullPointerException
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:542)
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:411)
        ... 5 more
Caused by: java.lang.NullPointerException
        at
org.apache.solr.handler.dataimport.LineEntityProcessor.initDelimitedOrFixedWidth(LineEntityProcessor.java:142)
        at
org.apache.solr.handler.dataimport.LineEntityProcessor.init(LineEntityProcessor.java:115)
        at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:74)
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:430)
        at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:498)
        ... 6 more

Regards.

zakibenz.