Sure Divye,
*Here's the config.....*
*conf/solr-config.xml:*
<lib dir="../../../../dist/" regex="solr-dataimporthandler-.*\.jar" />
<!-- DIH Starts -->
<requestHandler name="/dataimport"
class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str
name="config">/home/ec2-user/solr/solr-5.4.1/server/solr/test_core/conf/dataimport/data-source-config.xml</str>
</lst>
</requestHandler>
<!-- DIH ends -->
*schema.xml:*
has of all the field definitions
*conf/dataimport/data-source-config.xml*
<dataConfig>
<dataSource name="mongod" type="JdbcDataSource"
driver="com.mongodb.jdbc.MongoDriver" url="mongodb://<<IP
ADDRESS>>:27017/<<DB>>"/>
<document name="products">
<entity name="products"
dataSource="mongod"
transformer="<<Custom Transformer>>,TemplateTransformer"
onError="continue"
pk="uuid"
query="SELECT field1,field2,field3,...... FROM products"
deltaImportQuery="SELECT field1,field2,field3,...... FROM products WHERE
orgidStr = '${dataimporter.request.orgid}' AND idStr =
'${dataimporter.delta.idStr}'"
deltaQuery="SELECT idStr FROM products WHERE orgidStr =
'${dataimporter.request.orgid}' AND updatedAt >
'${dataimporter.last_index_time}'"
>
<field column="field1" name="fieldName1"/>
<field column="field2" name="fieldName2"/>
<field column="field3" name="fieldName3"/>
<entity name=categories">
.
.
. 4-5 more nested entities.......
On Mon, Aug 17, 2020 at 1:32 PM Divye Handa <divye.handaa...@gmail.com>
wrote:
> Can you share the dih configuration you are using for same?
>
> On Mon, 17 Aug, 2020, 23:52 Abhijit Pawar, <aapawar.s...@gmail.com> wrote:
>
> > Hello,
> >
> > We are indexing some 200K plus documents in SOLR 5.4.1 with no shards /
> > replicas and just single core.
> > It takes almost 3.5 hours to index that data.
> > I am using a data import handler to import data from the mongo database.
> >
> > Is there something we can do to reduce the time taken to index?
> > Will upgrade to newer version help?
> >
> > Appreciate your help!
> >
> > Regards,
> > Abhijit
> >
>
