hi,
it is probably correct to revisit your design/requirements, but it you
still find you need it, then there may be a different way
DIH is using a writer to commit documents, you can detect errors inside
these and try to recover - ie. in some situations, you want to commit,
instead of calling rollback
These writers can be specified in the solrconfig.xml, for example:
<requestHandler name="/invenio/import"
class="solr.WaitingDataImportHandler">
<lst name="defaults">
<str name="config">data-config.xml</str>
<bool name="clean">false</bool>
<bool name="commit">false</bool>
<str name="update.chain">blanketyblank</str>
<!-- this parameter activates the logging/restart of failed imports
-->
<str
name="writerImpl">org.apache.solr.handler.dataimport.FailSafeInvenioNoRollbackWriter</str>
</lst>
</requestHandler>
when error happens, DIH will call rollback - that is when you can inspect
what was going on (but alas, it is not always easy) and do something.
You can see an example here:
https://github.com/romanchyla/montysolr/blob/master/contrib/invenio/src/java/org/apache/solr/handler/dataimport/FailSafeInvenioNoRollbackWriter.java
This handler will find which documents were already indexed and call a
handler to register the missing ones into the queue.
But DIH needs to use its own interface properly if you want to write these
writers, please vote on this issue!
https://issues.apache.org/jira/browse/SOLR-3671
Best,
Roman
On Tue, Jan 22, 2013 at 8:57 AM, Gora Mohanty <g...@mimirtech.com> wrote:
> On 21 January 2013 17:06, ashimbose <ashimb...@gmail.com> wrote:
> [...]
> > Here I used two data config
> > 1. data_conf1.xml
> > 2. data_conf2.xml
> [...]
>
> Your configuration looks fine.
>
> > Any one of them running fine at a single instant. Means,
> > If I run first dataimport, it will successfully index, if after that I
> run
> > dataimport1, it is giving below error..
> >
> > Caused by: java.sql.SQLException: JBC0088E: JBC0002E: Socket timeout
> > detected: Read timed out
>
> Have you looked at the load on your database server?
> I am guessing that is where the bottleneck lies. This
> configuration is useful only of you can scale your
> database server, or have multiple servers, each with
> a different set of tables.
>
> As Upayavira, suggested you could look into SolrJ,
> or a similar library to control your indexing. I would
> once again suggest starting with smaller goals, and
> fixing issues one by one, rather than jumping in and
> trying to get everything working at once.
>
> Regards,
> Gora
>
