Indexing CSV without HTTP

[Rohit Gandhe]

Hi Everyone,

We are indexing quite a lot of data using update/csv handler. For
reasons I can't get into right now, I can't implement a DIH since I
can only access the DB using Stored Procs and stored proc support in
DIH is not yet available. Indexing takes about 3 hours and I don't
want to tax the server too much during indexing so I came up with a
two server solution. Indexing server to index the file every night and
subsequently copy the index on the search server. Maintaining a full
fledged Tomcat/Jetty for just indexing is too much of a pain, so I
wrote a small utility Java class which starts an Embedded Server,
indexes the CSV and shuts down the server. I would like the
community's input on this solution.

Is this Okay to do?
Is there a better way to do this without running two separate servers?
Is my class safe enough to run everynight in production environment?

Here's my utility calss. This is just a POC and before I productionize
it, I would like some input from Solr Czars here.

import org.apache.solr.client.solrj.SolrServer;
import org.apache.solr.client.solrj.embedded.EmbeddedSolrServer;
import org.apache.solr.client.solrj.request.ContentStreamUpdateRequest;
import org.apache.solr.common.util.NamedList;
import org.apache.solr.core.CoreContainer;
import org.apache.solr.core.CoreDescriptor;
import org.apache.solr.core.SolrConfig;
import org.apache.solr.core.SolrCore;

import java.io.File;

public class StandaloneSolrIndexer {

    public static void main(String args[]) throws Exception {

        SolrCore core = null;
        CoreContainer container = null;
        try {
            container = new CoreContainer();

            SolrConfig config = new SolrConfig("/tmp/solr",
"solrconfig.xml", null);
            CoreDescriptor descriptor = new CoreDescriptor(container,
"core1", "/tmp/solr");

            core = new SolrCore("core1", "/tmp/solr/data", config,
null, descriptor);
            container.register("core1", core, false);

            SolrServer server = new EmbeddedSolrServer(container, "core1");

            //Start by deleting everything
            server.deleteByQuery("*:*");

            ContentStreamUpdateRequest req = new
ContentStreamUpdateRequest("/update/csv");
            req.addFile(new File("/tmp/product-5k.tsv"));

            req.setParam("commit", "true");
            req.setParam("stream.contentType", "text/plain;charset=utf-8");
            req.setParam("escape", "\\");
            req.setParam("separator", "\t");
            req.setParam("fieldnames",
"product_id,account_id,name,category_tags,short_desc,upc,manu_mdl_num,ext_prd_id,brand,long_desc,sku,seller,seller_email,vertical,cat,subcat");
            req.setParam("skipLines", "1");

            NamedList<Object> result = server.request(req);
            System.out.println("Result
====================================================================================:
\n" + result);

        } finally {
            if (core != null) core.close();
            if (container != null) container.shutdown();
        }
    }
}


Thanks,
Rohit