Re: indexing db records via SolrJ

2015-03-16 Thread Jean-Sebastien Vachon
Do you have any references to such integrations (Solr + Storm)? Thanks From: mike st. john Sent: Monday, March 16, 2015 2:39 PM To: solr-user@lucene.apache.org Subject: Re: indexing db records via SolrJ Take a look at some of the integrations people are

Re: indexing db records via SolrJ

2015-03-16 Thread mike st. john
Take a look at some of the integrations people are using with apache storm, we do something similar on a larger scale , having created a pgsql spout and having a solr indexing bolt. -msj On Mon, Mar 16, 2015 at 11:08 AM, Hal Roberts < hrobe...@cyber.law.harvard.edu> wrote: > We import anywher

Re: indexing db records via SolrJ

2015-03-16 Thread Hal Roberts
We import anywhere from five to fifty million small documents a day from a postgres database. I wrestled to get the DIH stuff to work for us for about a year and was much happier when I ditched that approach and switched to writing the few hundred lines of relatively simple code to handle dire

Re: indexing db records via SolrJ

2015-03-16 Thread Shawn Heisey
On 3/16/2015 7:15 AM, sreedevi s wrote: > I had checked this post.I dont know whether this is possible but my query > is whether I can use the configuration for DIH for indexing via SolrJ You can use SolrJ for accessing DIH. I have code that does this, but only for full index rebuilds. It won't

Re: indexing db records via SolrJ

2015-03-16 Thread sreedevi s
Hi, I had checked this post.I dont know whether this is possible but my query is whether I can use the configuration for DIH for indexing via SolrJ Best Regards, Sreedevi S On Mon, Mar 16, 2015 at 4:17 PM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Hello, > > Did you see the great p

Re: indexing db records via SolrJ

2015-03-16 Thread Mikhail Khludnev
Hello, Did you see the great post http://lucidworks.com/blog/indexing-with-solrj/ ? On Mon, Mar 16, 2015 at 1:30 PM, sreedevi s wrote: > Hi, > > I am a beginner in Solr. I have a scenario, where I need to index data from > my MySQL db and need to query them. I have figured out to provide my db

Re: Indexing DB

2013-04-27 Thread Gora Mohanty
On 27 April 2013 22:50, Peri Subrahmanya wrote: > I hooked up the dataimorthandler to my db which has around 15M records. > When I run the full-imoport, its erroring out with java heap space > message. Is there something I need to configure? (a) Please do not hijack threads, but start your own. H

Re: Indexing DB

2013-04-27 Thread Peri Subrahmanya
I fixed it by setting the batchSize="-1" in the db-config.xml. Apparently the default fetch size of 500 rows is not honored by some of the db drivers. I was using MySQL server. Thank you, Peri Subrahmanya On 4/27/13 1:20 PM, "Peri Subrahmanya" wrote: >I hooked up the dataimorthandler to my d