I wrote a simple java microservice that did this about a year ago... It was pretty simple - and the kafka topic served as a way to re-create my collection from scratch without hitting the database again in the event of the Solr servers going down.
The code just read from Kafka topic one by one and shipped to Solr in batches of 500 (between commits) (It was a small data set, I was lucky that way) On Tue, Jan 31, 2017 at 3:41 PM, Joel Bernstein <joels...@gmail.com> wrote: > This would make a great Streaming Expression as well. If you're interested > in working on this I'll help out along the way. Here is an example Stream > that connects to a JDBC data source: > > https://github.com/apache/lucene-solr/blob/master/solr/ > solrj/src/java/org/apache/solr/client/solrj/io/stream/JDBCStream.java > > Joel Bernstein > http://joelsolr.blogspot.com/ > > On Tue, Jan 31, 2017 at 12:07 PM, Mike Thomsen <mikerthom...@gmail.com> > wrote: > > > Probably not, but writing your own little Java process to do it would be > > trivial with Kafka 0.9.X or 0.10.X. You can also look at the Confluent > > Platform as they have tons of connectors for Kafka to directly feed into > > other systems. > > > > On Mon, Jan 30, 2017 at 3:05 AM, Mahmoud Almokadem < > prog.mahm...@gmail.com > > > > > wrote: > > > > > Hello, > > > > > > Is there a way to get SolrCloud to pull data from a topic in Kafak > > > periodically using Dataimport Handler? > > > > > > Thanks > > > Mahmoud > > >
