Hi Hongu, Scaling shouldn't be a problem if you follow the proposed approach but if the updates are frequent then there can be issue with high latency.
I am also following the same approach where the data is first updated/written to Mysql and then in MongoDB. I believe using this we can take advantage of both RDBMS and NoSQL. I am only using MongoDB for non transactional data but planning to use it for transactional data e.g inventory. Thanks, Chandan On Tue, Oct 20, 2015 at 6:22 PM, Susheel Kumar <susheel2...@gmail.com> wrote: > Hello Hangu, > > > OPTION1. you can write complex/nested join queries with DIH and have > functions written in javascript for transformations in data-config if that > meets your domain requirement > OPTION2. use Java program with SolrJ and read data using jdbc and apply > domain specific rules, create solr document and then using SolrJ classes > like (ClouldSolrClient...) write the documents to Solr. This can scale and > will have more flexibility as you go further. > > The current solution you are suggesting to go, can work depending on how > much data you are talking about / how many updates etc. but will have its > own consequences like maintaining a separate state of data (if it is large > you are storing it twice etc...), may not scale etc. > > Thanks, > Susheel > > > > On Tue, Oct 20, 2015 at 2:47 AM, hangu choi <dev.ha...@gmail.com> wrote: > > > Hi, > > > > I am newbie for solr and I hope to check my idea is good or terrible if > > someone can help. > > > > # background > > * I have mysql as my primary data storage. > > * I want to import data from mysql to solr (solrcloud). > > * I have domain logics to make solr document - (means I can't make > solr > > document with plain sql query) > > > > # my idea > > * In mysql I make table for solr document, and insert/update whenever > > any changes made for solr document. > > * run data import handler (mostly delta query) from [solr document > > table] to solr with cron scheduler frequently with soft-commit. > > * hard-commit less frequently. > > > > Is it good or terrible idea? > > for any advice, I will be happy. > > > > > > Regards, > > Hangu > > >
