You can consider DataStax Enterprise (DSE) which deeply integrates Solr (not just a plugin) with the Cassandra database (DSE Search): http://www.datastax.com/products/datastax-enterprise-search
Solr's Join queries are supported across tables in DSE Search, so you could keep dynamic data in a separate table (use the same partition key to assure that the join will be more efficient by being on the same node.) -- Jack Krupansky On Fri, Dec 11, 2015 at 6:21 AM, Andrea Gazzarini <a.gazzar...@gmail.com> wrote: > Hi Vikram, > sounds like you're using those "dynamic" fields only for visualization > (i.e. you don't need to have them "indexed")...this is the big point that > could make the difference. > > If the answer is yes, about the first option (NOTE: I don't know Redis and > that plugin), a custom SearchComponent would be very easy to implement. It > would contribute to search results in a dedicated section of the response > (see for example the highlight or the facet component) > > I don't have a concrete experience about the second option, but still > assuming that > > - you need those fields stored, not indexed > - the response page size is not huge (this is considered a bad practice in > Solr) > > I would avoid to bomb Solr with repeated updates > > Best, > Andrea > > > > 2015-12-11 11:48 GMT+01:00 Vikram Parmar <parmar.vik...@gmail.com>: > > > We are creating a web application which would contain posts (something > like > > FB or say Youtube). For the stable part of the data (i.e.the facets, > search > > results & its content), we plan to use SOLR. > > > > What should we use for the unstable part of the data (i.e. dynamic and > > volatile content such as Like counts, Comments counts, Viewcounts)? > > > > > > Option 1) Redis > > > > What about storing the "dynamic" data in a different data store (like > > Redis)? Thus, everytime the counts get refreshed, I do not have to > reindex > > the data into SOLR at all. Thus SOLR indexing is only triggered when new > > posts are added to the site, and never on any activity on the posts by > the > > users. > > > > Side-note :- > > I also looked at the SOLR-Redis plugin at > > https://github.com/sematext/solr-redis > > > > The plugin looks good, but not sure if the plugin can be used to fetch > the > > data stored in Redis as part of the solr result set, i.e. in docs. The > > description looks more like the Redis data can be used in the function > > queries for boosting, sorting, etc. Anyone has experience with this? > > > > > > Option 2) SOLR NRT with Soft Commits > > > > We would depend on the in-built NRT features. Let's say we do > soft-commits > > every second and hard-commits every 10 seconds. Suppose huge amount of > > dynamic data is created on the site across hundreds of posts, e.g. 100000 > > likes across 10000 posts. Thus, this would mean soft-commiting on 10000 > > rows every second. And then hard-commiting those many rows every 10 > > seconds. Isn't this overkill? > > > > > > Which option is preferred? How would you compare both options in terms of > > scalibility, maintenance, feasibility, best-practices, etc? Any real-life > > experiences or links to articles? > > > > Many thanks! > > > > > > p.s. EFF (external file fields) is not an option, as I read that the data > > in that file can only be used in function queries and cannot be returned > as > > part of a document. > > >
