Like many people, Solr is not my primary data store. Not all of my data need be searchable and for simple and fast retrieval I store it in a database (Cassandra in my case). Actually I don't have this all built up yet, but my intention is that whenever new data is entered that it be added to my Cassandra database and simultaneously added to the Solr index (either by queuing up recent data before a commit or some other means; any suggestions on this front?).
But my main question is, how do I guarantee that data between my Cassandra database and Solr index are consistent and up-to-date? What if I write the data to Cassandra and then a failure occurs during the commit to the Solr index? I would need to be aware what data failed to commit and make sure that a re-attempt is made. Obviously inconsistency for a short duration is inevitable when using two different databases (Cassandra and Solr), but I certainly don't want a failure to create perpetual inconsistency. I'm curious what sort of mechanisms people are using to ensure consistency between their database (MySQL, Cassandra, etc.) and Solr. Thank you, Ben
