Hi, I'm actually not sure what Wunder is suggesting, but here is another way. Have an external app that talks to the DB either on demand or every N minutes/hours. When it talks to the DB it gets all merchants whose visibility flag was changed one way or the other since the last time the app checked. Then you can simply delete all products for those merchants whose products are supposed to be hidden, and reindex all those whose flag was switched to visible. Depending on the numbers this may or may not work well, but it's super simple and Solr and the DB don't know about each other.
Otis -- Performance Monitoring - http://sematext.com/spm/index.html Search Analytics - http://sematext.com/search-analytics/index.html On Fri, Nov 16, 2012 at 7:32 PM, Hayden Muhl <haydenm...@gmail.com> wrote: > I am working on migrating our system from Lucene to Solr, and my boss and I > are at an impasse over an architectural issue. Here's the basic setup. > > We index products from multiple retailers, and allow people to search > across all retailers for specific products. It is a regular occurrence for > us to disable or deactivate a retailer (on the order of once a day). What > that means is that when someone does a search, we will not show results for > products sold by deactivated retailers. The list of active/inactive > retailers is maintained in a table in our SQL database. We have to maintain > this functionality when we move to Solr, but we can't agree on how to > implement this in Solr. > > Currently, we load a new Lucene product index once every two hours. Every > time we load a new index, we run a SQL query to find the current list of > active retailers, and construct a filter based on that list. > > My boss wants to essentially do the same thing we do now. Implement a > custom filter that makes a call to the database to retrieve the list of > retailers, caches that list for some period of time, then refreshes itself > from time to time with another call to the database. I find it strange > having a dependency between Solr and the database like that, because it > would require a running database being present in order to even start Solr. > I am new to Solr, so I don't have any alternative solutions. > > tl;dr, We need to construct a Solr filter based on data stored in a > database. What's the best way to get that data from the database into Solr > and keep it updated? > > - Hayden >
