Thanks for that! I was thinking (B) too, but wanted guidance that I'm using the tool correctly.
Am still interested in hearing opinions from others, thanks! rh On Tue, Feb 21, 2017 at 8:17 PM, Dave <hastings.recurs...@gmail.com> wrote: > B is a better option long term. Solr is meant for retrieving flat data, > fast, not hierarchical. That's what a database is for and trust me you > would rather have a real database on the end point. Each tool has a > purpose, solr can never replace a relational database, and a relational > database could not replace solr. Start with the slow model (database) for > control/display and enhance with the fast model (solr) for retrieval/search > > > > > On Feb 21, 2017, at 7:57 PM, Robert Hume <rhum...@gmail.com> wrote: > > > > To learn how to properly use Solr, I'm building a little experimental > > project with it to search for used car listings. > > > > Car listings appear on a variety of different places ... central places > > Craigslist and also many many individual Used Car dealership websites. > > > > I am wondering, should I: > > > > (a) deploy a Solr search engine and build individual indexers for every > > type of web site I want to find listings on? > > > > or > > > > (b) build my own database to store car listings, and then build services > > that scrape data from different sites and feed entries into the database; > > then point my Solr search to my database, one simple source of listings? > > > > My concerns are: > > > > With (a) ... I have to be smart enough to understand all those different > > data sources and remove/update listings when they change; while this be > > harder to do with custom Solr indexers than writing something from > scratch? > > > > With (b) ... I'm maintaining a huge database of all my listings which > seems > > redundant; google doesn't make a *copy* of everything on the internet, it > > just knows it's there. Is maintaining my own database a bad design? > > > > Thanks for reading! >
