It seems like you have three components to your system:
1) Data indexing from multiple sources
2) Search for specific words in documents
3) Preserve rating and search term.
I think that Solr comes into play on #1 and #2. You can index content
in any number of approaches, either via the new DataImportHandler
architecture, or the more traditional write a loader script that puts
the documents in Solr. You can store in Solr when a document was
indexed, and use that to check against the original documents to see
if they changed. Check a last published tag on an RSS feed, or the
last updated time on a physical file. This is a very common use case
for Solr.
For #2, you could have users issue queries, and make them "favorites",
storing them in the DB. Assuming they like the results they mark the
documents with the ratings, which you could store in Solr, but I would
put in a DB.. Easier to manage User A says 1, User B says 0.
Then for the UI, just issue the search baseed on queries stored in the
db, and match the id's up with the ranking in the DB. Simple!
As far as the last part, Solr works best in filesystem, that is part
of why it is so fast, no clunky SQL. There are scripts for backing up
and restoring indexes that you can use, check the wiki http://wiki.apache.org/solr/SolrOperationsTools
.
Eric
On Apr 24, 2009, at 6:18 AM, Developer In London wrote:
Hi All,
I am new to the whole Solr/Lucene community. But I think this might
be the
solution ot what I am looking to do. I would appreciate any feedback
on how
I can go about doing this with Solr:
I am looking to make a system where -
a) mainly lots of different blog sites, web journals, articles are
indexed
on a regular basis. Data that has already been indexed needs to be
revisited
to see if there are any changes.
b) The end users has very fixed search terms, eg 'Lloyds TSB' and
'Corporate
Banking'. All the documents that are found matching this are
presented to a
human to analyse.
c) Once the human analyses the document he gives it a rating of 1, 0
or -1.
This rating needs to be saved somewhere and be linked with the
specific
document and also with the search term (eg 'Lloyds TSB' & 'Corporate
Banking' in this case).
d) End users can then see these documents with the ratings next to
them.
What would be the best approach to this?
Should I set up a different database to save the rating and relevant
mappings, or is there any way to put it in to Solr?
My 2nd question is, can Solr Index be saved in a database in any
way? Whats
the backup and recovery method on Solr?
Thanks in advance.
Nayeem
-----------------------------------------------------
Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 |
http://www.opensourceconnections.com
Free/Busy: http://tinyurl.com/eric-cal
