Re: dynamic categorization & transactional data

2010-03-20 Thread caman
@Grant Less than a minute. If we go with the meta-retrieval from the index, we will have to keep the index updated down to seconds. But that may not scale well. Probably a hybrid approach? I will look into classifier. thanks Grant Ingersoll-6 wrote: > > > On Mar 18, 2010, at 2:44 PM, cama

Re: dynamic categorization & transactional data

2010-03-18 Thread Grant Ingersoll
On Mar 18, 2010, at 2:44 PM, caman wrote: > > 1) Took care of the first one by Transformer. This is often also something done by a classifier that is trained to deal with all the statistical variations in your text. Tools like Weka, Mahout, OpenNLP, etc. can be applied here. > 2) Any input

Re: dynamic categorization & transactional data

2010-03-18 Thread caman
David, Much appreciated. This gives me enough to work with. I missed one important point. Our data changes pretty frequently which mean we may be running deltas every 5-10 minutes. in-memory should work thanks David Smiley @MITRE.org wrote: > > You'll probably want to influence your releva

Re: dynamic categorization & transactional data

2010-03-18 Thread Smiley, David W.
You'll probably want to influence your relevancy on this popularity number that is changing often. ExternalFileField looks like a possibility though I haven't used it. Another would be using an in-memory cache which stores all popularity numbers for any data that has its popularity updated sin

Re: dynamic categorization & transactional data

2010-03-18 Thread caman
1) Took care of the first one by Transformer. 2) Any input on 2 please? I need to store # of views and popularity with each document and that can change pretty often. Recommended to use database or can this be updated to SOLr directly? My issue with DB is that with every SOLR search hit, will have