Re: Near real-time search of user data

2009-02-19 Thread Mark Ferguson
Thanks Noble and Otis for your suggestions. After reading more messages on the mailing list relating to this problem, I decided to implement one suggestion which was to keep an archive index and a smaller delta index containing only recent updates, then do a distributed search across them. The del

Re: Near real-time search of user data

2009-02-19 Thread Noble Paul നോബിള്‍ नोब्ळ्
we have a similar usecase and I have raised an issue for the same (SOLR-880) currently we are using an internal patch and we hopw to submit one soon. we also use an LRU based automatic loading unloading feature. if a request comes up for a core that is 'STOPPED' . the core is 'STARTED' and the req

Re: Near real-time search of user data

2009-02-19 Thread Otis Gospodnetic
I've used a similar strategy for Simpy.com, but with raw Lucene and not Solr. The crucial piece is to close (inactive) user indices periodically and thus free the memory. Are you doing the same with your per-user Solr cores and still running into memory issues? Otis -- Sematext -- http://sem