Re: Near Real Time + Facets + Hierarchical Faceting (Pivot Table) with Date Range: huge data set

2012-08-20 Thread Fuad Efendi
NRT does not work because index updates hundreds times per second vs. "cache" warm-up time few minutesŠ and we are in a loopŠ > allowing you to query > your huge index in ms. Solr also allows to query in ms. What is the difference? No one can sort 1,000,000 terms in descending "counts" order fast

Re: Near Real Time + Facets + Hierarchical Faceting (Pivot Table) with Date Range: huge data set

2012-08-14 Thread Nagendra Nagarajayya
You should try realtime NRT available with Apache Solr 4.0 with RankingAlgorithm 1.4.4, allows faceting in realtime. RankingAlgorithm 1.4.4 also provides an age feature that allows you to retrieve the most recent changed docs in realtime, allowing you to query your huge index in ms. You can

Re: Near Real Time + Facets + Hierarchical Faceting (Pivot Table) with Date Range: huge data set

2012-08-13 Thread Mark Miller
There is a per segment faceting option - but I think just for single value fields right now? On Mon, Aug 13, 2012 at 2:38 PM, Fuad Efendi wrote: > SOLR-4.0 > > I am trying to implement this; funny idea to share: > > 1. http://wiki.apache.org/solr/HierarchicalFaceting > unfortunately it does not

Re: Near Real Time Indexing and Searching with solr 3.6

2012-07-03 Thread Michael McCandless
Hi, You might want to take a look at Solr's trunk (very soon to be 4.0.0 alpha release), which already has a near-real-time solution (using Lucene's near-real-time APIs). Lucene has NRTCachingDirectory (to use RAM for small / recently flushed segments), but I don't think Solr uses it yet. Mike M

Re: Near Real Time

2009-10-21 Thread Yonik Seeley
On Wed, Oct 21, 2009 at 10:19 PM, George Aroush wrote: >> Depends a lot on the nature of the requests and the size of the index, >> but one minute is often doable. >> On a large index that facets on many fields per request, one minute is >> probably still out of reach. > > With no facets, what ind

RE: Near Real Time

2009-10-21 Thread George Aroush
> >   Further without the NRT features present what's the closest I can > > expect to real time for the typical use case (obviously this will vary > > but the average deploy). One hour? One Minute? It seems like there are > > a few hacks to get somewhat close. Thanks so much. > > Depends a lot o

Re: Near Real Time

2009-10-21 Thread Matthew Rushton
Thanks, that's great to know. -Matt --- On Wed, 10/21/09, Yonik Seeley wrote: From: Yonik Seeley Subject: Re: Near Real Time To: solr-user@lucene.apache.org Date: Wednesday, October 21, 2009, 6:57 PM On Wed, Oct 21, 2009 at 6:35 PM, Matthew Rushton wrote: >   I'm investigating s

Re: Near Real Time

2009-10-21 Thread Yonik Seeley
On Wed, Oct 21, 2009 at 6:35 PM, Matthew Rushton wrote: >   I'm investigating several search indexing options and solr looks great but > I have a few questions I couldn't find answers to. To begin I don't think the > near real time features of Lucene are a requirement to start but I do think >

Re: Near real-time search of user data

2009-02-19 Thread Mark Ferguson
Thanks Noble and Otis for your suggestions. After reading more messages on the mailing list relating to this problem, I decided to implement one suggestion which was to keep an archive index and a smaller delta index containing only recent updates, then do a distributed search across them. The del

Re: Near real-time search of user data

2009-02-19 Thread Noble Paul നോബിള്‍ नोब्ळ्
we have a similar usecase and I have raised an issue for the same (SOLR-880) currently we are using an internal patch and we hopw to submit one soon. we also use an LRU based automatic loading unloading feature. if a request comes up for a core that is 'STOPPED' . the core is 'STARTED' and the req

Re: Near real-time search of user data

2009-02-19 Thread Otis Gospodnetic
I've used a similar strategy for Simpy.com, but with raw Lucene and not Solr. The crucial piece is to close (inactive) user indices periodically and thus free the memory. Are you doing the same with your per-user Solr cores and still running into memory issues? Otis -- Sematext -- http://sem