Re: NRT - Indexing

Hi, The issue was buildOnCommit=true on a SuggestComponent. Dominique Le mar. 2 févr. 2021 à 00:54, Shawn Heisey a écrit : > On 2/1/2021 12:08 AM, haris.k...@vnc.biz wrote: > > Hope you're doing good. I am trying to configure NRT - Indexing in my > > project. For this reason, I have configured

Re: NRT - Indexing

On 2/1/2021 12:08 AM, haris.k...@vnc.biz wrote: Hope you're doing good. I am trying to configure NRT - Indexing in my project. For this reason, I have configured *autoSoftCommit* to execute every second and *autoCommit* to execute every 5 minutes. Everything works as expected on the dev and tes

Re: NRT - Indexing

Hi, It is not the cause of your issue, but Solr version is 8.6.0, and solrconfig.xml includes 7.5.0 By "I am using a service that fetches data from the Postgres database and indexes it to solr. The service runs with a delay of 5 seconds.". You man, you are using DIH and launch a delta-import each

Re: NRT - Indexing

Hello,I am attaching the solrconfig.xml along with this email, also I am attaching a text document that has JSON object regarding the system information I am using a service that fetches data from the Postgres database and indexes it to solr. The service runs with a delay of 5 seconds.RegardsMit

Re: NRT - Indexing

Hi, What is your Solr version ? Can you share your solrconfig.xml ? How is your sharding ? Did you grep your solr logs on with the "commit' pattern in order to see hard and soft commit occurrences ? How are you pushing new docs or updates in the collection ? Regards. Dominique Le lun. 1 févr

Re: NRT - Indexing

I'm running into the same issue. I've set autoSoftCommit and autoCommit but the speed at which docs are indexed seems to be inconsistent with the settings. I have lowered the autoCommit to a minute but it still takes a few minutes for docs to show after indexing. Soft commit settings also seem to h

Re: NRT Real time Get with documentCache

Great stuff thank you Erick On 04/02/2020, 00:17, "Erick Erickson" wrote: The documentCache shouldn’t matter at all. RTG should return the latest doc by maintaining a pointer into the tlogs and returning that version. > On Feb 3, 2020, at 6:43 PM, Karl Stoney wrote: > > Hi,

Re: NRT Real time Get with documentCache

The documentCache shouldn’t matter at all. RTG should return the latest doc by maintaining a pointer into the tlogs and returning that version. > On Feb 3, 2020, at 6:43 PM, Karl Stoney > wrote: > > Hi, > Could anyone let me know if a real time get would return a cached, up to date > version

Re: NRT vs TLOG bulk indexing performances

Hi, Thank you Erick for your response. My documents are small. Here is a sample csv file http://gofile.me/2dlpH/66hv2NPhJ In the TLOG case, the CPU is not hot and not idling on leaders : - 1m load average between 1.5 and 2.5 (4 cpu cores) - CPU % between

Re: NRT vs TLOG bulk indexing performances

"I understand that while non leader TLOG is copying the index from leader, the leader stop indexing” This _better_ not be happening. If you can demonstrate this let’s open a JIRA. > On Oct 25, 2019, at 8:28 AM, Dominique Bejean > wrote: > > I understand that while non leader TLOG is copying th

Re: NRT vs TLOG bulk indexing performances

I’m also surpised that you see a slowdown, it’s worth investigating. Let’s take the NRT case with only a leader. I’ve seen the NRT indexing time increase when even a single follower was added (30-40% in this case). We believed that the issue was the time the leader sat waiting around for the fo

Re: NRT vs TLOG bulk indexing performances

Shawn Heisey kirjoitti 25.10.2019 klo 14.54: > With newer Solr versions, you can ask SolrCloud to prefer PULL replicas > for querying, so queries will be targeted to those replicas, unless they > all go down, in which case it will go to non-preferred replica types.  I > do not know how to do this,

Re: NRT vs TLOG bulk indexing performances

Shawn, So, I understand that while non leader TLOG is copying the index from leader, the leader stop indexing. One shot large heavy bulk indexing should be very much more impacted than continus ligth indexing. Regards. Dominique Le ven. 25 oct. 2019 à 13:54, Shawn Heisey a écrit : > On 10/25

Re: NRT vs TLOG bulk indexing performances

On 10/25/2019 1:16 AM, Dominique Bejean wrote: For collection created with all replicas as NRT * Indexing time : 22 minutes For collection created with all replicas as TLOG * Indexing time : 34 minutes NRT indexes simultaneously on all replicas. So when indexing is done on one, it is a

Re: NRT vs TLOG bulk indexing performances

Hi Jörn , I am using version 8.2. I repeated the test twice for each mode. I restarted solr nodes and deleted / created empty collection each time. Regards. Dominique Le ven. 25 oct. 2019 à 09:20, Jörn Franke a écrit : > Which Solr version are you using and how often you repeated the test? >

Re: NRT vs TLOG bulk indexing performances

Which Solr version are you using and how often you repeated the test? > Am 25.10.2019 um 09:16 schrieb Dominique Bejean : > > Hi, > > I made some benchmarks for bulk indexing in order to compare performances > and ressources usage for NRT versus TLOG replica. > > Environnent : > * Solrcloud wi

Re: NRT for new items in index

Thank you for the interesting reply. You confirmed our assumptions about that. The usage of two or more collections, as Jörn Franke said, is more complicated for developing. And for a now we will only try split image to more shards and servers and try to reduce commit times too. I think that

Re: NRT for new items in index

Good afternoon! Thank you for your recent email regarding a career opportunity! For your reference, here are links to my updated cover letter and resume. Cover Letter - https://drive.google.com/open?id=1TUi_KeuqcruESeypu5S3-krEOKulxL51 Resume - https://drive.google.com/open?id=1nVRPm-dFVvvjeHK-j

Re: NRT for new items in index

On 7/31/2019 6:47 AM, profiuser wrote: we have something about 400 000 000 items in a solr collection. We have set up auto commit property for this collection to 15 minutes. Is a big collection and we using some caches etc. Therefore we have big autocommit value. I would set autoCommit to 60 se

Re: NRT for new items in index

On 2019/08/06 06:43:20, Jörn Franke wrote: > Do you have some more information on index and size? > > Do you have to store everything in the Index? Can you store some data (blobs > etc) outside ? > > I think you are generally right with your solution, but also be aware that it > is someti

Re: NRT for new items in index

Do you have some more information on index and size? Do you have to store everything in the Index? Can you store some data (blobs etc) outside ? I think you are generally right with your solution, but also be aware that it is sometimes cheaper to have several servers instead keeping engineer b

Re: NRT for new items in index

On 2019/08/03 18:00:28, Furkan KAMACI wrote: > Hi, > > First of all, could you check here: > https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ > to > better understand hard commits, soft commits and transaction logs to > achieve NRT search. > > Ki

Re: NRT for new items in index

Hi, First of all, could you check here: https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ to better understand hard commits, soft commits and transaction logs to achieve NRT search. Kind Regards, Furkan KAMACI On Wed, Jul 31, 2019 at 3:47 PM profiuser

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

essage- > From: Webster Homer [mailto:webster.ho...@sial.com] > Sent: Friday, March 2, 2018 10:29 AM > To: solr-user@lucene.apache.org > Subject: Re: NRT replicas miss hits and return duplicate hits when paging > solrcloud searches > > I am trying to test if enabling stats

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

Thanks Shawn. Commenting it out works to remove it. If I change the values e.g. change the 512 to 0, it does require a restart to take effect. Tested using statsCache set to org.apache.solr.search.stats.ExactSharedStatsCache, with the queryResultCache disabled, and I still see the problem with NR

RE: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

pplication they belong to. It wasn’t a problem until we had 2 collections for one of the apps. -Original Message- From: Webster Homer [mailto:webster.ho...@sial.com] Sent: Friday, March 2, 2018 10:29 AM To: solr-user@lucene.apache.org Subject: Re: NRT replicas miss hits and return

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

On 3/2/2018 9:28 AM, Webster Homer wrote: I've never disabled this before. I edited the solrconfig.xml setting the sizes to 0. I'm not sure if this is how to disable the cache or not. To completely disable a cache, either comment it out or remove it from the config.  I do not know wheth

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

I am trying to test if enabling stats cache as suggested by Eric would also address this issue. I added this to my solrconfig.xml I executed queries and saw no differences. Then I re-indexed the data, again I saw no differences in behavior. Then I found this, SOLR-10952. It seems we need to di

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

Emir, Using tlog replica types addresses my immediate problem. The secondary issue is that all of our searches show inconsistent results. These are all normal paging use cases. We regularly test our relevancy, and these differences creates confusion in the testers. Moreover, we are migrating from

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

Hi Webster, Since you are returning all hits, returning the last page is almost as heavy for Solr as returning all documents. Maybe you should consider just returning one large page and completely avoid this issue. I agree with you that this should be handled by Solr. ES solved this issue with “

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

Erick, No we didn't look at that. I will add it to the list. We have not seen performance issues with solr. We have much slower technologies in our stack. This project was to replace a system that was too slow. Thank you, I will look into it Webster On Mon, Feb 26, 2018 at 1:13 PM, Erick Erick

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

Did you try enabling distributed IDF (statsCache)? See: https://lucene.apache.org/solr/guide/6_6/distributed-requests.html It's may not totally fix the issue, but it's worth trying. It does come with a performance penalty of course. Best, Erick On Mon, Feb 26, 2018 at 11:00 AM, Webster Homer wr

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

Thanks Shawn, I had settled on this as a solution. All our use cases for Solr is to return results in order of relevancy to the query, so having a deterministic sort would defeat that purpose. Since we wanted to be able to return all the results for a query, I originally looked at using the Stream

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

On 2/26/2018 10:26 AM, Webster Homer wrote: > We need the results by relevancy so the application sorts the results by > score desc, and the unique id ascending as the tie breaker This is the reason for the discrepancy, and why the different replica types don't have the same issue. Each NRT repli

Re: NRT updates

Hi, Any pointers, suggestions, experiences ... please.. Thanks! Chris. On Mon, Jun 6, 2016 at 10:27 AM, Chris Vizisa wrote: > Hi, > > Does number of fields in a document affect NRT updates? > I have around 1.6 million products. Each product can be available in about > 3000 stores. > In addition

Re: NRT vs Redis for Dynamic Data in SOLR (like counts, viewcounts, etc) -

On 15/12/2015 14:13, Vikram Parmar wrote: Hi Mikhail, Hi, In case you're interested, several years ago we prototyped a Lucene codec using Redis for just this sort of application: http://www.flax.co.uk/blog/2012/06/22/updating-individual-fields-in-lucene-with-a-redis-backed-codec/ It's a sli

Re: NRT vs Redis for Dynamic Data in SOLR (like counts, viewcounts, etc) -

Hi Mikhail, Thanks for chiming in. Looking forward to your post regarding updatable numeric DocValues. What would be the 2nd most promising approach for now, would you say EFF should be ok to go with? Updating and reloading the EFF external file (containing a millions lines) at very short interv

Re: NRT vs Redis for Dynamic Data in SOLR (like counts, viewcounts, etc) -

I believe https://issues.apache.org/jira/browse/SOLR-5944 is the most promising approach for such scenarios. Despite it's not delivered in distro. We are going to publish a post about it at blog.griddynamics.com. FWIW, I suppose EFF can be returned in result list. On Fri, Dec 11, 2015 at 1:48 P

Re: NRT vs Redis for Dynamic Data in SOLR (like counts, viewcounts, etc) -

You can consider DataStax Enterprise (DSE) which deeply integrates Solr (not just a plugin) with the Cassandra database (DSE Search): http://www.datastax.com/products/datastax-enterprise-search Solr's Join queries are supported across tables in DSE Search, so you could keep dynamic data in a separ

Re: NRT vs Redis for Dynamic Data in SOLR (like counts, viewcounts, etc) -

Hi Vikram, sounds like you're using those "dynamic" fields only for visualization (i.e. you don't need to have them "indexed")...this is the big point that could make the difference. If the answer is yes, about the first option (NOTE: I don't know Redis and that plugin), a custom SearchComponent w

Re: NRT - softCommit

You shouldn't need to specify openSearcher on soft commits, I suspect there's something else going on. Is it possible that you're getting results from your browser cache? Try turning browser caching off would be one place to start. To check your autocommit, one thing you can do is issue q=*:*, the

Re: NRT - softCommit

You have to open searchers for the new data to show up. Try this: 1 false 2000 true Make sure that you have low autowarm counts otherwise you need to reduce the freq. of commits. On 15 March 2013 18:59, Arkadi Colson wrote: > NRT seems not to work in my case when doing a softcommit ev

Re: Nrt and caching

is approach any time. I do not now about "soon". Best regards Karsten in context: http://lucene.472066.n3.nabble.com/Nrt-and-caching-tp3993612p3993700.html Original-Nachricht > Datum: Sat, 7 Jul 2012 17:32:52 -0700 (PDT) > Von: Andy > An: "solr-u

Re: NRT and multi-value facet - what is Solr's limit?

Hi Andy, as long as the cache for facetting is not per segment there is no NRT together with facetting. This is what Jason told you in http://lucene.472066.n3.nabble.com/Nrt-and-caching-td3993612.html and I am agree. Possible you could use multicore. Beste regards Karsten Original

Re: Nrt and caching

documents? > > Which open source library are you referring to? Will Solr adopt this > per-segment approach any time soon? > > Thanks > > > > From: Jason Rutherglen > To: solr-user@lucene.apache.org > Sent: Saturday, July 7, 2012 2:05

Re: Nrt and caching

hanks From: Jason Rutherglen To: solr-user@lucene.apache.org Sent: Saturday, July 7, 2012 2:05 PM Subject: Re: Nrt and caching Andy, You'd need to hack on the Solr code, specifically the SimpleFacets class. Solr uses UnInvertedField to build an in memory do

Re: Nrt and caching

ache to per-segment? How do I do that? > > Thanks. > > > > From: Jason Rutherglen > To: solr-user@lucene.apache.org > Sent: Saturday, July 7, 2012 11:32 AM > Subject: Re: Nrt and caching > > The field caches are per-segment, whi

Re: Nrt and caching

et with NRT I'd need to convert the > cache to per-segment? How do I do that? > > Thanks. > > > > From: Jason Rutherglen > To: solr-user@lucene.apache.org > Sent: Saturday, July 7, 2012 11:32 AM > Subject: Re: Nrt and caching > >

Re: Nrt and caching

So If I want to use multi-value facet with NRT I'd need to convert the cache to per-segment? How do I do that? Thanks. From: Jason Rutherglen To: solr-user@lucene.apache.org Sent: Saturday, July 7, 2012 11:32 AM Subject: Re: Nrt and caching The

Re: Nrt and caching

The field caches are per-segment, which are used for sorting and basic [slower] facets. The result set, document, filter, and multi-value facet caches are [in Solr] per-multi-segment. Of these, the document, filter, and multi-value facet caches could be converted to be [performant] per-segment, a

Re: Nrt and caching

On Sat, Jul 7, 2012 at 9:59 AM, Jason Rutherglen wrote: > Currently the caches are stored per-multiple-segments, meaning after each > 'soft' commit, the cache(s) will be purged. Depends which caches. Some caches are per-segment, and some caches are top level. It's also a trade-off... for some th

Re: Nrt and caching

Hi Amit, If the caches were per-segment, then NRT would be optimal in Solr. Currently the caches are stored per-multiple-segments, meaning after each 'soft' commit, the cache(s) will be purged. On Fri, Jul 6, 2012 at 9:45 PM, Amit Nithian wrote: > Sorry I'm a bit new to the nrt stuff in solr b

Re: NRT or similar for Solr 3.5?

I followed the steps and they worked fine for me ... On further investigation I found that I had not changed the *solr.war* file in myapp/*webapps* folder to the *solr.war* with ranking algorithm .. @Steven check that out Thanks Nagendra Regards Vikram Kamath On 17 December 2011 22:35, Nagend

Re: NRT or similar for Solr 3.5?

Regarding docs not showing up, can you try adding docs to the demo index (MBArtists that is available with the download). The start_solr.sh points to the index, which is under solr under the examples directory. start solr with the start_solr.sh script and add a document to the index using the

Re: NRT or similar for Solr 3.5?

@Steven even I have done the same thingEven I am user btw @Nagendra I have an index on about 15 million mysql records. I am using the solr-ra Solr 3.5 (NRT support) with RankingAlgorithm 1.3 (bundle 118MB) and have added my index (indexed using solr 3.4)

Re: NRT or similar for Solr 3.5?

Hey Vikram, I finally got around to getting Solr-RA installed but I'm having trouble getting the NRT to work. Could you help me out? I added these four lines immediately after in solrconfig.xml: true rankingalgorithm true rankingalgorithm Is that correct? I also read something about

Re: NRT or similar for Solr 3.5?

@Steven .. try some alternate email address(besides google/yahoo) and check your spam [image: twitter] [image: facebook][image: google-buzz] [image: linkedin] R

Re: NRT or similar for Solr 3.5?

Yeah, running Chrome on OSX and doesn't do anything. Just switched to Firefox and it works. *But*, also don't seem to be receiving confirmation email. -- Steven Ou | 歐偉凡 *ravn.com* | Chief Technology Officer steve...@gmail.com | +1 909-569-9880 2011/12/12 vikram kamath > The Onclick handler d

Re: NRT or similar for Solr 3.5?

The Onclick handler does not seem to be called on google chrome (Ubuntu ). Also , I dont seem to receive the email with the confirmation link on registering (I have checked my spam) Regards Vikram Kamath 2011/12/12 Nagendra Nagarajayya > Steven: > > There is an onclick handler that allows

Re: NRT or similar for Solr 3.5?

Steven: There is an onclick handler that allows you to download the src. BTW, an early access Solr 3.5 with RankingAlgorithm 1.3 (NRT) release is available for download. So please give it a try. Regards, - Nagendra Nagarajayya http://solr-ra.tgels.org http://rankingalgorithm.tgels.org On 12/10

Re: NRT or similar for Solr 3.5?

All the links on the download section link to http://solr-ra.tgels.org/# -- Steven Ou | 歐偉凡 *ravn.com* | Chief Technology Officer steve...@gmail.com | +1 909-569-9880 2011/12/11 Nagendra Nagarajayya > Steven: > > Not sure why you had problems, #downloads ( > http://solr-ra.tgels.org/#downloads

Re: NRT or similar for Solr 3.5?

Steven: Not sure why you had problems, #downloads ( http://solr-ra.tgels.org/#downloads ) should point you to the downloads section showing the different versions available for download ? Please share if this is not so ( there were downloads yesterday with no problems ) Regarding NRT, you can swi

Re: NRT or similar for Solr 3.5?

Hey Nagendra, I took a look and Solr-RA looks promising - but: - I could not figure out how to download it. It seems like all the download links just point to "#" - I wasn't looking for another ranking algorithm, so would it be possible for me to use NRT but *not* RA (i.e. just use th

Re: NRT or similar for Solr 3.5?

Steven: Please take a look at Solr with RankingAlgorithm. It offers NRT functionality. You can set your autoCommit to about 15 mins. You can get more information from here: http://solr-ra.tgels.com/wiki/en/Near_Real_Time_Search_ver_3.x Regards, - Nagendra Nagarajayya http://solr-ra.tgels.o

Re: NRT and replication

thanks Mark and Tomas. Tomas, you mean doing soft commits to all the slave nodes right? If so, that is what I'm planning to do with the update processor commented above. 2011/10/21 Tomás Fernández Löbbe : > I was thinking in this, would it make sense to keep the master / slave > architecture, ad

Re: NRT and replication

I was thinking in this, would it make sense to keep the master / slave architecture, adding documents to the master and the slaves, do soft commits (only) to the slaves and hard commits to the master? That way you wouldn't be doing any merges on slaves. Would that make sense? On Fri, Oct 21, 2011

Re: NRT and replication

Yeah - a distributed update processor like the one Yonik wrote will do fine in simple situations. On Oct 17, 2011, at 7:33 PM, Esteban Donato wrote: > thanks Yonik. Any idea of when this should be completed? In the > meantime I think I will have to add docs to every replica, possibly > impleme

Re: NRT and replication

thanks Yonik. Any idea of when this should be completed? In the meantime I think I will have to add docs to every replica, possibly implementing an update processor. Something similar to SOLR-2355? On Fri, Oct 14, 2011 at 7:31 PM, Yonik Seeley wrote: > On Fri, Oct 14, 2011 at 5:49 PM, Esteban

Re: NRT and replication

On Fri, Oct 14, 2011 at 5:49 PM, Esteban Donato wrote: >  I found soft commits very useful for NRT search requirements. > However I couldn't figure out how replication works with this feature. >  I mean, if I have N replicas of an index for load balancing purposes, > when I soft commit a doc in on

Re: NRT and commit behavior

nsidered. > > > > I am trying to find the winning combination. > > Tirthankar > > -Original Message- > > From: Erick Erickson [mailto:erickerick...@gmail.com] > > Sent: Friday, September 16, 2011 7:46 AM > > To: solr-user@lucene.apache.org > > Subject: Re:

Re: NRT and commit behavior

ilto:erickerick...@gmail.com] > Sent: Friday, September 16, 2011 7:46 AM > To: solr-user@lucene.apache.org > Subject: Re: NRT and commit behavior > > Uhm, you're putting  a lot of index into not very much memory. I really think > you're going to have to shard your index ac

RE: NRT and commit behavior

] Sent: Friday, September 16, 2011 7:46 AM To: solr-user@lucene.apache.org Subject: Re: NRT and commit behavior Uhm, you're putting a lot of index into not very much memory. I really think you're going to have to shard your index across several machines to get past this probl

Re: NRT and commit behavior

che" >      size="512" >      initialSize="512" >      autowarmCount="512"/> > > -Original Message- > From: Tirthankar Chatterjee [mailto:tchatter...@commvault.com] > Sent: Wednesday, September 14, 2011 7:31 AM > To: solr-u

RE: NRT and commit behavior

Chatterjee [mailto:tchatter...@commvault.com] Sent: Wednesday, September 14, 2011 7:31 AM To: solr-user@lucene.apache.org Subject: RE: NRT and commit behavior Erick, Here is the answer to your questions: Our index is 267 GB We are not optimizing... No we have not profiled yet to check the bottleneck

RE: NRT and commit behavior

and Tomcat -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Sunday, September 11, 2011 11:37 AM To: solr-user@lucene.apache.org Subject: Re: NRT and commit behavior Hmm, OK. You might want to look at the non-cached filter query stuff, it's quite recent

Re: NRT and commit behavior

as false and found that inside the code it hard > coded to be true. Is there any specific reason. Can we change that value to > honor what is being passed. > > Thanks, > Tirthankar > > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: T

RE: NRT and commit behavior

--- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Thursday, September 01, 2011 8:38 AM To: solr-user@lucene.apache.org Subject: Re: NRT and commit behavior Hmm, I'm guessing a bit here, but using an invalid query doesn't sound very safe, but I suppose it *might* be OK.

Re: NRT and commit behavior

kar > > -Original Message- > From: Jonathan Rochkind [mailto:rochk...@jhu.edu] > Sent: Monday, July 18, 2011 11:38 AM > To: solr-user@lucene.apache.org; yo...@lucidimagination.com > Subject: Re: NRT and commit behavior > > In practice, in my experience at least, a ve

RE: NRT and commit behavior

er. Are we doing something wrong here? Thanks, Tirthankar -Original Message- From: Jonathan Rochkind [mailto:rochk...@jhu.edu] Sent: Monday, July 18, 2011 11:38 AM To: solr-user@lucene.apache.org; yo...@lucidimagination.com Subject: Re: NRT and commit behavior In practice, in my experienc

Re: NRT in Master- Slave setup, crazy?

On Aug 11, 2011, at 9:53 AM, eks dev wrote: > Thinking aloud and grateful for sparing .. > > I need to support high commit rate (low update latency) in a master > slave setup and I have a bad feelings about it, even with disabling > warmup and stripping everything down that slows down refresh.

Re: NRT in Master- Slave setup, crazy?

Hmmm, it almost seems like you're better off turning off replication entirely. Your "master" becomes a machine used as a source for rapidly spinning up a new slave or resetting a slave. I have no hard data to back up my misgivings about committing to the slaves then having replication overwrite th

Re: NRT and commit behavior

From one of the users of NRT, their system was freezing with commits at about 1.5 million docs due to the frequency of commits but with NRT (Solr with RankingAlgorithm) update document performance and a commit interval of about 15 mins they no longer have the freeze problem. Regards, - Nagen

Re: NRT and commit behavior

I've written a blog post on some of the recent improvements that explains things a bit: http://www.lucidimagination.com/blog/2011/07/11/benchmarking-the-new-solr-%E2%80%98near-realtime%E2%80%99-improvements/ On Jul 18, 2011, at 10:53 AM, Nicholas Chase wrote: > Very glad to hear that NRT is fin

Re: NRT and commit behavior

In practice, in my experience at least, a very 'expensive' commit can still slow down searches significantly, I think just due to CPU (or i/o?) starvation. Not sure anything can be done about that. That's my experience in Solr 1.4.1, but since searches have always been async with commits, it p

Re: NRT and commit behavior

On Mon, Jul 18, 2011 at 10:53 AM, Nicholas Chase wrote: > Very glad to hear that NRT is finally here!  But my question is this: will > things still come to a standstill during a commit? New updates can now proceed in parallel with a commit, and searches have always been completely asynchronous w.

Re: NRT facet search options comparison

endra Nagarajayya wrote: From: Nagendra Nagarajayya Subject: Re: NRT facet search options comparison To: solr-user@lucene.apache.org Date: Wednesday, June 1, 2011, 11:29 PM Hi Andy: Here is a white paper that shows screenshots of faceting working with Solr and RankingAlgorithm under NRT: http:/

Re: NRT facet search options comparison

hmark comparing the performance of facet search with and without NRT? Thanks Andy --- On Wed, 6/1/11, Nagendra Nagarajayya wrote: > From: Nagendra Nagarajayya > Subject: Re: NRT facet search options comparison > To: solr-user@lucene.apache.org > Date: Wednesday, June 1, 2011, 1

Re: NRT facet search options comparison

Hi Andy: Here is a white paper that shows screenshots of faceting working with Solr and RankingAlgorithm under NRT: http://solr-ra.tgels.com/wiki/en/Near_Real_Time_Search The implementation (src) is also available with the download and is described in the below document: http://solr-ra.tgels

Re: NRT and warmupTime of filterCache

>> Maybe the article is out of date? - maybe .. i dont know in my case it make no sense and i use another configuration ... - --- System One Server, 12 GB RAM, 2 Solr Instances, 7 Cores, 1 Core with 31 Million Document

Re: NRT and warmupTime of filterCache

> - yes, i think so, thats the reason because i dont understand the > wiki-article ... Maybe the article is out of date? I think it's grossly inefficient to warm the searchers at all in the NRT case. Queries are being performed across *all* segments, even though there should only be 1 that's new

Re: NRT in Solr

Bill, I think all of the improvements can be made, however they are fairly large structural changes that would require perhaps several patches. The other issue is we'll likely land RT this year (or next) and then the cached values need to be appended to as the documents are added, that and they'll

Re: NRT and warmupTime of filterCache

okay, not the time ... the items ... - --- System One Server, 12 GB RAM, 2 Solr Instances, 7 Cores, 1 Core with 31 Million Documents other Cores < 100.000 - Solr1 for Search-Requests - commit every Minute - 5GB Xmx - Solr

Re: NRT and warmupTime of filterCache

>> it'll negatively impact the desired goal of low latency new index readers? - yes, i think so, thats the reason because i dont understand the wiki-article ... i set the warmupCount to 500 and i got no error messages, that solr isnt available ... but solr-stats.jsp show me a warmuptime of "warmup

Re: NRT in Solr

Please start new threads for new conversations. On Wed, Mar 9, 2011 at 2:27 AM, stockii wrote: > question: http://wiki.apache.org/solr/NearRealtimeSearchTuning > > > 'PERFORMANCE WARNING: Overlapping onDeckSearchers=x > > i got this message. > in my solrconfig.xml: maxWarmingSearchers=4, if i set

Re: NRT in Solr

So it looks like can handle adding new documents, and expiring old documents. Updating a document is not part of the game. This would work well for message boards or tweet type solutions. Solr can do this as well directly. Why wouldn't you just improve the document and facet caching so that when y

Re: NRT in Solr

/ - Original Message > From: Jonathan Rochkind > To: "solr-user@lucene.apache.org" > Cc: "Smiley, David W." > Sent: Wed, March 9, 2011 5:34:01 PM > Subject: Re: NRT in Solr > > Interesting, does anyone have a summary of what techniques zoie u

Re: NRT in Solr

Interesting, does anyone have a summary of what techniques zoie uses to do this? I don't see any docs on the technical details. On 3/9/2011 5:29 PM, Smiley, David W. wrote: Zoie adds NRT to Solr: http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin I haven't tried it yet but looks c

Re: NRT in Solr

Zoie adds NRT to Solr: http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin I haven't tried it yet but looks cool. ~ David Smiley Author: http://www.packtpub.com/solr-1-4-enterprise-search-server/ On Mar 9, 2011, at 9:01 AM, Jason Rutherglen wrote: > Jae, > > NRT hasn't been implemen

Re: NRT and warmupTime of filterCache

I think it's best to turn the warmupCount to zero because usually there isn't time in between the creation of a new searcher to run the warmup queries, eg, it'll negatively impact the desired goal of low latency new index readers? On Wed, Mar 9, 2011 at 3:41 AM, stockii wrote: > I tried to create

Re: NRT in Solr

Jae, NRT hasn't been implemented NRT as of yet in Solr, I think partially because major features such as replication, caching, and uninverted faceting suddenly are no longer viable, eg, it's another round of testing etc. It's doable, however I think the best approach is a separate request call pa

  1   2   >