Also a good rule of thumb I found is set your xmx and xms, maximum and minimum memory for the heap to be exactly the same, you don’t want Java to try to figure it out,
> On Jul 5, 2022, at 7:52 AM, Ritvik Sharma <[email protected]> wrote: > > Just Add java memory parameters in solr config which should not be more > than 75% of total RAM. and use G1GC. > > >> On Tue, 5 Jul 2022 at 4:59 PM, Deepak Goel <[email protected]> wrote: >> >> I would also suggest you look at which GC mechanism you use. Increasing RAM >> and Heap-Size might result in the application freezed for a long time >> (during GC). >> >> Deepak >> "The greatness of a nation can be judged by the way its animals are treated >> - Mahatma Gandhi" >> >> +91 73500 12833 >> [email protected] >> >> Facebook: https://www.facebook.com/deicool >> LinkedIn: www.linkedin.com/in/deicool >> >> "Plant a Tree, Go Green" >> >> Make In India : http://www.makeinindia.com/home >> >> >>> On Tue, Jul 5, 2022 at 4:43 PM Dave <[email protected]> wrote: >>> >>> Exactly. You could have the best engineer on your continent but the end >>> result is the same, more metal. I could build a very fast search server >>> for less than a week of my salary so what’s the point of wasting two >> weeks >>> trying to solve a problem when the solution is literally just right >> there, >>> a big ssd and a lot of memory, then I could work on things that are >>> actually important rather than try to squeeze blood from a turnip. >>> >>>> On Jul 5, 2022, at 6:11 AM, Charlie Hull < >>> [email protected]> wrote: >>>> >>>> I think you're missing my point. >>>> >>>> Good engineers, even mediocre ones, are expensive, and the great ones >>> are rare. It's a tedious task chasing tiny performance gains when you >> know >>> you're limited by the hardware and a bored engineer might just go and >> look >>> for another job. So if you fail to realise that a capital expense for >>> hardware or hosting is necessary, you run the risk of losing the people >>> that make your search engine work (even if they stay they could also be >>> doing something more useful to your business). >>>> >>>> Charlie >>>> >>>>> On 05/07/2022 10:49, Deepak Goel wrote: >>>>> If you are tearing your hair out on 'Number of Hours' required for >>> tuning >>>>> your software, it's time you switch to a better quality performance >>>>> engineer. >>>>> >>>>> Deepak >>>>> "The greatness of a nation can be judged by the way its animals are >>> treated >>>>> - Mahatma Gandhi" >>>>> >>>>> +91 73500 12833 >>>>> [email protected] >>>>> >>>>> Facebook:https://www.facebook.com/deicool >>>>> LinkedIn:www.linkedin.com/in/deicool >>>>> >>>>> "Plant a Tree, Go Green" >>>>> >>>>> Make In India :http://www.makeinindia.com/home >>>>> >>>>> >>>>> On Tue, Jul 5, 2022 at 3:12 PM Charlie Hull< >>> [email protected]> >>>>> wrote: >>>>> >>>>>> Equally it's not a good management practice to burn engineering hours >>>>>> trying to optimise performance to avoid spending (often much less) >>> money >>>>>> on sufficient hardware to do the job. I've seen this happen many >> times, >>>>>> sadly. >>>>>> >>>>>> Charlie >>>>>> >>>>>> On 05/07/2022 10:33, Deepak Goel wrote: >>>>>>> Not a good software engineering practice to beef up the hardware >>> blindly. >>>>>>> Of Course when you have tuned the software to a point where you >> can't >>>>>> tune >>>>>>> anymore, you can then turn your eyes to hardware. >>>>>>> >>>>>>> Deepak >>>>>>> "The greatness of a nation can be judged by the way its animals are >>>>>> treated >>>>>>> - Mahatma Gandhi" >>>>>>> >>>>>>> +91 73500 12833 >>>>>>> [email protected] >>>>>>> >>>>>>> Facebook:https://www.facebook.com/deicool >>>>>>> LinkedIn:www.linkedin.com/in/deicool >>>>>>> >>>>>>> "Plant a Tree, Go Green" >>>>>>> >>>>>>> Make In India :http://www.makeinindia.com/home >>>>>>> >>>>>>> >>>>>>> On Tue, Jul 5, 2022 at 1:01 AM Dave<[email protected]> >>>>>> wrote: >>>>>>>> Also for $115 I can buy a terabyte of a Samsung ssd, which helps a >>> lot. >>>>>> It >>>>>>>> comes to a point where money on hardware will outweigh money on >>>>>> engineering >>>>>>>> man power hours, and still come to the same conclusion. As much ram >>> as >>>>>> your >>>>>>>> rack can take and as big and fast of a raid ssd drive it can take. >>>>>> Remember >>>>>>>> since solr is always meant to be destroyed and recreated you don’t >>> have >>>>>> to >>>>>>>> worry much about hardware failure if you just buy two of everything >>> and >>>>>>>> have a backup server ready and waiting to take over while the >>> original >>>>>>>> fails and is reconstructed. >>>>>>>> >>>>>>>>> On Jul 4, 2022, at 1:32 PM, Shawn Heisey<[email protected]> >>> wrote: >>>>>>>>> >>>>>>>>> On 7/4/22 03:01, Mike wrote: >>>>>>>>>> My Solr index size is around 500GB and I have 64GB of RAM. Solr >>> eats >>>>>> up >>>>>>>> all >>>>>>>>>> the memory and because of that PHP works very, very slowly. What >>> can I >>>>>>>> do? >>>>>>>>> Solr is a Java program. A Java program will never directly use >> more >>>>>>>> memory than you specify for the max heap size. We cannot make any >>>>>> general >>>>>>>> recommendations about what heap size you need, because there is a >>> good >>>>>>>> chance that any recommendation we make would be completely wrong >> for >>>>>> your >>>>>>>> install. I did see that someone recommended not going above 31G >> ... >>> and >>>>>>>> this is good advice. At 32 GB, Java switches to 64-bit pointers >>>>>> instead of >>>>>>>> 32-bit. So a heap size of 32 GB actually has LESS memory available >>>>>> than a >>>>>>>> heap size of 31 GB. >>>>>>>>> The OS will use additional memory beyond the heap for caching the >>> index >>>>>>>> data, but that is completely outside of Solr's control. Note that >>> 64GB >>>>>>>> total memory for a 500GB index is almost certainly not enough >> memory, >>>>>>>> ESPECIALLY if the same server is used for things other than Solr. >> I >>>>>> wrote >>>>>>>> the following wiki page: >>>>>> >>> https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems >>>>>>>>> Others have recommended that you run Solr on dedicated hardware >>> that is >>>>>>>> not used for any other purpose. I concur with that recommendation. >>>>>>>>> Thanks, >>>>>>>>> Shawn >>>>>>>>> >>>>>> -- >>>>>> Charlie Hull - Managing Consultant at OpenSource Connections Limited >>>>>> Founding member of The Search Network< >> http://www.thesearchnetwork.com> >>>>>> and co-author of Searching the Enterprise >>>>>> < >>>>>> >>> >> https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf >>>>>> tel/fax: +44 (0)8700 118334 >>>>>> mobile: +44 (0)7767 825828 >>>>>> >>>>>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin >>>>>> Amtsgericht Charlottenburg | HRB 230712 B >>>>>> Geschäftsführer: John M. Woodell | David E. Pugh >>>>>> Finanzamt: Berlin Finanzamt für Körperschaften II >>>>>> >>>>>> -- >>>>>> This email has been checked for viruses by AVG. >>>>>> https://www.avg.com >>>>>> >>>> -- >>>> Charlie Hull - Managing Consultant at OpenSource Connections Limited >>>> Founding member of The Search Network <http://www.thesearchnetwork.com >>> >>> and co-author of Searching the Enterprise < >>> >> https://opensourceconnections.com/wp-content/uploads/2020/08/ES_book_final_journal_version.pdf >>>> >>>> tel/fax: +44 (0)8700 118334 >>>> mobile: +44 (0)7767 825828 >>>> >>>> OpenSource Connections Europe GmbH | Pappelallee 78/79 | 10437 Berlin >>>> Amtsgericht Charlottenburg | HRB 230712 B >>>> Geschäftsführer: John M. Woodell | David E. Pugh >>>> Finanzamt: Berlin Finanzamt für Körperschaften II >>>> >>>> -- >>>> This email has been checked for viruses by AVG. >>>> https://www.avg.com >>> >>
