Hi Glen, On 2/24/19 9:21 PM, Glen Baars wrote: > I am tracking down a performance issue with some of our mimic 13.2.4 OSDs. It > feels like a lack of memory but I have no real proof of the issue. I have > used the memory profiling ( pprof tool ) and the OSD's are maintaining their > 4GB allocated limit.
What are the symptoms? Does performance drop at a certain point? Did it drop compared to a previous configuration? You're saying that only *some* OSDs have a performance issue? > My questions are: > > 1.How do you know if the allocated memory is enough for the OSD? My 1TB disks > and 12TB disks take the same memory and I wonder if the OSDs should have > memory allocated based on the size of the disks? > 2.In the past, SSD disks needs 3 times the memory and now they don't, why is > that? ( 1GB ram per HDD and 3GB ram per SSD both went to 4GB ) I think you're talking about the BlueStore caching settings for SSDs and HDDs. You should take a look at the memory autotuning (notably osd_memory_target): http://docs.ceph.com/docs/mimic/rados/configuration/bluestore-config-ref/#automatic-cache-sizing > 3.I have read that the number of placement groups per OSD is a significant > factor in the memory usage. Generally I have ~200 placement groups per OSD, > this is at the higher end of the recommended values and I wonder if its > causing high memory usage? > > For reference the hosts are 1 x 6 core CPU, 72GB ram, 14 OSDs, 2 x 10Gbit. > LSI cachecade / writeback cache for the HDD and LSI JBOD for SSDs. 9 hosts in > this cluster. > > Kind regards, > Glen Baars > This e-mail is intended solely for the benefit of the addressee(s) and any > other named recipient. It is confidential and may contain legally privileged > or confidential information. If you are not the recipient, any use, > distribution, disclosure or copying of this e-mail is prohibited. The > confidentiality and legal privilege attached to this communication is not > waived or lost by reason of the mistaken transmission or delivery to you. If > you have received this e-mail in error, please notify us immediately. > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
