Hi all,
I have a similar question regarding a cluster configuration consisting
of HDDs, SSDs and NVMes. Let's say I would setup a OSD configuration in
a yaml file like this:
service_type:osd
service_id:osd_spec_default
placement:
host_pattern:'*'
spec:
data_devices:
model:HDD-Model-XY
db_devices:
model:NVME-Model-A
wal_devices:
model:NVME-Model-A Afaik that would result in having the db and wal on
the nvmes, whereas all data is put on the HDD. I assume I would get
significant IOPS performance when using Block device in Openstack (or
Proxmox), but I have to put up with wearing out the NVMes, right? In the
Ceph documentation, I saw that it would be also sufficient to define the
2,5" SSDs as db_devices. Nevertheless, I need some SSDs as OSDs since
the Ceph FS metadata pool and NFS pool needs to be put on a SSD OSD. I
might use 3 out of 4 SSDs each node as db_devices, but I need some for
the metadata pools. Any suggestions? Regards, Mevludin
Am 02.01.2023 um 15:03 schrieb Erik Lindahl:
Depends.
In theory, each OSD will have access to 1/4 of the separate WAL/DB device, so
to get better performance you need to find an NVMe device that delivers
significantly more than 4x the IOPS rate of the pm1643 drives, which is not
common.
That assumes the pm1643 devices are connected to a high-quality well-configured
12Gb SAS controller that really can deliver the full IOPS rate of 4 drives
combined. The only way to find that out is likely to benchmark.
Having said that, for a storage cluster where write performance is expected to
be the main bottleneck, I would be hesitant to use drives that only have 1DWPD
endurance since Ceph has fairly high write amplification factors. If you use
3-fold replication, this cluster might only be able to handle a few TB of
writes per day without wearing out the drives prematurely.
In practice we've been quite happy with Samsung drives that have often far
exceeded their warranty endurance, but that's not something I would like to
rely on when providing a commercial service.
Cheers,
Erik
--
Erik Lindahl<[email protected]>
On 2 Jan 2023 at 10:25 +0100,[email protected] <[email protected]>,
wrote:
Hi Experts,I am seeking for if there is achievable significant write
performance improvements when separating WAL/DB in a ceph cluster with all SSD
type OSD.I have a cluster with 40 SSD (PM1643 1.8 TB SSD Enterprise Samsung). I
have 10 Storage node each with 4 OSD. I want to know that can I get better
write IOPs and throughput if I add one NVMe OSD per node and separate WAL/DB on
it?Is the result of this separation, meaningful performance improvement or not?
My ceph cluster is block storage back-end of Openstack cinder in a public cloud
service.
Thanks in advance.
_______________________________________________
ceph-users mailing list [email protected]
To unsubscribe send an email [email protected]
_______________________________________________
ceph-users mailing list [email protected]
To unsubscribe send an email [email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
