> On Aug 20, 2025, at 1:57 PM, jbuburuzlist <[email protected]>
> wrote:
>
> Hello
>
> I just added more nodes to quincy cluster(existing drives are SAS). The new
> nodes are sata drives.
>
> The problem is after creating a new replicated crush rule and applying to a
> new pool. The ceph cluster is still using osd with "hdd" class and not using
> sata drives.
What command are you using to create an image? Does it specify the new pool
> I tried a number of crush rules using replicated and EC and I cannot get the
> pools to use sata osds.
Please send:
`ceph osd df`
`ceph osd crush rule dump`
`ceph osd dump | grep pool`
`ceph osd df tree`
Let's make sure that your new pool uses CRUSH rule 9.
>
> Here is crush rule for sata:
> {
> "rule_id": 9,
> "rule_name": "rbdsatareplicated",
> "type": 1,
> "steps": [
> {
> "op": "take",
> "item": -33,
> "item_name": "default~sata"
> },
> {
> "op": "chooseleaf_firstn",
> "num": 0,
> "type": "host"
> },
> {
> "op": "emit"
> }
> ]
> }
>
> I see in the crush rule "default~sata". I am wondering if this "default~" is
> causing me problems?
`default` is the CRUSH root. It is highly likely this is what you want.
> The ceph cluster is still using osd with "hdd" class and not using sata
> drives.
How are you determining this? Remember that by default RBD volumes are
thin-provisioned, and won't take up significant underlying capacity (a la `ceph
df) until data is written to them.
> Just another note. After I added the new sata nodes, my first crush rule used
> ec instead of replicated. I was getting errors when I tried to create a image
> on the ec pool "qemu-img error rbd create: Operation not supported on new
> pool".
When using EC for RBD you need a separate, replicated metadata pool. Did you
have one? Did you configure the client(s) to specify EC and both data and
metadata pools? Did you explicitly specify the EC pool?
rbd create --size IMAGE_SIZE --data-pool EC_POOL REPLICATED_POOL/IMAGE_NAME
? Did you set
ceph osd pool set <ERASURE_CODED_POOL_NAME> allow_ec_overwrites true
ceph osd pool set POOL_NAME application rbd
?
Note that with current Ceph releases the write latency to an EC RBD pool
usually precludes the use of EC, especially when backed by spinners.
That said, do you really need to segregate the SATA and SAS drives into
different pools?
> So I tried a test pool using replicated instead.
>
> Currently on the cluster I can create any number of pools and images if I use
> replicated/ec and the SAS drives which have a class of "hdd".
>
> Hope this makes sense?
>
> Here is some additional details:
>
> My existing nodes used SAS and are labelled:
>
> ceph osd tree
>
> ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
> -1 1122.64075 root default
> -3 34.03793 host node-01
> 0 hdd 2.26920 osd.0 up 1.00000 1.00000
> 1 hdd 2.26920 osd.1 up 1.00000 1.00000
> 2 hdd 2.26920 osd.2 up 1.00000 1.00000
> ..etc
>
> After adding SATA nodes I have:
>
>
>
> ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
> -1 1122.64075 root default
> -3 34.03793 host node-01
> 0 hdd 2.26920 osd.0 up 1.00000 1.00000
> 1 hdd 2.26920 osd.1 up 1.00000 1.00000
> 2 hdd 2.26920 osd.2 up 1.00000 1.00000
> ....
> ...
> -34 133.78070 host node-20
> 141 sata 11.14839 osd.141 up 1.00000 1.00000
> 142 sata 11.14839 osd.142 up 1.00000 1.00000
> 143 sata 11.14839 osd.143 up 1.00000 1.00000
> 144 sata 11.14839 osd.144 up 1.00000 1.00000
> 145 sata 11.14839 osd.145 up 1.00000 1.00000
> 146 sata 11.14839 osd.146 up 1.00000 1.00000
>
> ...
>
>
> Thanks
> jerry
> _______________________________________________
> ceph-users mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
