I have 2 clusters; both have HDDs and SSDs. Reporting only the HDDs which have
their own pools:
"rep-cluster": hdd-pool 3-replication, 86 OSDs (16 TiB each), 1024 PGs, 78
%RAW USED, 100 M objects
"ec-cluster": hdd-pool erasure k=4 m=2, 58 OSDs (16 TiB each), 256 PGs, 60
%RAW USED, 450 M objects
Both are Ceph 18.2.1, Bluestore, and have the autoscaler enabled.
As you can see, I have many small objects.
My PGs-copies-per-OSD seem far off from the recommendation of 100 PGs per OSD
(`mon_target_pg_per_osd`):
rep-cluster: 35 PGs/OSD (= 1024*3/86)
ec-cluster: 26 PGs/OSD (= 256*6/58)
So I'm at least 3x-4x off.
Why?
Should the autoscaler not have increased the PGs here?
`ceph osd pool autoscale-status`:
rep-cluster:
POOL SIZE TARGET SIZE RATE RAW CAPACITY RATIO TARGET RATIO
EFFECTIVE RATIO BIAS PG_NUM NEW PG_NUM AUTOSCALE BULK
data 349.4T 3.0 1343T 0.7802
1.0 1024 on False
ec-cluster:
data_ec 347.3T 1.5 876.4T 0.5944
1.0 256 on False
I believe that because of this I suffer some drawbacks:
* On ec-cluster, a PG contains ~2 TiB and ~2 M objects, causing rebalances to
happen in coarse, slow steps.
Should I take some steps to force the autoscaler to increase PGs, and if yes,
which approach would be best here?
Thanks for your tips!
Niklas
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
