Hi,
Are there any performance tuning settings for RGW that you can share please
?
With the hardware I have, I believe I should be able to "squeeze" more
7 Supermicro dual CPU, 128 cores ( Intel Gold 6530), 1 TB RAM, 2 x 100GB
bonded cluster network, 2 x 25 GB bonded private network
cephadm, 5 RGW daemons (one per server) with haproxy in front of them
Uploading a 5.5 GB file using s3cmd takes 2m21 ( the client has a 10GB
connection)
hardware/other tweaks
HDD +DB/WAL on NVME
pool EC 4+2 with meta on NVME
vm.swapineess = 0
CPU intel_idle.max_cstate=0 processor.max_cstate=0
The servers have a less than 2 % CPU utilization and 30% memory usage
Cluster is not under heavy usage - I only have one client using the bucket
which has 7001 shards
I have been trying these ones with very little success
(taken from here
https://links.imagerelay.com/cdn/3404/ql/3cd809aeba7c42f395ca8a7256ba488a/BP-1072-SD20_Optimizing_Ceph_deployments_for_high_performance.pdf
)
rgw thread pool size = 512 #default 100, increase for heavier load
rgw max chunk size = 4194304# default is 128k, increase to 4MB helps with
larger objects rgw_obj_stripe_size = 4194304 # (default 4M for luminous)
rgw_list_bucket_min_readahead = 4000 #(default 1000)
rgw_max_listing_results = 4000
rgw_cache_expiry_interval = 1800 #(default 900s)
rgw_enable_usage_log = false
rgw_enable_ops_log = false
rgw dynamic resharding = false
rgw override bucket index max shards = 50 #
rgw bucket index max aio = 16 # default 8
rgw cache lru size = 50000 #number of entries in the cache
rgw_gc_obj_min_wait = 21600 #in seconds(default 2_hr)
rgw gc processor period = 7200 #(default 1hr)
rgw objexp gc interval = 3600 # default 10_min,
objecter inflight op bytes = 1073741824 # default 100_M, this is 1GB
objecter inflight ops = 24576
Many thanks
Steven
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
