On 26.09.25 09:17, Igor Korkin wrote:
Hi all,I'm observing a steady and abnormal increase in the `nr_cpu` value reported by the `credit` Xen scheduler (visible via `sudo xl dmesg; sudo xl debug-keys r`). This behavior occurs consistently when the system is subjected to heavy synthetic load (e.g., multiple VMs running stress workloads that fully saturate vCPUs). Over time, `nr_cpu` grows far beyond the actual number of physical or logical CPUs (48 in our case), and this correlates with noticeable performance degradation, especially under high VM density.
Can you please share the related output? Especially one instance of the "xl debug-keys r; xl dmesg" before the test is starting, and one with the wrong number of cpus in the data.
We’re running on a dual-socket x86_64 server (2 × 12-core Intel Xeon Silver 4310 CPUs with Hyper-Threading, total 48 logical CPUs) under Xen 4.19. Is this growth of `nr_cpu` expected in the credit scheduler? If not, it may indicate a bug in CPU accounting or runqueue management that warrants further investigation. Environment details: - xen_version : 4.19.0-5.25.0.38431 - xen_caps : xen-3.0-x86_64 hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_641 - xen_scheduler : credit - Hardware : Dual-socket Intel Xeon Silver 4310 @ 2.10GHz (12 cores/socket, HT enabled, 48 logical CPUs) - NUMA nodes : 2 - Dom0 kernel : Debian 6.1.147-1 (6.1.0-38-amd64, SMP PREEMPT_DYNAMIC) - nr_cpus : 48 - nr_nodes : 2 - release : 6.1.0-38-amd64 - version : #1 SMP PREEMPT_DYNAMIC Debian 6.1.147-1 (2025-08-02) - machine : x86_64 - nr_nodes : 2 - cores_per_socket : 12 - threads_per_core : 2 - cpu_mhz : 2100.000 - virt_caps : pv hvm hvm_directio pv_directio hap shadow iommu_hap_pt_share vmtrace gnttab-v1 gnttab-v2 - total_memory : 130724 - free_memory : 54064
Please share the complete "xl info" output, including the xen command line. Are there any cpupools involved? Any cpu hotplug operations? Juergen
OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
