Thanks for the 7.3-rc3 test. It was a genuine negative: two of the patches were 
defective
and the reproducer's 50 GB anchor hides the fix anyway. Both are corrected and 
verified;
details are in the attached tarball (REPLY-SUMMARY.md, patches, VM harness and 
logs).

What was wrong: patch 2 filtered on HK_TYPE_KTHREAD, which 8f78b749f3da 
(v7.1-rc3) made an
alias of the domain mask, so it did nothing on 7.1+; it now names 
HK_TYPE_KERNEL_NOISE.
Patch 3 let a folio sit in the per-CPU batch for a few instructions, which the 
lock-free
counter check in __lru_add_drain_all() was catching; it now uses an on-stack 
batch. 7.3
additionally flushes the migration workqueue from cpuset_fork() (65e510cd30d0), 
which PID 1
hits on every service start; new patch 6 fixes that (7.0/7.2 don't have the 
path).

Verification: QEMU/KVM VM, real aws config, isolcpus=nohz/nohz_full CPUs in a 
non-last RCU
leaf, a SCHED_FIFO/80 fault-storm spinner on each. Unpatched 
7.0.0-1004/1006/1008/1011-aws
all hang indefinitely in synchronize_rcu_expedited <- lru_cache_disable with 
the comment-#2
stack (same offsets) once the fair server can't help, and queue 16-28 drain 
work items onto
the isolated CPUs per cpuset.mems write. With the series (tested on 1004 and 
1013): RCU/kthread
workers on housekeeping CPUs only, nothing queued to isolated CPUs, a 
30-process cpuset.mems
write 6.5 s -> 0.07 s, O_NONBLOCK write (systemd >= 259) returns in 0 ms with 
the migration
completing in the background (512 MB anchor fully moved in 0.3 s vs 1.6 s 
unpatched).

The "no NUMA migration" observation: with patch 5 the write returns at once and 
the migration
runs in the background; on your build the inert patch 2 left ~150 one-second 
waits queued
ahead of the anchor, and do_migrate_pages() isolates all 13 M folios before 
pgmigrate_success
or numastat move at all. `validate.sh migrate 900 anchor_alloc` shows the 
progress live.

Series: patches/0001-0005 for resolute (base 7.0.0-1013.13, applies to 
1004.4-1013.13),
patches-mainline/0001-0006 for 7.3. For the c8a rerun please use a 2 GiB anchor 
(GIB_50 in
anchor_alloc.c) or validate.sh migrate/trace, and check 
/sys/kernel/debug/sched/fair_server/cpuN/runtime on an isolated CPU (50000000 
vs 0).


** Attachment added: "lp2165410-proposal.tar.gz"
   
https://bugs.launchpad.net/ubuntu/+source/linux-aws/+bug/2165410/+attachment/6000646/+files/lp2165410-proposal.tar.gz

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2165410

Title:
  Ubuntu LTS 26.04 linux-aws: systemd enters D state and blocks SSH
  during cpuset migration on nohz_full CPUs

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-aws/+bug/2165410/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to