Hello Julien, Thank you for the quick response. Please answers below.
________________________________ From: Julien Grall <[email protected]> Sent: Wednesday, July 21, 2021 4:09 PM To: Oleksii Moisieiev <[email protected]>; [email protected] <[email protected]> Cc: Andrii Anisov <[email protected]>; Stefano Stabellini <[email protected]> Subject: Re: Dom0 hangs after enable KROBE_EVENTS and/or UPROBE_EVENTS in kernel config >(+ Stefano) >On 21/07/2021 12:44, Oleksii Moisieiev wrote: >> Hello, >Hi, >Thanks for the report. >I nearly miss this e-mail because the title doesn't suggest this is an >Arm and I wasn't CCed. In future, would you be able to CC the Arm >maintainers (you can find them in MAINTAINERS) and mention arm in the title? I'm sorry for inconvenience, fixed topic and added arm maintainers to CC. >> I've got a problem that Dom0 hangs without any output from kernel once I >> enable CONFIG_KPROBE_EVENTS and/or CONFIG_UPROBE_EVENTS in dom0 kernel. >> Everything works fine when kprobe/uprobe events are disabled. >By disabled, do you mean compile out? Yes. Just changed .config lines: CONFIG_KPROBE_EVENTS=y CONFIG_UPROBE_EVENTS=y to #CONFIG_KPROBE_EVENTS is not set #CONFIG_UPROBE_EVENTS is not set and recompiled kernel. >> >> My setup: >> Board: H3ULCB Kinfisher board >> Xen: revision dba774896f7dd74773c14d537643b7d7477fefcd (stable-4.15) >> https://urldefense.com/v3/__https://github.com/xen-project/xen.git__;!!GF_29dbcQIUBPA!m4NHC2XbbSHWWZjQ7CX1ZZhaET6l0bQhZo581jtCmpst8E8JBp8Qri3haIaks6cbo7Ri$ >> [github[.]com] >> <https://urldefense.com/v3/__https://github.com/xen-project/xen.git__;!!GF_29dbcQIUBPA!m4NHC2XbbSHWWZjQ7CX1ZZhaET6l0bQhZo581jtCmpst8E8JBp8Qri3haIaks6cbo7Ri$ >> [github[.]com]>; >> Kernel: revision 09162bc32c880a791c6c0668ce0745cf7958f576 (v5.10-rc4) >Hmmm... 5.10 was released a few months ago and there are probably a few >stable release for the version. Can you try the latest 5.10 stable? Switched to tag v5.10 rev: 2c85ebc57b3e of https://github.com/torvalds/linux.git and got the same problem, that I see no output from kernel. All tests were done with earlycon parameter set in the kernel cmdline. >> >> https://urldefense.com/v3/__https://github.com/torvalds/linux.git__;!!GF_29dbcQIUBPA!m4NHC2XbbSHWWZjQ7CX1ZZhaET6l0bQhZo581jtCmpst8E8JBp8Qri3haIaks29w69MC$ >> [github[.]com] >> <https://urldefense.com/v3/__https://github.com/torvalds/linux.git__;!!GF_29dbcQIUBPA!m4NHC2XbbSHWWZjQ7CX1ZZhaET6l0bQhZo581jtCmpst8E8JBp8Qri3haIaks29w69MC$ >> [github[.]com]>; >> >> kernel config: see attached; >> >> dtb: see attached; >Please avoid large attachment as they will be duplicated on every >mailbox. Instead, in the future, please upload them somewhere (your own >webserve, pastebin...) and provide a link in the e-mail. I'm sorry for that. >> >> >> If kprobe/uprobe events are enabled - I see no output after xen switched >> input to Dom0, if disabled - system boots up successfully. >The console subsystem tends to be enabled quite late in the boot >process. So this may mean a panic during early boot. >If you haven't done yet, I would suggest to add earlycon=xenboot on the >dom0 command line. This will print some messages during early boot. >ing. All tests were done with earlycon parameter set in the kernel command line (xen, dom0-bootargs). >> >> Both configs work fine when I boot without xen. >> >> >> Dom0 information from Xen console shows that only one CPU works, and PC >> stays in "__arch_counter_get_cntvct" function on read_sysreg call. // >> >> I did further investigation and found that kernel 5.4 doesn't have such >> kind of issues. >> After bisecting kernel,between 5.10 and 5.4, I found that output >> disappeared on commit: >> >> 76085aff29f585139a37a10ea0a7daa63f70872c > From the information you provided so far, I am a bit confused how this >could be the source of the problem. But given this is not the latest >5.10, I will wait for you to confirm the bug is still present before >providing more input. I was confused with this commit either. As I mentioned above, I've checked with the latest stable 5.10 kernel and still got the same problem. >> >> >> Another issue, which was revealed after I got kernel output was kernel >> oops with message that CPU is in inconsistent state. >> >> [0.415612] EFI services will not be available. >> >> [0.420267] smp: Bringing up secondary CPUs ... >> >> [0.425185] Detected PIPT I-cache on CPU1 >> >> [0.425267] Xen: initializing cpu1 >> >> [0.425292] CPU1: Booted secondary processor 0x0000000001 [0x411fd073] >> >> [0.425815] Detected PIPT I-cache on CPU2 >> >> [0.425879] Xen: initializing cpu2 >> >> [0.425899] CPU2: Booted secondary processor 0x0000000002 [0x411fd073] >> >> [0.426362] Detected PIPT I-cache on CPU3 >> >> [0.426425] Xen: initializing cpu3 >> >> [0.426444] CPU3: Booted secondary processor 0x0000000003 [0x411fd073] >> >> [0.426537] smp: Brought up 1 node, 4 CPUs >> >> [0.472807] SMP: Total of 4 processors activated. >> >> [0.477551] CPU features: detected: 32-bit EL0 Support >> >> [0.482745] CPU features: detected: CRC32 instructions >> >> [0.499470] ------------[ cut here ]------------ >> >> [0.504034] CPU: CPUs started in inconsistent modes >Looking at Linux 5.7 code, this is printed when not all the CPUs are >booted in the same mode (i.e. EL1 or EL2). >This is quite odd. So let me ask a question first, did you see this >error during the bisection or on the latest 5.7? Switched to kernel v5.7 tag, rev:3d77e6a8804. On 5.7 kernel I can see printk output, but getting CPUs started in inconsistent modes error. Also, I tried with hmp-unsafe=false ( in xen cmdline, so only 0-3 CPU were enabled. And still got the same issue. >Cheers, >-- >Julien Grall
