---------- Forwarded message ----------
From: Peter Teoh <[email protected]>
Date: 2009/4/13
Subject: Re: Fw: systemtap on network I/O tracing under the KVM environment
To: zshan <[email protected]>
Cc: ananth <[email protected]>, acme <[email protected]>
based a simplified stap script (which breakpoints everything):
probe module("kvm").function("kv...@virt/kvm/*.c").call {
printf ("%s -> %s\n", thread_indent(1), probefunc())
print_backtrace();
}
probe module("kvm").function("kv...@arch/x86/kvm/*.c").call {
printf ("%s -> %s\n", thread_indent(1), probefunc())
print_backtrace();
}
I got the following trace (while logging into the kvm-modified guest
qemu - which is FC9, and host OS is FC10):
0xffffffffa01ce982 : kvm_arch_vcpu_load+0x1/0x32 [kvm]
0xffffffffa01cd0fa : vcpu_load+0x35/0x3a [kvm] (inexact)
0xffffffffa01cd1b4 : kvm_vcpu_block+0xb5/0xd1 [kvm] (inexact)
0xffffffffa01d3681 : kvm_arch_vcpu_ioctl_run+0x831/0xa2c [kvm] (inexact)
0xffffffffa01ccd7c : kvm_vm_ioctl+0x857/0x882 [kvm] (inexact)
0xffffffffa01ca623 : kvm_vcpu_ioctl+0xfb/0x617 [kvm] (inexact)
128 qemu-system-x86(11252): -> kvm_cpu_has_interrupt
0xffffffffa01e1594 : kvm_cpu_has_interrupt+0x1/0x40 [kvm]
0xffffffffa01cd165 : kvm_vcpu_block+0x66/0xd1 [kvm] (inexact)
0xffffffffa01d3681 : kvm_arch_vcpu_ioctl_run+0x831/0xa2c [kvm] (inexact)
0xffffffffa01ccd7c : kvm_vm_ioctl+0x857/0x882 [kvm] (inexact)
0xffffffffa01ca623 : kvm_vcpu_ioctl+0xfb/0x617 [kvm] (inexact)
243 qemu-system-x86(11252): -> kvm_cpu_has_pending_timer
0xffffffffa01e15d4 : kvm_cpu_has_pending_timer+0x1/0x29 [kvm]
0xffffffffa01cd171 : kvm_vcpu_block+0x72/0xd1 [kvm] (inexact)
0xffffffffa01d3681 : kvm_arch_vcpu_ioctl_run+0x831/0xa2c [kvm] (inexact)
0xffffffffa01ccd7c : kvm_vm_ioctl+0x857/0x882 [kvm] (inexact)
0xffffffffa01ca623 : kvm_vcpu_ioctl+0xfb/0x617 [kvm] (inexact)
357 qemu-system-x86(11252): -> kvm_inject_pending_timer_irqs
<snip>
1027 qemu-system-x86(11252): -> kvm_mmu_page_fault
0xffffffffa01d8e75 : kvm_mmu_page_fault+0x1/0x8b [kvm]
0xffffffffa01fa72b : __kvm_migrate_pit_timer+0x16c9d/0x0 [kvm] (inexact)
0xffffffffa01fab1b : __kvm_migrate_pit_timer+0x1708d/0x0 [kvm] (inexact)
0xffffffffa01d365e : kvm_arch_vcpu_ioctl_run+0x80e/0xa2c [kvm] (inexact)
0xffffffffa01ccd7c : kvm_vm_ioctl+0x857/0x882 [kvm] (inexact)
0xffffffffa01ca623 : kvm_vcpu_ioctl+0xfb/0x617 [kvm] (inexact)
1151 qemu-system-x86(11252): -> kvm_read_guest
0xffffffffa01cb8be : kvm_read_guest+0x1/0x83 [kvm]
0xffffffffa01d97a3 : paging32_walk_addr+0xad/0x24f [kvm] (inexact)
0xffffffffa01d9a09 : paging32_page_fault+0x73/0x424 [kvm] (inexact)
0xffffffffa0201950 : __kvm_migrate_pit_timer+0x1dec2/0x0 [kvm] (inexact)
0xffffffffa01d8e93 : kvm_mmu_page_fault+0x1f/0x8b [kvm] (inexact)
0xffffffffa01fa72b : __kvm_migrate_pit_timer+0x16c9d/0x0 [kvm] (inexact)
0xffffffffa01fab1b : __kvm_migrate_pit_timer+0x1708d/0x0 [kvm] (inexact)
0xffffffffa01d365e : kvm_arch_vcpu_ioctl_run+0x80e/0xa2c [kvm] (inexact)
0xffffffffa01ccd7c : kvm_vm_ioctl+0x857/0x882 [kvm] (inexact)
0xffffffffa01ca623 : kvm_vcpu_ioctl+0xfb/0x617 [kvm] (inexact)
1322 qemu-system-x86(11252): -> kvm_read_guest
0xffffffffa01cb8be : kvm_read_guest+0x1/0x83 [kvm]
0xffffffffa01d97a3 : paging32_walk_addr+0xad/0x24f [kvm] (inexact)
0xffffffffa01d9a09 : paging32_page_fault+0x73/0x424 [kvm] (inexact)
0xffffffffa0201950 : __kvm_migrate_pit_timer+0x1dec2/0x0 [kvm] (inexact)
0xffffffffa01d8e93 : kvm_mmu_page_fault+0x1f/0x8b [kvm] (inexact)
0xffffffffa01fa72b : __kvm_migrate_pit_timer+0x16c9d/0x0 [kvm] (inexact)
0xffffffffa01fab1b : __kvm_migrate_pit_timer+0x1708d/0x0 [kvm] (inexact)
0xffffffffa01d365e : kvm_arch_vcpu_ioctl_run+0x80e/0xa2c [kvm] (inexact)
0xffffffffa01ccd7c : kvm_vm_ioctl+0x857/0x882 [kvm] (inexact)
0xffffffffa01ca623 : kvm_vcpu_ioctl+0xfb/0x617 [kvm] (inexact)
1487 qemu-system-x86(11252): -> kvm_read_guest_page
0xffffffffa01cb865 : kvm_read_guest_page+0x1/0x59 [kvm]
0xffffffffa01cb8fe : kvm_read_guest+0x41/0x83 [kvm] (inexact)
0xffffffffa01d97a3 : paging32_walk_addr+0xad/0x24f [kvm] (inexact)
0xffffffffa01d9a09 : paging32_page_fault+0x73/0x424 [kvm] (inexact)
0xffffffffa0201950 : __kvm_migrate_pit_timer+0x1dec2/0x0 [kvm] (inexact)
0xffffffffa01d8e93 : kvm_mmu_page_fault+0x1f/0x8b [kvm] (inexact)
0xffffffffa01fa72b : __kvm_migrate_pit_timer+0x16c9d/0x0 [kvm] (inexact)
0xffffffffa01fab1b : __kvm_migrate_pit_timer+0x1708d/0x0 [kvm] (inexact)
0xffffffffa01d365e : kvm_arch_vcpu_ioctl_run+0x80e/0xa2c [kvm] (inexact)
0xffffffffa01ccd7c : kvm_vm_ioctl+0x857/0x882 [kvm] (inexact)
0xffffffffa01ca623 : kvm_vcpu_ioctl+0xfb/0x617 [kvm] (inexact)
1664 qemu-system-x86(11252): -> kvm_read_guest_page
etc etc.
Steps to reproduce are as follows:
1. download kvm-84, compile. the binaries qemu-* are actually
kvm-84's modified version of the original qemu.
2. compile linus git tree's kvm kernel module. then modprobe kvm.
3. qemu-img create -f qcow fc9_vdisk.img 10G
4. qemu-system-x86_64 -hda fc9_vdisk.img -cdrom
/sdb2/download/iso/Fedora-9-i386-DVD.iso ===> then install FC9.
5. qemu-system-x86_64 -hda fc9_vdisk.img ===> after installation,
reboot the guest fc9 this way.
6. Because kvm.ko is running in the host OS, just stap the commands
above in the host OS, and u should get the above tracing.
It clearly shows the path the host OS is taking to reroute to kvm.ko.
2009/4/9 zshan <[email protected]>:
> I would be appreciate if anyone can help me?
>
> 2009-04-09
> ________________________________
> zshan
> ________________________________
> 发件人： zshan
> 发送时间： 2009-04-09 23:00:25
> 收件人： systemtap
> 抄送： [email protected]; Mark Wielaard; Roland McGrath
> 主题： systemtap on network I/O tracing under the KVM environment
>
> hi:
> what I meant path tracing is that we want to get the ingressing path of a
> packet under the environment of KVM (Kernel based Virtual Machine ), what we
> want to know exactly is the ingressing path of a packet from the host
> machine to the virtual machine ,
> ps: KVM used Qemu to process its network I/O , which is actually a process
> in the host OS.
> Any ideas?
> Thank you very much~
>
> 2009-04-09
> ________________________________
> zshan
--
Regards,
Peter Teoh
--
Regards,
Peter Teoh
