On Mon, 2019-04-01 at 14:01 +0000, Elijah Shakkour wrote: > > > -----Original Message----- > > From: Peter Xu <pet...@redhat.com> > > Sent: Monday, April 1, 2019 1:25 PM > > To: Elijah Shakkour <elija...@mellanox.com> > > Cc: Knut Omang <knut.om...@oracle.com>; Michael S. Tsirkin > > <m...@redhat.com>; Alex Williamson <alex.william...@redhat.com>; > > Marcel Apfelbaum <marcel.apfelb...@gmail.com>; Stefan Hajnoczi > > <stefa...@gmail.com>; qemu-devel@nongnu.org > > Subject: Re: QEMU and vIOMMU support for emulated VF passthrough to > > nested (L2) VM > > > > On Mon, Apr 01, 2019 at 09:12:38AM +0000, Elijah Shakkour wrote: > > > > > > > > > > -----Original Message----- > > > > From: Peter Xu <pet...@redhat.com> > > > > Sent: Monday, April 1, 2019 5:47 AM > > > > To: Elijah Shakkour <elija...@mellanox.com> > > > > Cc: Knut Omang <knut.om...@oracle.com>; Michael S. Tsirkin > > > > <m...@redhat.com>; Alex Williamson <alex.william...@redhat.com>; > > > > Marcel Apfelbaum <marcel.apfelb...@gmail.com>; Stefan Hajnoczi > > > > <stefa...@gmail.com>; qemu-devel@nongnu.org > > > > Subject: Re: QEMU and vIOMMU support for emulated VF passthrough > > to > > > > nested (L2) VM > > > > > > > > On Sun, Mar 31, 2019 at 11:15:00AM +0000, Elijah Shakkour wrote: > > > > > > > > [...] > > > > > > > > > I didn't have DMA nor MMIO read/write working with my old command > > > > line. > > > > > But, when I removed all CPU flags and only provided "-cpu host", I > > > > > see that > > > > MMIO works. > > > > > Still, DMA read/write from emulated device doesn't work for VF. > > > > > For > > > > example: > > > > > Driver provides me a buffer pointer through MMIO write, this > > > > > address > > > > (pointer) is GPA of L2, and when I try to call pci_dma_read() with > > > > this address I get: > > > > > " > > > > > Unassigned mem read 0000000000000000 " > > > > > > > > I don't know where this error log was dumped but if it's during DMA > > > > then I agree it can probably be related to vIOMMU. > > > > > > > > > > This log is dumped from: > > > memory.c: unassigned_mem_read() > > > > > > > > As I said, my problem now is in translation of L2 GPA provided by > > > > > driver, > > > > when I call DMA read/write for this address from VF. > > > > > Any insights? > > > > > > > > I just noticed that you were using QEMU 2.12 [1]. If that's the > > > > case, please rebase to the latest QEMU, at least >=3.0 because > > > > there's major refactor of the shadow logic during 3.0 devel cycle > > > > AFAICT. > > > > > > > > > > Rebased to QEMU 3.1 > > > Now I see the address I'm trying to read from in log but still same error: > > > " > > > Unassigned mem read 00000000f0481000 > > > " > > > What do you suggest? > > > > Would you please answer the questions that Knut asked? Is it working for L1 > > guest? How about PF? > > Both VF and PF are working for L1 guest. > I don't know how to passthrough PF to nested VM in hyper-v.
On Linux passing through VFs and PFs are the same. Maybe you can try passthrough with all Linux first? (first PF then VF) ? > I don't invoke VF manually in hyper-v and pass it through to nested VM. I use > hyper-v > manager to configure and provide a VF for nested VM (I can see the VF only in > the nested > VM). > > Did someone try to run emulated device in linux RH as nested L2 where L1 is > windows > hyper-v? Does DMA read/write work for this emulated device in this case? I have never tried that, I have only used Linux as L2, Windows might be pickier about what it expects, so starting with Linux to rule that out is probably a good idea. > > > > You can also try to enable VT-d device log by appending: > > > > -trace enable="vtd_*" > > > > In case it dumps anything useful for you. > > Is there a way to open those traces to be dumped to stdout/stderr on the fly, > instead of > dtrace? It's up to you what tracer(s) to configure when you build QEMU - check out docs/devel/tracing.txt . There's a few trace events defined in the SR/IOV patch set, you might want to enable them as well. Knut > > -- > > Peter Xu
