-----Original Message----- From: Andrew Cooper <[email protected]> Sent: Wednesday, March 15, 2023 5:22 AM To: Johnson, Ethan <[email protected]>; [email protected] Subject: [EXT] Re: Best way to use altp2m to support VMFUNC EPT-switching?
> On 15/03/2023 2:01 am, Johnson, Ethan wrote: >> Hi all, >> >> I'm looking for some pointers on how Xen's altp2m system works and how it's >> meant to be used with Intel's VMFUNC EPT-switching for secure isolation >> within an HVM/PVH guest's kernelspace. >> >> Specifically, I am attempting to modify Xen to create (on request by an >> already-booted, cooperative guest with a duly modified Linux kernel) a >> second set of extended page tables that have access to additional >> privileged regions of host-physical memory (specifically, a page or two to >> store some sensitive data that we don't want the guest kernel to be able to >> overwrite, plus some host-physical MMIO ranges, specifically the xAPIC >> region). The idea is that the guest kernel will use VMFUNC to switch to the >> alternate EPTs and call "secure functions" provided (by the hypervisor) as >> read-only code to be executed in non-root mode on the alternate EPT, >> allowing certain VM-exit scenarios (namely, sending an IPI to another vCPU >> of the same domain) to be handled without exiting non-root mode. Hence, >> these extra privileged pages should only be visible to the alternative p2m >> that the "secure realm" functions live in. (Transitions between the secure- >> and insecure-realm EPTs will be through special read-only "trampoline" code >> pages that ensure the untrusted guest kernel can only enter the secure >> realm at designated entry points.) >> >> Looking at Xen's existing altp2m code, I get the sense that Xen is already >> designed to support something at least vaguely like this. I have not, >> however, been able to find much in the way of documentation on altp2m, so I >> am reaching out to see if anyone can offer pointers on how to best use it. >> >> What is the intended workflow (either in the toolstack or within the >> hypervisor itself) for creating and configuring an altp2m that should have >> access to additional host-physical frames that are not present in the >> guest's main p2m? >> >> FWIW, once the altp2m has been set up in this fashion, we don't anticipate >> needing to fiddle with its mappings any further as long as the guest is >> running (so I'm thinking *maybe* the "external" altp2m mode will suffice >> for this). In fact, we may not even need to have any "overlap" between the >> primary and alternative p2m except the trampoline pages themselves >> (although this aspect of our design is still somewhat in flux). >> >> I've noticed a function, do_altp2m_op(), in the hypervisor >> (xen/arch/x86/hvm/hvm.c) that seems to implement a number of altp2m-related >> hypercalls intended to be called from the dom0. Do these hypercalls already >> provide a straightforward way to achieve my goals described above entirely >> via (a potentially modified version of) the dom0 toolstack? Or would I be >> better off creating and configuring the altp2m from within the hypervisor >> itself, since I want to map low-level stuff like xAPIC MMIO ranges into the >> altp2m? >> >> Thank you in advance for your time and assistance! > > Hello, > > There's a lot to unpack here, but before I do so, one question. In your > usecase, are you wanting to map any frames with reduced permissions > (i.e. such that you'd get a #VE exception), or are you just looking to > add new frames with RWX perms into an alternative view? > > I suspect the latter, but it's not completely clear, and changes the answer. > > ~Andrew Yes, the latter is correct: I am looking to add new frames with RWX perms into an alternative view. I don't currently envision needing #VE in any form for this work. (We're using a modified PVH Linux guest for this, so rather than needing to intercept and react to EPT faults via #VE, we can expect the guest kernel to explicitly call our secure-realm functions via VMFUNC, replacing what would otherwise be some hypercalls out to Xen in root mode. I suppose supporting unmodified kernels by intercepting #VE could be an interesting enhancement for future work, but for now we're content to work with a cooperative modified PVH guest as a proof of concept. :-)) Basically, the primary p2m will be (largely) as it is normally, and the untrusted guest kernel and userspace will run on it as an HVM/PVH guest normally would. The alternate p2m will have some additional private code and data pages mapped in (RWX in the altp2m, but either read-only or completely unmapped in the primary p2m), as well as the host's xAPIC MMIO range so it can send IPIs to other vCPUs without having to VM-exit. To facilitate safe transitions between these two "realms", we'll be adding a couple of R/X-permissioned "trampoline pages" that will contain the VMFUNC instructions themselves and will be present in both p2ms. Thanks, Ethan Johnson Computer Science PhD candidate, Systems group, University of Rochester [email protected]
