On Tue, Sep 10, 2019 at 05:14:36PM +0200, Johannes Berg wrote: > On Tue, 2019-09-10 at 17:03 +0200, Stefan Hajnoczi wrote: > > > > > Now, this means that the CPU (that's part of the simulation) has to > > > *wait* for the device to add an entry to the simulation calendar in > > > response to the kick... That means that it really has to look like > > > > > > CPU device calendar > > > ---[kick]--> > > > ---[add entry]--> > > > <---[return]----- > > > > What are the semantics of returning from the calendar? Does it mean > > "it's now your turn to run?", "your entry has been added and you'll be > > notified later when it's time to run?", or something else? > > The latter - the entry was added, and you'll be notified when it's time > to run; but we need to have that state on the calendar so the CPU won't > converse with the calendar before that state is committed.
Is the device only adding a calendar entry and not doing any actual
device emulation at this stage?
If yes, then this suggests the system could be structured more cleanly.
The vhost-user device process should focus on device emulation. It
should not be aware of the calendar. The vhost-user protocol also
shouldn't require modifications.
Instead, Linux arch/um code would add the entry to the calendar when the
CPU wants to kick a vhost-user device. I assume the CPU is suspended
until arch/um code completes adding the entry to the calendar.
When the calendar decides to run the device entry it signals the
vhost-user kick eventfd. The vhost-user device processes the virtqueue
as if it had been directly signalled by the CPU, totally unaware that
it's running within a simulation system.
The irq path is similar: the device signals the callfd and the calendar
adds an entry to notify UML that the request has completed.
Some pseudo-code:
arch/um/drivers/.../vhost-user.c:
void um_vu_kick(struct um_vu_vq *vq)
{
if (simulation_mode) {
calendar_add_entry({
.type = CAL_ENTRY_TYPE_VHOST_USER_KICK,
.device = vq->dev,
.vq_idx = vq->idx,
});
return;
}
/* The normal code path: signal the kickfd */
uint64_t val = 1;
write(vq->kickfd, &val, sizeof(val));
}
I'm suggesting this because it seems like a cleaner approach than
exposing the calendar concept to the vhost-user devices. I'm not 100%
sure it offers the semantics you need to make everything deterministic
though.
A different topic: vhost-user does not have a 1:1 vq buffer:kick
relationship. It's possible to add multiple buffers and kick only once.
It is also possible for the device to complete multiple buffers and only
call once.
This could pose a problem for simulation because it allows a degree of
non-determinism. But as long as the both the CPU and the I/O completion
of the device happen on a strict schedule this isn't a problem.
Stefan
signature.asc
Description: PGP signature
