* Peter Xu ([email protected]) wrote: > On Tue, Feb 08, 2022 at 11:24:14AM +0000, Dr. David Alan Gilbert wrote: > > > The current model is we only have 1 postcopy channel and 1 precopy > > > channel, but > > > it should be easier if we want to make it N post + 1 pre base on this > > > series. > > > > It's not clear to me if we need to be able to do N post + M pre, or > > whether we have a rule like always at least 1 post, but if there's more > > pagefaults in the queue then you can steal all of the pre channels. > > Right, >1 queue length should easily happen with workloads in real cloud > environment. Though even with only 1post channel we can already hit at least > <~1ms with this series even if there're 16 pending requests per my test. I > think that may cover quite some real workloads. > > One thing to mention is that we should always assume the pre-channels are > filled up with tons of pages already in the NIC send buffer, so they won't be > good candidate for postcopy requests, IMHO. So I'm not sure whether we can > mixly use the pre/post channels - we may need to leave the post channels idle.
No I'm not sure either; even with separate channels do we have problems with contention on the NIC? Dave > Then, if we keep some of the multifd channels idle, then it will become some > other thing rather than the existing multifd, since we will start to treat > threads and channels differently and break the "equality" rule in the strict > version of multifd world. > > > > This also reminded me that, instead of a new capability, should I simply > > > expose > > > a parameter "postcopy-channels=N" to CLI so that we can be prepared with > > > multi > > > postcopy channels? > > > > I'm not sure we know enough yet about what configuration it would have; > > I'd be tempted to just make it work for the user by enabling both > > multifd and preemption and then using this new mechanism rather than > > having to add yet another parameter. > > Let me stick with the current capability bit then, so as to make it > 1pre+1post. > And we can leave Npre+1post for later. > > Thanks, > > -- > Peter Xu > -- Dr. David Alan Gilbert / [email protected] / Manchester, UK
