On Tue, Dec 11, 2018 at 09:41:37AM +0000, David Laight wrote: > From: Josh Poimboeuf > > Sent: 30 November 2018 16:27 > > > > On Thu, Nov 29, 2018 at 03:04:20PM -0800, Linus Torvalds wrote: > > > On Thu, Nov 29, 2018 at 12:25 PM Josh Poimboeuf <[email protected]> > > > wrote: > ... > > > > Maybe that would be ok. If my math is right, we would use the > > > > out-of-line version almost 5% of the time due to cache misalignment of > > > > the address. > > > > > > Note that I don't think cache-line alignment is necessarily sufficient. > > > > > > The I$ fetch from the cacheline can happen in smaller chunks, because > > > the bus between the I$ and the instruction decode isn't a full > > > cacheline (well, it is _now_ in modern big cores, but it hasn't always > > > been). > > > > > > So even if the cacheline is updated atomically, I could imagine seeing > > > a partial fetch from the I$ (old values) and then a second partial > > > fetch (new values). > > > > > > It would be interesting to know what the exact fetch rules are. > > > > I've been doing some cross-modifying code experiments on Nehalem, with > > one CPU writing call destinations while the other CPUs are executing > > them. Reliably, one of the readers goes off into the weeds within a few > > seconds. > > > > The writing was done with just text_poke(), no #BP. > > > > I wasn't able to figure out the pattern in the addresses of the > > corrupted call sites. It wasn't cache line. > > > > That was on Nehalem. Skylake didn't crash at all. > > Interesting thought? > > If it is possible to add a prefix that can be overwritten by an int3 > is it also possible to add something that the assembler will use > to align the instruction so that a write to the 4 byte offset > will be atomic? > > I'd guess that avoiding 8 byte granularity would be sufficient. > So you'd need a 1, 2 or 3 byte nop depending on the actual > alignment - although a 3 byte one would always do.
The problem is that the call is done in C code, and we don't have a feasible way to use inline asm to call functions with more than five arguments. BTW, my original experiments (mentioned above) were a bit... flawed. I used text_poke(), which does memcpy(), which writes one byte at a time. No wonder it wasn't atomic. I'll need to do some more experiments. -- Josh
