>> Hmm... I'm not about device writing to memory.
>
> This absolutely is about whether the device wrote into the
> area or not.
Not only.
>> Sequence in igb driver is:
>>
>> dma_map(full_page)
>> <device writes here>
>> sync_to_cpu(half_page);
>> skb_add_rx_frag(skb, half_page);
>> napi_gro_receive(skb);
>> ...
>> dma_unmap(full_page)
>>
>> What I'm concerned about is - same area is first passed up to network
>> stack, and _later_ dma_unmap()ed. Is this indeed safe?
>
> dma_unmap() should never write anything unless the device has
> meanwhile written to that chunk of memory.
dma_unmap() for DMA_FROM_DEVICE never writes whatever to memory,
regardless of what device did.
dma_unmap() for DMA_FROM_DEVICE ensures that data written to memory
by device (if any) is visible to CPU. Cache may contain stale data
for that memory region. To drop that from cache, dma_unmap() for
DMA_FROM_DEVICE does cache invalidation.
static void arm_dma_unmap_page(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir, unsigned long attrs)
{
if ((attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
__dma_page_dev_to_cpu(pfn_to_page(dma_to_pfn(dev, handle)),
handle & ~PAGE_MASK, size, dir);
}
static void __dma_page_dev_to_cpu(struct page *page, unsigned long off,
size_t size, enum dma_data_direction dir)
{
...
if (dir != DMA_TO_DEVICE) {
outer_inv_range(paddr, paddr + size);
dma_cache_maint_page(page, off, size, dir, dmac_unmap_area);
}
...
}
> If the device made no intervening writes into the area, dma_unmap()
> should not cause any data to be written to that area, period.
I'm not about writing.
I'm about just the opposite - dropping not-written data from cache.
- napi_gro_receive(skb) passes area to upper layers of the network stack,
- something in those layers - perhaps packet mangling or such - writes
to the area,
- this write enters cache but does not end into memory immediately,
- at this moment, igb does dma_unmap(),
- write that was in cache but not yet in memory gets lost.
> In your example above, consider the case where the device never
> writes into the memory area after sync_to_cpu(). In that case
> there is nothing that dma_unmap() can possibly write.
> All the data has been synced
Non-synced data is write done by CPU executing upper layers of network stack,
Nikita
