New submission from Giovanni Pizzi :
Hello,
when creating deleting (with `os.remove`/`os.unlink`) a file and opening it in
a different process at (almost) the same time, on Mac OS X relatively often I
get a file that is empty instead of either a `FileNotFoundError` exception, or
an open handle with the original content, i.e. at least one of the two
operations (unlinking or opening) seems to be non-atomic.
(With empty I mean that if after I open the file (mode 'rb'), then
`fhandle.read()` returns me b''.
More in particular, after quite some debugging I noticed that what happens is
that if I stat the file descriptor, `st_ino` is zero.
This can reproduced very easily.
I set up a GitHub repository here:
https://github.com/giovannipizzi/concurrent-delete-race-problem
with simple examples and tests (and both a Python implementation and a C
implementation).
There are also GitHub Actions that run on Mac, Ubuntu and Windows, and
apparently the problem exists only on Mac (and happens on all versions).
For completeness I attach also here a tar.gz with the two very short python
files that just need be run in parallel - in my Mac OS X (macOS 10.14.6, 15"
2016 MacBook Pro, python 3.6) I get the error essentially at every run.
Important: while much harder to reproduce, I can get the same error and
behaviour also with the (I think equivalent) C code.
Therefore, this seems to be more a problem with the Mac OS standard libraries?
My first question is: has anybody ever seen this problem? It seems to me quite
easy to reproduce, but I'm surprised that I couldn't find any reference in the
internet even after searching for various hours (maybe I used the wrong
keywords?)
Second question: should this be reported directly to Apple?
Third question: Even if this is a bug, would it make sense to implement a patch
in python that raises some exception if st_ino is zero when opening a file? Or
am I simplifying too much and in some conditions st_ino=0 is valid on some
types of mounted filesystems? Is there some other way to have a more atomic
behaviour for this race condition in python?
Thanks a lot!
--
components: IO, macOS
files: concurrency-tests.tar.gz
messages: 373606
nosy: Giovanni Pizzi, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Race conditions when opening and deleting a file on Mac OS X
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file49315/concurrency-tests.tar.gz
___
Python tracker
<https://bugs.python.org/issue41291>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
