faucct opened a new issue, #48332:
URL: https://github.com/apache/arrow/issues/48332
### Describe the enhancement requested
Our C++ application is using fibers for concurrency inside its
`RandomAccessFile` implementation.
`arrow::*Reader` classes switch contexts and blocks threads interacting with
them.
This causes thread starvation and a dead-lock inside our `RandomAccessFile`
implementation, as it needs free threads to finish the work started by
`ReadNext`.
I have tried to completely avoid hidden threads and context switches here,
by implementing a direct executor:
```
static struct
: public arrow::internal::Executor
{
int GetCapacity() override { return 1; };
arrow::Status SpawnReal(
arrow::internal::TaskHints,
arrow::internal::FnOnce<void()> task,
arrow::StopToken,
StopCallback&&) override
{
std::move(task)();
return arrow::Status::OK();
}
} DirectExecutor;
```
Unfortunately, this also blocks, because of the recursive lock happening
here:
https://github.com/apache/arrow/blob/apache-arrow-22.0.0/cpp/src/arrow/util/async_generator.h#L1708
https://github.com/apache/arrow/blob/apache-arrow-22.0.0/cpp/src/arrow/util/async_generator.h#L1776
Is it possible to rewrite `DoRestartTask` in a way, so it would release the
lock and if `!spawn_status.ok()`, retake it back?
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
