pepijnve commented on PR #23522: URL: https://github.com/apache/datafusion/pull/23522#issuecomment-4984479940
I flipped around the nesting of `SpillPoolWriter` and `SharedSpillPoolWriter`. You can then create a `SpillPoolWriter` via `SharedSpillPoolWriter::new_writer`. This has the benefit of not needing the `SpillWriter` wrapper in `RepartitionExec`. The tradeoff is that the writer type itself no longer strictly communicates the spsc vs mpsc and ordering characteristic of the channel. We still retain the guarantee from the compiler that `channel` is single producer and `shared_channel` can be multi producer. I've edited the documentation a bit to be less verbose. Claude seemed kind of happy to repeat the fact that indeed SharedSpillPoolWriter _is_ Clone over and over again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
