Satyr09 commented on PR #22649: URL: https://github.com/apache/datafusion/pull/22649#issuecomment-4622520751
> > I am working on a follow up that teaches the parallel writer in DF to honour this config as well - but I suspect that would be a larger change and might be better served by being its own PR. > > I think this is a good idea, I suggest to: > > 1. update the doc for the limitation > 2. Open an issue for the follow-up work > 3. Merge this PR first, then iterate on that Thanks @2010YOUY01 , added "Currently only honored when `allow_single_file_parallelism` is `false`; by default the parallel file writer ignores this limit. " to the documentation for the config. Will create the follow-up issue/PR soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
