sandugood opened a new pull request, #2039:
URL: https://github.com/apache/datafusion-ballista/pull/2039
# Which issue does this PR close?
Closes #1980.
# Rationale for this change
Beforehand, only LZ4 compression type was being used in the shuffle write
process. Also, there was no way to disable compression at all.
# What changes are included in this PR?
Introduced new config entry: `ballista.shuffle.compression.codec` as well as
a helper function, that would pick up compression codec during runtime shuffle
write process.
# Are there any user-facing changes?
Yes, but not breaking. Compression codec defaults to "lz4" if not provided.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
