Hello, I'm reading via s3a objects with key "something.json" which are gzip encoded with the content-encoding set as gzip. However spark on yarn doesn't seem to interpret the content-encoding header and so not reading via spark.read.json . Does anybody know how I can get spark to read based on the content-encoding metadata via s3a?
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
