lidavidm commented on code in PR #108:
URL: https://github.com/apache/iceberg-cpp/pull/108#discussion_r2104016736
##########
src/iceberg/table_metadata.cc:
##########
@@ -153,14 +155,42 @@ Result<MetadataFileCodecType>
TableMetadataUtil::CodecFromFileName(
return MetadataFileCodecType::kNone;
}
+Result<std::string> DecompressGZIPFile(const std::string& filepath) {
+ gzFile file = gzopen(filepath.c_str(), "rb");
Review Comment:
How is this supposed to work with files that aren't local filesystem files?
##########
src/iceberg/table_metadata.cc:
##########
@@ -153,14 +155,42 @@ Result<MetadataFileCodecType>
TableMetadataUtil::CodecFromFileName(
return MetadataFileCodecType::kNone;
}
+Result<std::string> DecompressGZIPFile(const std::string& filepath) {
+ gzFile file = gzopen(filepath.c_str(), "rb");
+ if (!file) {
+ return IOError("Failed to open gzip file:{} ", filepath);
+ }
+
+ static const int CHUNK_SIZE = 32768; // 32KB chunks
+ std::array<char, CHUNK_SIZE> buffer;
+ std::string result;
+ int bytes_read;
+
+ while ((bytes_read = gzread(file, buffer.data(), CHUNK_SIZE)) > 0) {
+ result.append(buffer.data(), bytes_read);
+ }
+
+ int err;
+ const char* error_msg = gzerror(file, &err);
+ if (err != Z_OK) {
+ gzclose(file);
+ return IOError("Error during gzip decompression:{} ",
std::string(error_msg));
Review Comment:
I'm not convinced that `error_msg` is valid after `file` is closed.
##########
src/iceberg/table_metadata.cc:
##########
@@ -153,14 +155,42 @@ Result<MetadataFileCodecType>
TableMetadataUtil::CodecFromFileName(
return MetadataFileCodecType::kNone;
}
+Result<std::string> DecompressGZIPFile(const std::string& filepath) {
+ gzFile file = gzopen(filepath.c_str(), "rb");
+ if (!file) {
+ return IOError("Failed to open gzip file:{} ", filepath);
+ }
+
+ static const int CHUNK_SIZE = 32768; // 32KB chunks
+ std::array<char, CHUNK_SIZE> buffer;
+ std::string result;
+ int bytes_read;
+
+ while ((bytes_read = gzread(file, buffer.data(), CHUNK_SIZE)) > 0) {
+ result.append(buffer.data(), bytes_read);
+ }
+
+ int err;
+ const char* error_msg = gzerror(file, &err);
+ if (err != Z_OK) {
+ gzclose(file);
+ return IOError("Error during gzip decompression:{} ",
std::string(error_msg));
Review Comment:
In fact, if you peek at the implementation it points into the file structure
https://github.com/madler/zlib/blob/5a82f71ed1dfc0bec044d9702463dbdf84ea3b71/gzlib.c#L489-L505
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
