siddharthteotia commented on a change in pull request #6876:
URL: https://github.com/apache/incubator-pinot/pull/6876#discussion_r626232264
##########
File path:
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/creator/SegmentGeneratorConfig.java
##########
@@ -262,6 +266,21 @@ private void
extractH3IndexConfigsFromTableConfig(TableConfig tableConfig) {
}
}
+ private void
extractNoDictionaryColumnCompressionCodecConfigsFromTableConfig(TableConfig
tableConfig) {
+ Map<String, String> rawIndexColumnsToCompressionTypeMap = new HashMap<>();
+ List<String> rawIndexColumns = new ArrayList<>();
+ List<FieldConfig> fieldConfigList = tableConfig.getFieldConfigList();
+ if (fieldConfigList != null) {
+ for (FieldConfig fieldConfig : fieldConfigList) {
+ if (fieldConfig.getEncodingType() == FieldConfig.EncodingType.RAW &&
fieldConfig.getNoDictionaryColumnCompressionCodec() != null) {
+ rawIndexColumns.add(fieldConfig.getName());
+ rawIndexColumnsToCompressionTypeMap.put(fieldConfig.getName(),
fieldConfig.getNoDictionaryColumnCompressionCodec().name());
+ }
+ }
+ }
+ setRawIndexColumnCompressionType(rawIndexColumns,
rawIndexColumnsToCompressionTypeMap);
Review comment:
I don't think this still fixes the problem. At line 200, the following
method will be called
```
public void setRawIndexCompressionType(Map<String, ChunkCompressionType>
rawIndexCompressionType) {
_rawIndexCompressionType.clear();
_rawIndexCompressionType.putAll(rawIndexCompressionType);
}
```
When it is called again at line 281, clear() will be called and old
compression config coming from noDictionaryConfig will be wiped out from it's
first invocation.
I think implementing the suggestion in
https://github.com/apache/incubator-pinot/pull/6876#discussion_r626128003 is a
simple fix. No need to call `setRawIndexColumnCompressionType`() from
`extractNoDictionaryColumnCompressionCodecConfigsFromTableConfig`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
