annurahar opened a new pull request, #15393:
URL: https://github.com/apache/iceberg/pull/15393
## Description
This PR fixes a bug where the `iceberg.tables.schema-case-insensitive`
configuration is ignored when name mapping (`schema.name-mapping.default`) is
present in `RecordConverter`.
### Problem
When both name mapping and case-insensitive mode are configured, the
`lookupStructField` method does not apply case-insensitive logic for field
lookups. This causes fields with case differences (e.g., `"II"` vs `"ii"`) to
fail matching the existing column.
During schema evolution, the unmatched field is treated as a new field,
causing a duplicate column to be added to the table. This results in unintended
schema changes with redundant columns and data written to wrong columns.
### Solution
Modified `lookupStructField` and `createStructNameMap` methods to respect
the `schemaCaseInsensitive` configuration when name mapping is present:
1. **`createStructNameMap`**: Store keys in lowercase when case-insensitive
mode is enabled
2. **`lookupStructField`**: Convert the lookup key to lowercase when
case-insensitive mode is enabled
### Changes
```java
// lookupStructField - Before
return structNameMap
.computeIfAbsent(structFieldId, notUsed -> createStructNameMap(schema))
.get(fieldName);
// lookupStructField - After
Map<String, NestedField> nameMap =
structNameMap.computeIfAbsent(structFieldId, notUsed ->
createStructNameMap(schema));
String lookupKey = config.schemaCaseInsensitive() ? fieldName.toLowerCase()
: fieldName;
return nameMap.get(lookupKey);
```
```java
// createStructNameMap - Before
if (mappedField != null && !mappedField.names().isEmpty()) {
mappedField.names().forEach(name -> map.put(name, col));
} else {
map.put(col.name(), col);
}
// createStructNameMap - After
boolean caseInsensitive = config.schemaCaseInsensitive();
if (mappedField != null && !mappedField.names().isEmpty()) {
mappedField
.names()
.forEach(name -> map.put(caseInsensitive ? name.toLowerCase() : name,
col));
} else {
map.put(caseInsensitive ? col.name().toLowerCase() : col.name(), col);
}
```
## Testing
Added comprehensive test cases in `TestRecordConverter.java`:
1. **`testNameMappingWithDifferentCaseAlias`** - Tests field lookup with
alias in different case (parameterized for case-sensitive/insensitive)
2. **`testNameMappingWithNestedStruct`** - Tests nested struct field lookup
with case sensitivity (parameterized)
3. **`testLookupStructFieldWithNameMappingAndCaseSensitivity`** - Explicit
test for `lookupStructField` behavior with name mapping and both case
sensitivity modes
## Checklist
- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Unit tests added
- [x] Code follows project style guidelines (spotlessCheck passes)
- [x] All existing tests pass
## Related Issue
Closes: #15392
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
