Shahzeb-DataEngineer opened a new pull request, #4142: URL: https://github.com/apache/iceberg-python/pull/4142
Closes #4141 # Rationale for this change `timestamp_to_micros` and `timestamptz_to_micros` rejected valid ISO-8601 strings with 7-9 fractional second digits because `ISO_TIMESTAMP` / `ISO_TIMESTAMPTZ` capped the fraction at 6 digits, and `ISO_TIMESTAMPTZ` did not accept zone offsets with seconds. These converters are used when binding string literals to `timestamp`/`timestamptz` columns and when parsing JSON single values (partition values and initial defaults), so timestamps written by other engines with nanosecond precision failed to load or filter. The Java implementation accepts 0-9 fractional digits (`appendFraction(NANO_OF_SECOND, 0, 9, true)`) and truncates to the target precision. This change mirrors that behavior: - Widen `ISO_TIMESTAMP` and `ISO_TIMESTAMPTZ` to accept 1-9 fractional digits, and allow optional seconds in the zone offset (also in `ISO_TIMESTAMPTZ_NANO`). - Parse micros timestamps through the existing nanosecond path and truncate to microseconds with `nanos_to_micros`, so the sub-microsecond handling stays in one place. Nanosecond types keep full precision. ## Are these changes tested? Yes. Added parametrized cases for 7, 8, and 9 fractional digits for both `timestamp_to_micros` and `timestamptz_to_micros` in `tests/utils/test_datetime.py`, including the offset-with-seconds form and the existing specific error messages, plus a JSON single-value deserialization test with 9-digit fractions in `tests/test_conversions.py`. Ran `tests/utils/test_datetime.py`, `tests/test_conversions.py`, and `tests/expressions/test_literals.py` locally; all pass (283 + 172 tests). ## Are there any user-facing changes? Yes: strings that were previously rejected with `ValueError` are now accepted and truncated to microsecond precision, matching the Java implementation. No previously valid input changes meaning. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
