Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]

[via GitHub]

xinyiZzz commented on code in PR #38215:
URL: https://github.com/apache/doris/pull/38215#discussion_r1688012582


##########
be/src/vec/data_types/serde/data_type_datetimev2_serde.cpp:
##########
@@ -112,13 +114,23 @@ void DataTypeDateTimeV2SerDe::write_column_to_arrow(const 
IColumn& column, const
                     binary_cast<UInt64, 
DateV2Value<DateTimeV2ValueType>>(col_data[i]);
             datetime_val.unix_timestamp(&timestamp, ctz);
 
-            if (scale > 3) {
+            if (scale > 0) {
                 uint32_t microsecond = datetime_val.microsecond();
-                timestamp = (timestamp * 1000000) + microsecond;
-            } else if (scale > 0) {
-                uint32_t millisecond = datetime_val.microsecond() / 1000;
-                timestamp = (timestamp * 1000) + millisecond;
+
+                if (scale > 3) {
+                    timestamp = (timestamp * 1000000) + microsecond;
+                } else {
+                    uint32_t millisecond = microsecond / 1000;
+                    timestamp = (timestamp * 1000) + millisecond;
+                }
+            } else {
+                timestamp *= 1000000;
             }
+            timestamp -= std::chrono::duration_cast<std::chrono::seconds>(

Review Comment:
   
   ```
   if (scale > 0) {
       uint32_t microsecond = datetime_val.microsecond();
   
       if (scale > 3) {
           timestamp = (timestamp * 1000000) + microsecond;
       } else {
           uint32_t millisecond = microsecond / 1000;
           timestamp = (timestamp * 1000) + millisecond;
       }
   } else {
       timestamp *= 1000000;
   }
   timestamp -= std::chrono::duration_cast<std::chrono::seconds>(
                        cctz::convert(cctz::civil_second(1970, 1, 1, 0, 0, 0), 
ctz)
                                .time_since_epoch())
                        .count() *
                1000000;
   ```
   
   in above code, no matter how much the scale is, the timestamp is millisecond 
precision, maybe change to:
   
   ```
   int64_t time_offset = std::chrono::duration_cast<std::chrono::seconds>(
                       cctz::convert(cctz::civil_second(1970, 1, 1, 0, 0, 0), 
ctz)
                               .time_since_epoch())
                       .count();
   if (scale > 3) {
       uint32_t microsecond = datetime_val.microsecond();
       timestamp = (timestamp - time_offset) * 1000000 + microsecond;
   } else if (scale > 0) {
       uint32_t millisecond = datetime_val.microsecond() / 1000;
       timestamp = (timestamp - time_offset) * 1000 + millisecond;
   } else {
       timestamp -= time_offset;
   }
   
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org