Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1787544377 @jacobmarble, I updated the table to be more clear about when types can be used and merge this. Thanks! -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue merged PR #8683: URL: https://github.com/apache/iceberg/pull/8683 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1375335785 ## format/spec.md: ## @@ -177,8 +177,10 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. | **`decimal(P,S)`** | Fixed-point dec

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1375314183 ## format/spec.md: ## @@ -177,8 +177,10 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. | **`decimal(P,S)`** | Fixed-point decimal;

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1783700784 > @jacobmarble indeed, thanks! do we expect any type promotions being allowed around the new types being added here? @findepi when we discussed in the last community meeting, th

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1375124724 ## format/spec.md: ## @@ -187,10 +189,11 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. Notes: 1. Decimal scale is fixed a

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1375124724 ## format/spec.md: ## @@ -187,10 +189,11 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. Notes: 1. Decimal scale is fixed a

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

Fokko commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1375028979 ## format/spec.md: ## @@ -187,10 +189,11 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. Notes: 1. Decimal scale is fixed and can

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

findepi commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1783460277 @jacobmarble indeed, thanks! do we expect any type promotions being allowed around the new types being added here? -- This is an automated message from the Apache Git Service. To res

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1783184731 > > Now that we have separate types, we can use the type to carry that information, so that you can promote a `long` to `timestamp_ms` or `timestamp` and we know how to interpret the

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1374794900 ## format/spec.md: ## @@ -187,10 +189,11 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. Notes: 1. Decimal scale is fixed a

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

findepi commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1782599430 > Now that we have separate types, we can use the type to carry that information, so that you can promote a `long` to `timestamp_ms` or `timestamp` and we know how to interpret the value.

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1782035943 @jacobmarble sorry for the delay, I was travelling. I think we just need to call out more clearly that the new types are v3 only and can't be used in v2 without breaking forward compatibil

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1373910153 ## format/spec.md: ## @@ -187,10 +189,11 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. Notes: 1. Decimal scale is fixed and ca

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1777840610 @rdblue @Fokko what concerns remain regarding this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

Fokko commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1362161257 ## format/spec.md: ## @@ -874,6 +878,11 @@ Maps with non-string keys must use an array representation with the `map` logica |**`list`**|`array`|| |**`map`**|`array` of

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1361135225 ## format/spec.md: ## @@ -971,8 +985,10 @@ The 32-bit hash implementation is 32-bit Murmur3 hash, x86 variant, seeded with | **`decimal(P,S)`** | `hashBytes(minBi

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1361006657 ## format/spec.md: ## @@ -874,6 +878,11 @@ Maps with non-string keys must use an array representation with the `map` logica |**`list`**|`array`|| |**`map`**|`arr

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1361004482 ## format/spec.md: ## @@ -948,6 +961,7 @@ Lists must use the [3-level representation](https://github.com/apache/parquet-fo Notes: 1. ORC's [TimestampColumnVec

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1361003504 ## format/spec.md: ## @@ -177,8 +177,10 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. | **`decimal(P,S)`** | Fixed-point dec

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360997998 ## format/spec.md: ## @@ -971,8 +985,10 @@ The 32-bit hash implementation is 32-bit Murmur3 hash, x86 variant, seeded with | **`decimal(P,S)`** | `hashBytes(minBi

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

Fokko commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360945526 ## format/spec.md: ## @@ -874,6 +878,11 @@ Maps with non-string keys must use an array representation with the `map` logica |**`list`**|`array`|| |**`map`**|`array` of

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360938772 ## format/spec.md: ## @@ -874,6 +878,11 @@ Maps with non-string keys must use an array representation with the `map` logica |**`list`**|`array`|| |**`map`**|`arr

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

Fokko commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360934237 ## format/spec.md: ## @@ -874,6 +878,11 @@ Maps with non-string keys must use an array representation with the `map` logica |**`list`**|`array`|| |**`map`**|`array` of

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360930771 ## format/spec.md: ## @@ -874,6 +878,11 @@ Maps with non-string keys must use an array representation with the `map` logica |**`list`**|`array`|| |**`map`**|`arr

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

Fokko commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360913216 ## format/spec.md: ## @@ -874,6 +878,11 @@ Maps with non-string keys must use an array representation with the `map` logica |**`list`**|`array`|| |**`map`**|`array` of

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360873696 ## format/spec.md: ## @@ -177,8 +177,10 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. | **`decimal(P,S)`** | Fixed-point decimal;

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360871361 ## format/spec.md: ## @@ -971,8 +985,10 @@ The 32-bit hash implementation is 32-bit Murmur3 hash, x86 variant, seeded with | **`decimal(P,S)`** | `hashBytes(minBigEndi

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360870420 ## format/spec.md: ## @@ -948,6 +961,7 @@ Lists must use the [3-level representation](https://github.com/apache/parquet-fo Notes: 1. ORC's [TimestampColumnVector](

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360866229 ## format/spec.md: ## @@ -874,6 +878,11 @@ Maps with non-string keys must use an array representation with the `map` logica |**`list`**|`array`|| |**`map`**|`array` o

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360863357 ## format/spec.md: ## @@ -862,10 +864,12 @@ Maps with non-string keys must use an array representation with the `map` logica |**`float`**|`float`|| |**`double`**|`dou

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

rdblue commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1360863357 ## format/spec.md: ## @@ -862,10 +864,12 @@ Maps with non-string keys must use an array representation with the `map` logica |**`float`**|`float`|| |**`double`**|`dou

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1355775890 ## format/spec.md: ## @@ -177,20 +177,24 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. | **`decimal(P,S)`** | Fixed-point de

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

Fokko commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1355723510 ## format/spec.md: ## @@ -908,10 +919,12 @@ Lists must use the [3-level representation](https://github.com/apache/parquet-fo | **`float`**| `float`

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

Fokko commented on code in PR #8683: URL: https://github.com/apache/iceberg/pull/8683#discussion_r1355722230 ## format/spec.md: ## @@ -177,20 +177,24 @@ A **`map`** is a collection of key-value pairs with a key type and a value type. | **`decimal(P,S)`** | Fixed-point decimal;

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1758430601 > @jacobmarble Yes, this looks good to me. Thanks for joining the community sync. Could you avoid reformatting the table? History is especially important in the spec. Done. -

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

Fokko commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1758166192 @jacobmarble Yes, this looks good to me. Thanks for joining the community sync. Could you avoid reformatting the table? History is especially important in the spec. -- This is an automat

Re: [PR] Spec: add nanosecond timestamp types [iceberg]

jacobmarble commented on PR #8683: URL: https://github.com/apache/iceberg/pull/8683#issuecomment-1758143965 In the Iceberg community sync today, it was decided that we will not be adding millisecond timestamps after all. Current status of this PR: - millisecond timestamps removed