xiasongh commented on PR #11313:
URL: https://github.com/apache/iceberg/pull/11313#issuecomment-2427381877
> Thanks for the PR @xiasongh ! I'm interested in your thoughts on using a
single property to define routing. If you agree, I can re-review once you make
that change.
Hi @bryanck, thanks for the review!
I'm a little apprehensive about re-using `iceberg.tables.route-field` since
the semantics would be inconsistent.
`iceberg.tables.route-field` is used to point at a field in the record and
use its value as the target iceberg table, whereas
`iceberg.tables.route-pattern` is used as a format string that results in the
table. I think mixing the two as it is now would be a bit confusing.
For example, if we had `iceberg.tables.route-field = "{topic}"`, does that
mean the target table is the value of the `{topic}` field or is the target
table `{topic}`? Is the rule: if there's a format string, then it's the target
table, otherwise it's a field that contains the target table?
How about we introduce `iceberg.tables.route-pattern` and support adding
fields from the record using the expression syntax? For example,
`iceberg.tables.route-pattern="{field}"` would be equivalent to
`iceberg.tables.route-field="field"`, but something like
`iceberg.tables.route-pattern="db.{_topic}_{field}"` would also be supported.
Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
