navina commented on issue #12217: URL: https://github.com/apache/pinot/issues/12217#issuecomment-1879104512
> For context, we have some customers internally whose Kafka topics have string and int columns for indicating the deleted column field. Changing topic schema isn't trivial. This requirement makes sense as changing topic schema is not trivial and ingestion transforms can help circumvent these issues. > If the column happens to be some other type like string or int, then the usual type coercion rules can be applied. @ankitsultana How does the type coercion rule work when in the input string is unexpected? I haven't looked into pinot's mechanism of handling type coercion for its config values. Is it actually being used during config parsing? > that's not the original intend of this deleteRecordColumn right @navina ? IIUC this is not suppose to be modified by user during ingestion The intention of adding the check was to make sure that the users are not misconfiguring it. This can be system-managed in case of CDC connectors or user-managed in other stream connectors. It is a user-facing config, iirc. @ankitsultana / @tibrewalpratik17 If you can add more validation in the TableConfigUtils for this config, instead of strict type check, I am fine with that. Blindly removing the check is a bit much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
