timsants commented on issue #8500: URL: https://github.com/apache/pinot/issues/8500#issuecomment-1095670691
When a realtime table is being created, the table config is first uploaded to the ZK property store. After that, we try to create the realtime consumer. But as pointed out, if there is an invalid config for the consumer, an exception is thrown. The next time you try to create the table (with correct configs), it checks the ZK property store for the existence of the table config and throws an exception saying that the table already exists (although the table creation process did not finish). **Why does the table not show up with GET `/tables`?** It checks for the tables existence in the ideal state [resources](https://github.com/apache/pinot/blob/master/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java#L604). Because the realtime table creation process never completed, the realtime table was not added to the ZK ideal state. **Why did deleting the table not fix the issue?** Table deletion should have actually fixed the issue because it deletes the config from the property store (verified that this was the case). The return message from the operation was misleading because it only returns a success message if the table was originally in the ideal state. **Proposed solution** The table existence check for table creation is inconsistent with the other table operations (GET and DELETE). For table creation, we should check the table existence in the ideal state instead of checking for the table config in the property store. If the table config already existed in the property store, it should be ok to overwrite since it was never fully created. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
