daniellavoie opened a new issue #6524: URL: https://github.com/apache/incubator-pinot/issues/6524
# Summary Operating Pinot on behalf of data engineers represents a challenge when these users only have access to the Controller endpoints. SRE aren't specifically aware of tables configurations details as created by the system users. If something is wrong, there is no other means for data engineers to get the details of the error than requesting log extraction from SREs. # Observed use cases ## Realtime table ingestion Any Kafka connectivity issue coming from a bad server, port or credential will not be reported back in the table status. ## BatchConfig Recently on `0-7-0.SNAPSHOT`, a new awesome `batchIngestionConfig` allows minion tasks to ingest data. For misc reason, that ingestion may fail because of user provided configuration (invalid S3, or GCS credentials, etc). Sadly, the only way for a data engineer to get feedback from the root cause not seing anything being ingested is ask an SRE to investigate the Pinot logs. # Suggestion Include a table ingestion state as part of the table status. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
