kirkrodrigues opened a new issue, #9819: URL: https://github.com/apache/pinot/issues/9819
We want to be able to store JSON log events in Pinot so that they can be queried efficiently and so that we can reduce storage costs. Part of this involves encoding unstructured message fields in the log event using a new log compressor called CLP. The other part is to transform the log event to fit a table's schema (e.g., extracting nested fields and storing them in a column). We think this can be done with a custom `StreamMessageDecoder` and a few UDFs. We've written more about the motivation and proposal [here](https://docs.google.com/document/d/1n5qpZgNHRDWk1Hjbu5kwjwIY8T_H5EScFoPndIt8Fis). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
