cbalci commented on issue #12424: URL: https://github.com/apache/pinot/issues/12424#issuecomment-1946957794
> regarding "Archival format": > is archiving a one way thing or should there also be a "restore" option which then needs to be able to deal with the different formats (and all these conversion needs to be lossless)? Good question. Of course any restore functionality needs to recognize the archival format. We can add one for standard Pinot format, but other ones (say Parquet) can be left as an exercise to the user. In the end there are multitudes of ways to read parquet and generate Pinot segments. > is there an intersection between "archive" and the "mass export" #12315 I'm not entirely clear on the proposal there but I suppose there is at least one common theme: offloading unnecessary data from Pinot servers. In this case (archival), there is no aggregation or filtering other than retention. I imagine if you archive in a suitable format such as parquet, you can make older segments accessible through another query engine such as Trino. You may even find a way to merge data from archived and non-archived segments, again, using Trino. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
