siddharthteotia commented on issue #8862: URL: https://github.com/apache/pinot/issues/8862#issuecomment-1301394396
On a related note, do we think we can expose such details via SQL which is typically done by DBMSes as this metadata information is also stored in tables (under SYS user) in many systems. In our case, this information is in per segment flat file but in theory we can try to expose this via SQL and that can be handy for debugging imo. Examples (which I have often done via some CLI script in our production when analyzing issues).... - Find the segment name with largest forward index for a particular column - Find the top 10 segments by size or by forward index size. - Find the segment name with highest cardinality for a particular column. API based is also fine I guess. Is the goal here to expose all this for some debugging ? From the description, it looks like the suggestion is to move information tracked in `index_map` file into `metadata.properties` file. Curious, what is the benefit ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
