siddharthteotia commented on pull request #6710: URL: https://github.com/apache/incubator-pinot/pull/6710#issuecomment-805198697
> With the addition of new data structure in this PR, there are essentially two places in DataTable where the key-value / name-value style structure is located. > > * First is the existing DataTable metadata which is also a series of key-value pairs where key is string and value is some statistic/metric. This is towards the beginning of the byte stream > * Second is the structure introduced in this PR which is written as a footer. > > Since we are anyway bumping up the version, how about we move the existing metadata of key-value pairs to the end of file to keep consistency in the format. So, all the metadata stuff (aka key-value pairs) + new positional stuff can be a file footer. KV pair might be misleading here. Within a KV pair, the value part is indeed a arbitrary serialized object. The KV concept in the footer is just to give it some structure. So, we can keep growing the footer by adding a key to the enum and then the corresponding serialized bytes into the payload -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
