gortiz opened a new pull request, #11600: URL: https://github.com/apache/pinot/pull/11600
Most index types use a single file extension. But some others do not. Thinking that ForwardIndexType was the only that supports multiple file extensions, which depends on whether the column is sorted, dictionary encoded, etc. We initially added a ColumnMetadata argument to `IndexType.getFileExtension`. But recent bug reports (see https://github.com/apache/pinot/issues/11529 and https://github.com/apache/pinot/issues/11520) prove that we were wrong and Text index does also use different file extensions depending on whether it uses the native or lucene type. This PR changes the code a bit. It replaces `String IndexType.getFileExtension(ColumnMetadata)` with `List<String> IndexType.getFileExtensions(@Nullable ColumnMetadata)`. This means that each index may return several extensions. The code that called this method has been changed accordingly. This mainly affects `FilePerIndexDirectory.getFileFor`, which now depends on `FilePerIndexDirectory.getFilesFor`, which returns a list of them. The older method returns the first file that does exist or any on the list in case there is no file with a valid extension. That is compatible with the older implementation, which my return files that didn't exist. A couple of new test were added. They reproduce the error detected in https://github.com/apache/pinot/issues/11529. These test fail on master and run successfully in this PR. Creating a test that replicates https://github.com/apache/pinot/issues/11520 is more difficult and I propose to merge this without having an automatic test for that case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
