davidradl commented on code in PR #27553:
URL: https://github.com/apache/flink/pull/27553#discussion_r2787741902
##########
flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemTableSource.java:
##########
Review Comment:
does this need to be changed as well? If not it might be worth adding tests
for it anyway
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/connector/file/table/FileSystemTableSourceTest.java:
##########
@@ -80,4 +90,35 @@ void testMetadataReading() {
util.verifyRelPlanInsert(
"insert into MySink(a, b, c) select a, b, filemeta from
MyTableWithMeta");
}
+
+ @ParameterizedTest(name = "extractFileName({0}) -> {1}")
+ @MethodSource("fileNameCases")
+ void testFileNameExtraction(String rawPath, String expected) {
+ String extractedFileName = FileSystemTableSource.extractFileName(new
Path(rawPath));
+ assertThat(extractedFileName).isEqualTo(expected);
+ }
+
+ @ParameterizedTest(name = "file.name accessor for {0}")
+ @MethodSource("fileNameCases")
+ void testFileNameMetadataAccessor(String rawPath, String expected) {
+ FileSourceSplit split =
+ new FileSourceSplit("test-split", new Path(rawPath), 0L, 1L,
0L, 1L);
+
+ Object actual =
+
FileSystemTableSource.ReadableFileInfo.FILENAME.getAccessor().getValue(split);
+
+ assertThat(actual).isEqualTo(StringData.fromString(expected));
+ }
+
+ static Stream<Arguments> fileNameCases() {
+ return Stream.of(
+
Arguments.of("file:/D:/AI-Book/FlinkApplication/data/input/user.csv",
"user.csv"),
+ Arguments.of("file:/D:/tmp/input/test.csv", "test.csv"),
+ Arguments.of("file:/C:/Users/me/Desktop/thing.txt",
"thing.txt"),
+ Arguments.of("file:///tmp/input/user.csv", "user.csv"),
+ Arguments.of("file:/tmp/input/dir/", "dir"),
+ Arguments.of("file://localhost/tmp/input/user.csv",
"user.csv"),
+ Arguments.of("s3://bucket/a/b/c.parquet", "c.parquet"),
+ Arguments.of("/tmp/input/dir/file.txt", "file.txt"));
Review Comment:
other tests could include
- tmp/input/dir/file.txt
-../ type paths if they are supported
- filenames with escaped characters in like quotes and dots.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
