npawar commented on a change in pull request #7237:
URL: https://github.com/apache/pinot/pull/7237#discussion_r682927034
##########
File path:
pinot-plugins/pinot-input-format/pinot-csv/src/main/java/org/apache/pinot/plugin/inputformat/csv/CSVRecordReader.java
##########
@@ -85,6 +85,10 @@ public void init(File dataFile, @Nullable Set<String>
fieldsToRead, @Nullable Re
if (csvHeader == null) {
format = format.withHeader();
} else {
+ //we assume that the record delimiter also delimits the header
+ if (delimiterNotPresentInHeader(delimiter, csvHeader)) {
Review comment:
an edge case that we should prolly handle - what if the record contains
a single value? This is quite possible especially if users have a long string
that they want to split up into multiple fields using ingestion transforms or
use json index on it.
##########
File path:
pinot-plugins/pinot-input-format/pinot-csv/src/test/java/org/apache/pinot/plugin/inputformat/csv/CSVRecordReaderTest.java
##########
@@ -103,4 +104,17 @@ protected void checkValue(RecordReader recordReader,
List<Map<String, Object>> e
}
Assert.assertFalse(recordReader.hasNext());
}
+
+ @Test
+ public void
whenConfiguredHeaderDoesNotContainConfiguredDelimiterThenExceptionShouldBeThrown()
{
Review comment:
please use a short test name (testIncorrectHeaderDelimiter ?) and add
the javadoc to this method if you want to have this description somewhere
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
