sajjad-moradi commented on a change in pull request #6546:
URL: https://github.com/apache/incubator-pinot/pull/6546#discussion_r577268997
##########
File path:
pinot-tools/src/main/java/org/apache/pinot/tools/data/generator/StringGenerator.java
##########
@@ -56,11 +65,18 @@ public void init() {
@Override
public Object next() {
+ if (numberOfValuesPerEntry == 1) {
+ return getNextString();
+ }
+ return
MultiValueGeneratorHelper.generateMultiValueEntries(numberOfValuesPerEntry,
rand, this::getNextString);
+ }
+
+ private String getNextString() {
return vals.get(rand.nextInt(cardinality));
Review comment:
I'm not sure why we need to generate millions of records. I assume
something below 500K records should suffice and my assumption is that we won't
get into trouble with that number of rows.
BTW I have tested it on my mac with 10M rows with high cardinality on
different columns in order of millions and no issue observed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
