siddharthteotia commented on a change in pull request #6546:
URL: https://github.com/apache/incubator-pinot/pull/6546#discussion_r572457056
##########
File path:
pinot-tools/src/main/java/org/apache/pinot/tools/data/generator/StringGenerator.java
##########
@@ -56,11 +65,18 @@ public void init() {
@Override
public Object next() {
+ if (numberOfValuesPerEntry == 1) {
+ return getNextString();
+ }
+ return
MultiValueGeneratorHelper.generateMultiValueEntries(numberOfValuesPerEntry,
rand, this::getNextString);
+ }
+
+ private String getNextString() {
return vals.get(rand.nextInt(cardinality));
Review comment:
Vals is a hashmap. This is where we are going to run into issues with
generation based approach where the characteristics file specifies a very high
cardinality for column(s). The HashMap will likely hit OOM. So the other
approach of estimating the memory without the completed segment all together
becomes quite essential.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
