Jackie-Jiang commented on issue #8294:
URL: https://github.com/apache/pinot/issues/8294#issuecomment-1059589884
`kafka` uses `murmur2` partition function by default, which also converts
value to string first before hashing it.
It is possible to use the `hashCode` function for individual value type, but
it might also cause inconsistency if value is ingested as a different type
(e.g. `INT` is ingested as `DOUBLE`). To get consistent result, currently we
always use string literal to calculate the partition.
On the kafka side, the partition function should match the one within pinot:
```
public int getPartition(Object value) {
return abs(value.toString().hashCode()) % _numPartitions;
}
private static int abs(int n) {
return (n == Integer.MIN_VALUE) ? 0 : Math.abs(n);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
