gortiz commented on PR #12243:
URL: https://github.com/apache/pinot/pull/12243#issuecomment-1943432997
Could we use a different configuration model?
We have historically abuse of `properties` fields. They are easy to
implement (just map them to `Map<String, String>` with Jackson!), but they are
a very poor interface. They are very difficult to document and verify and they
limit the values we can use as strings.
One of the nicer things in `index-spi` is that we created a way to escape
from the `properties` field limitation. It would be nice to continue going that
way in new features like this one.
My recommendation is, instead of:
```json
"enrichmentConfigs": [
{
"enricherClassName":
"org.apache.pinot.segment.local.recordenricher.clp.CLPEncodingEnricher",
"properties": {
"fieldsForClpEncoding": "logLine" <--- original record field
}
},
{
"enricherClassName":
"org.apache.pinot.segment.local.recordenricher.function.CustomFunctionEnricher",
"properties": {
"newColumnToAddIntoRecord": "<groovy / regular transform
function>" <--- original record field
}
}
],
```
Configure enrichers as:
```json
"enrichmentConfigs": [
{
"enricher": "CLP", // using a id instead of a class is easier for
users and not so difficult for us
"config": {
"fields": ["logLine"] // I'm assuming several fields may be
enriched
}
},
{
"enricher": "generatedColumn", // this is the name used in Postgres
and MySQL for the same concept.
"config": {
"column": "columnName",
"language": "groovy",
"expression": "<groovy / regular transform function>"
}
}
],
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
