npawar commented on pull request #5642: URL: https://github.com/apache/incubator-pinot/pull/5642#issuecomment-657733441
> > > @mayankshriv @siddharthteotia Please verify that LinkedIn use cases are not using the preserved SQL keyword as the column name > > > > > > I believe some reserved keywords are being used. You will want to do this > > ``` > > A queryParam validateFieldNames has been provided to the /add /update and /validate schema calls. It is by default true. For operations on schemas with invalid field names to be successful, this flag needs to be set to false. > > ``` > > IIUC, this will require use cases at Li to change their application code and pass this param? I have seen use cases having reserved keywords (like date, count etc) as field names. Calcite allows us to escape and that was the original plan to work around this problem when migrating to SQL. Yes, calls to add schemas with any of these keywords or update/validate existing schemas with keywords, will have to be changed to pass the param. This might not mean that every application has to change their code. It could be handled in any of the blocks between the controller and the application (wrapper, onboarding platform). We could look into making this a cluster level config - do you think that would be easier? It is true that Calcite allows escaping. However we're seeing more and more users on the community use these keywords as columns and then spend time debugging/wondering why some queries are failing. Plus, with so many external integrations like SuperSet, ThirdEye, it is not easy to control the escaping of the keywords. Another alternative is to parse the query on broker, identify keywords, and escape them, before feeding to calcite parser. But then, we'd be introducing expensive regex, and it will be cumbersome to get completely exhaustive. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
