xiangfu0 commented on issue #17193:
URL: https://github.com/apache/pinot/issues/17193#issuecomment-3539554085
+1 for this feature.
Here is my high level thoughts:
1. create `BrokerQueryEventListener` interface for query lifecycle:
```
/** Invoked when the broker receives the query request */
void onQuerySubmitted(QueryContext queryContext, long requestTimestampMs);
/** Invoked when the broker receives responses from servers */
void onQueryResponseReceived(QueryContext queryContext,
BrokerResponse brokerResponse,
long responseTimestampMs);
/** Invoked when the full query lifecycle completes successfully */
void onQueryCompleted(QueryEventRecord eventRecord);
/** Invoked when an error occurs during query execution */
void onQueryFailed(QueryEventRecord eventRecord, Throwable error);
```
2. QueryEventRecord(mainly coming from query planner and response), this
need to be reviewed/reworked carefully for compliance:
```
{
"version": 1,
"fields": {
"timestamp": "long",
"queryId": "string",
"tableName": "string",
"tenant": "string",
"query": "string",
"brokerId": "string",
"clientIp": "string",
"totalTimeMs": "int",
"brokerTimeMs": "int",
"serverTimeMs": "int",
"numServersQueried": "int",
"numServersResponded": "int",
"segmentsScanned": "long",
"docsScanned": "long",
"entriesScannedInFilter": "long",
"entriesScannedPostFilter": "long",
"exceptions": {
"type": "array",
"items": "string"
},
"multistage": {
"type": "map",
"values": "string"
},
"metadata": {
"type": "map",
"values": "string"
}
}
}
````
3. write your implementation for the query logs persistency like Parquet
file on deep store etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
