minwoo-jung commented on issue #12224:
URL: https://github.com/apache/pinot/issues/12224#issuecomment-1886261632
@somandal
I followed your guide and started an instance and ran some tests.
The expr_min and expr_max functions seem to work for the queries I want.
**However, I noticed a couple of issues.**
# problem 1
### Symptoms of the Issue
**A**
LIMIT is not working properly.
I ran the simple query below and noticed that it returns a lot of data even
though the limit is 10.
```sql
SELECT
EXPR_MAX(hostName, fieldValue, hostName)
FROM systemMetricDouble
limit 10
```
**B**
I also had intermittent query failures.
The logs from the failures are shown below.
<details>
<summary>controller log</summary>
[2024-01-10 15:24:01.938] ERROR [PinotQueryResource] [grizzly-http-server-1]
Caught exception while sending query request
java.io.IOException: Failed : HTTP error code : 500. Root Cause: Unknown
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendPostRaw(PinotQueryResource.java:418)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendRequestRaw(PinotQueryResource.java:456)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendRequestToBroker(PinotQueryResource.java:350)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.getQueryResponse(PinotQueryResource.java:270)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.executeSqlQuery(PinotQueryResource.java:181)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:120)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at jdk.internal.reflect.GeneratedMethodAccessor338.invoke(Unknown
Source) ~[?:?]
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:134)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:177)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:81)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at java.lang.Thread.run(Thread.java:829) [?:?]
[2024-01-10 15:24:01.939] ERROR [PinotQueryResource] [grizzly-http-server-1]
Caught exception in sendQueryRaw
java.io.IOException: Failed : HTTP error code : 500. Root Cause: Unknown
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendPostRaw(PinotQueryResource.java:418)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendRequestRaw(PinotQueryResource.java:456)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendRequestToBroker(PinotQueryResource.java:350)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.getQueryResponse(PinotQueryResource.java:270)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.executeSqlQuery(PinotQueryResource.java:181)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:120)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at jdk.internal.reflect.GeneratedMethodAccessor338.invoke(Unknown
Source) ~[?:?]
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:134)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:177)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:81)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at java.lang.Thread.run(Thread.java:829) [?:?]
[2024-01-10 15:24:01.939] ERROR [PinotQueryResource] [grizzly-http-server-1]
Caught exception while processing post request
java.io.IOException: Failed : HTTP error code : 500. Root Cause: Unknown
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendPostRaw(PinotQueryResource.java:418)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendRequestRaw(PinotQueryResource.java:456)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.sendRequestToBroker(PinotQueryResource.java:350)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.getQueryResponse(PinotQueryResource.java:270)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.executeSqlQuery(PinotQueryResource.java:181)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:120)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at jdk.internal.reflect.GeneratedMethodAccessor338.invoke(Unknown
Source) ~[?:?]
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
~[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:134)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:177)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:81)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at
org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
[pinot-all-1.0.0-jar-with-dependencies.jar:1.0.0-b6bdf6c9686b286a149d2d1aea4a385ee98f3e79]
at java.lang.Thread.run(Thread.java:829) [?:?]
</details>
<details>
<summary>broker log</summary>
[2024-01-10 15:24:01.937] ERROR [PinotClientRequest]
[jersey-server-managed-async-executor-2] Caught exception while processing POST
request
java.lang.NullPointerException: null
</details>
<details>
<summary>server log</summary>
server 0
[2024-01-10 16:19:53.407] INFO [Metadata] [uriStat__5__4__20231016T0259Z]
[Consumer clientId=uriStat_REALTIME-url-stat-5, groupId=null] Cluster ID:
nKzujV0nQYSCxgoKn3xB9A
[2024-01-10 16:19:53.939] INFO [QueryScheduler] [pqr-2] Processed
requestId=385128946000000410,table=systemMetricDouble_OFFLINE,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=16/15/15/-1/0/0/0/0/1,schedulerWaitMs=0,reqDeserMs=0,totalExecMs=116,resSerMs=1,totalTimeMs=117,minConsumingFreshnessMs=-1,broker=Broker_alpha-pinot-broker-0.alpha-pinot-broker-headless.n3r-project-pinot-alpha.svc.cluster.local_8099,numDocsScanned=10085435,scanInFilter=10085435,scanPostFilter=20170870,sched=FCFS,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
[2024-01-10 16:19:53.939] INFO [InstanceRequestHandler]
[nioEventLoopGroup-3-7] Slow query: request handler processing time: 117, send
response latency: 0, total time to handle request: 117
[2024-01-10 16:19:54.009] INFO [QueryScheduler] [pqr-6] Processed
requestId=385128946000000410,table=systemMetricDouble_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=167/164/164/24/0/0/0/0/0,schedulerWaitMs=0,reqDeserMs=0,totalExecMs=184,resSerMs=3,totalTimeMs=187,minConsumingFreshnessMs=1704871184145,broker=Broker_alpha-pinot-broker-0.alpha-pinot-broker-headless.n3r-project-pinot-alpha.svc.cluster.local_8099,numDocsScanned=10289406,scanInFilter=10289406,scanPostFilter=20578812,sched=FCFS,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
[2024-01-10 16:19:54.009] INFO [InstanceRequestHandler]
[nioEventLoopGroup-3-3] Slow query: request handler processing time: 187, send
response latency: 0, total time to handle request: 187
server 1
[2024-01-10 16:19:53.937] INFO [QueryScheduler] [pqr-2] Processed
requestId=385128946000000410,table=systemMetricDouble_OFFLINE,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=16/13/13/-1/0/0/0/0/3,schedulerWaitMs=0,reqDeserMs=0,totalExecMs=112,resSerMs=3,totalTimeMs=115,minConsumingFreshnessMs=-1,broker=Broker_alpha-pinot-broker-0.alpha-pinot-broker-headless.n3r-project-pinot-alpha.svc.cluster.local_8099,numDocsScanned=8712881,scanInFilter=8712881,scanPostFilter=17425762,sched=FCFS,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
[2024-01-10 16:19:53.938] INFO [InstanceRequestHandler]
[nioEventLoopGroup-3-7] Slow query: request handler processing time: 115, send
response latency: 1, total time to handle request: 116
[2024-01-10 16:19:53.962] INFO
[LLRealtimeSegmentDataManager_systemMetricDouble__32__49__20240109T1000Z]
[systemMetricDouble__32__49__20240109T1000Z] Consumed 36 events from
(rate:0.59995/s), currentOffset=1012103969, numRowsConsumedSoFar=46073,
numRowsIndexedSoFar=46073
[2024-01-10 16:19:53.962] INFO
[LLRealtimeSegmentDataManager_systemMetricDouble__15__48__20240109T2231Z]
[systemMetricDouble__15__48__20240109T2231Z] Consumed 12 events from
(rate:0.19999/s), currentOffset=159385186, numRowsConsumedSoFar=6339,
numRowsIndexedSoFar=6339
[2024-01-10 16:19:53.996] INFO [QueryScheduler] [pqr-3] Processed
requestId=385128946000000410,table=systemMetricDouble_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=166/165/165/24/0/0/0/0/0,schedulerWaitMs=0,reqDeserMs=0,totalExecMs=174,resSerMs=0,totalTimeMs=174,minConsumingFreshnessMs=1704871184146,broker=Broker_alpha-pinot-broker-0.alpha-pinot-broker-headless.n3r-project-pinot-alpha.svc.cluster.local_8099,numDocsScanned=10792825,scanInFilter=10792825,scanPostFilter=21585650,sched=FCFS,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
server 2
[2024-01-10 16:19:54.138] INFO [QueryScheduler] [pqr-5] Processed
requestId=385128946000000410,table=systemMetricDouble_OFFLINE,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=17/15/15/-1/0/0/0/0/2,schedulerWaitMs=0,reqDeserMs=0,totalExecMs=313,resSerMs=3,totalTimeMs=316,minConsumingFreshnessMs=-1,broker=Broker_alpha-pinot-broker-0.alpha-pinot-broker-headless.n3r-project-pinot-alpha.svc.cluster.local_8099,numDocsScanned=9508324,scanInFilter=9508324,scanPostFilter=19016648,sched=FCFS,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
[2024-01-10 16:19:54.138] INFO [InstanceRequestHandler]
[nioEventLoopGroup-3-8] Slow query: request handler processing time: 316, send
response latency: 0, total time to handle request: 316
[2024-01-10 16:19:54.146] INFO
[LLRealtimeSegmentDataManager_systemMetricDouble__25__51__20240110T0604Z]
[systemMetricDouble__25__51__20240110T0604Z] Consumed 72 events from
(rate:1.19996/s), currentOffset=159369061, numRowsConsumedSoFar=5441,
numRowsIndexedSoFar=5441
[2024-01-10 16:19:54.147] INFO
[LLRealtimeSegmentDataManager_systemMetricDouble__38__51__20240110T0604Z]
[systemMetricDouble__38__51__20240110T0604Z] Consumed 68 events from
(rate:1.1332955/s), currentOffset=130356752, numRowsConsumedSoFar=5441,
numRowsIndexedSoFar=5441
[2024-01-10 16:19:54.147] INFO
[LLRealtimeSegmentDataManager_systemMetricDouble__55__52__20240110T0500Z]
[systemMetricDouble__55__52__20240110T0500Z] Consumed 48 events from
(rate:0.79998666/s), currentOffset=109973534, numRowsConsumedSoFar=6698,
numRowsIndexedSoFar=6698
[2024-01-10 16:19:54.147] INFO
[LLRealtimeSegmentDataManager_systemMetricDouble__53__52__20240109T1545Z]
[systemMetricDouble__53__52__20240109T1545Z] Consumed 54 events from
(rate:0.899955/s), currentOffset=154929857, numRowsConsumedSoFar=50441,
numRowsIndexedSoFar=50441
[2024-01-10 16:19:54.148] INFO
[LLRealtimeSegmentDataManager_systemMetricDouble__21__51__20240109T1213Z]
[systemMetricDouble__21__51__20240109T1213Z] Consumed 51 events from
(rate:0.8473309/s), currentOffset=38389488, numRowsConsumedSoFar=55030,
numRowsIndexedSoFar=55030
[2024-01-10 16:19:54.164] INFO [QueryScheduler] [pqr-2] Processed
requestId=385128946000000410,table=systemMetricDouble_REALTIME,segments(queried/processed/matched/consumingQueried/consumingProcessed/consumingMatched/invalid/limit/value)=167/167/167/16/0/0/0/0/0,schedulerWaitMs=0,reqDeserMs=0,totalExecMs=341,resSerMs=1,totalTimeMs=342,minConsumingFreshnessMs=1704871184147,broker=Broker_alpha-pinot-broker-0.alpha-pinot-broker-headless.n3r-project-pinot-alpha.svc.cluster.local_8099,numDocsScanned=10981394,scanInFilter=10981394,scanPostFilter=21962788,sched=FCFS,threadCpuTimeNs(total/thread/sysActivity/resSer)=0/0/0/0
</details>
- The server component will appear to have processed the data as normal.
### ETC
- In borker, the exception log is not printing all the stacktrace of the
exception.
- Other simple queries don't throw any exceptions and LIMIT seems to work
fine.
```sql
select * from systemMetricDouble limit 10
```
# problem 2
### Symptoms of the Issue
AS does not work.
When I run the following query, the columns that I specified as AS are not
aliased.
There is no problem with the resulting data.
```
SELECT
MAX(fieldValue) AS maxValue,
EXPR_MAX(hostName, fieldValue, hostName),
MIN(fieldValue) AS minValue,
EXPR_MIN(hostName, fieldValue, hostName),
DATETIMECONVERT(eventTime, '1:MILLISECONDS:EPOCH',
'1:MILLISECONDS:EPOCH', '12100000:MILLISECONDS') AS eventTime
FROM systemMetricDouble
WHERE tenantId = 'pinpoint'
AND hostGroupName = 'AlphaPinpointWeb'
AND metricName = 'mem'
AND fieldName = 'used'
AND eventTime BETWEEN 1704101884000 AND 1704706684000
GROUP BY eventTime
ORDER BY eventTime asc
LIMIT 201
```
Controller UI image
# problem 3
### Symptoms of the Issue
Cannot use aliases specified by AS in group by and order by.
I also checked that the options related to expr_min and expr_max worked fine
before I added them.
sql
```
SELECT
AVG(fieldValue) AS avgValue,
DATETIME_CONVERT(eventTime, '1:MILLISECONDS:TIMESTAMP',
'1:MILLISECONDS:EPOCH', '12100000:MILLISECONDS') as avgTime
FROM systemMetricDouble
WHERE tenantId = 'pinpoint'
AND hostGroupName = 'AlphaPinpointWeb'
AND metricName = 'mem'
AND hostName = 'dev-pinpoint-olap02-ncl'
AND fieldName = 'used'
AND eventTime BETWEEN 1704101884000 AND 1704706684000
GROUP BY avgTime
ORDER BY avgTime asc
LIMIT 201
```
error log
```
Error Code: 150
SQLParsingError:
org.apache.pinot.sql.parsers.SqlCompilationException:
'as(datetimeconvert(eventTime, null, null, null), avgTime)' should appear in
GROUP BY clause.
at
org.apache.pinot.sql.parsers.CalciteSqlParser.validateGroupByClause(CalciteSqlParser.java:220)
at
org.apache.pinot.sql.parsers.CalciteSqlParser.validate(CalciteSqlParser.java:203)
at
org.apache.pinot.sql.parsers.CalciteSqlParser.queryRewrite(CalciteSqlParser.java:552)
at
org.apache.pinot.sql.parsers.CalciteSqlParser.compileSqlNodeToPinotQuery(CalciteSqlParser.java:486)
```
### Workaround
Duplicate use of DATETIME_CONVERT function without using alias for group by
and order by.
sql
```sql
SELECT
AVG(fieldValue) AS avgValue,
DATETIME_CONVERT(eventTime, '1:MILLISECONDS:TIMESTAMP',
'1:MILLISECONDS:EPOCH', '12100000:MILLISECONDS') as avgTime
FROM systemMetricDouble
WHERE tenantId = 'pinpoint'
AND hostGroupName = 'AlphaPinpointWeb'
AND metricName = 'mem'
AND hostName = 'dev-pinpoint-olap02-ncl'
AND fieldName = 'used'
AND eventTime BETWEEN 1704101884000 AND 1704706684000
GROUP BY DATETIME_CONVERT(eventTime, '1:MILLISECONDS:TIMESTAMP',
'1:MILLISECONDS:EPOCH', '12100000:MILLISECONDS')
ORDER BY DATETIME_CONVERT(eventTime, '1:MILLISECONDS:TIMESTAMP',
'1:MILLISECONDS:EPOCH', '12100000:MILLISECONDS') asc
LIMIT 201
```
# ETC
The table schema and definitions can be found in the links below.
- [table definition
1](https://github.com/pinpoint-apm/pinpoint/blob/master/metric-module/metric/src/main/pinot/pinot-double-realtime-table.json),
[table definition
2](https://github.com/pinpoint-apm/pinpoint/blob/master/metric-module/metric/src/main/pinot/pinot-double-offline-table.json)
- [table
schema](https://github.com/pinpoint-apm/pinpoint/blob/master/metric-module/metric/src/main/pinot/pinot-double-schema.json)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
