BohanZhang0222 opened a new issue, #7363: URL: https://github.com/apache/kyuubi/issues/7363
### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [x] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug I am performing a concurrency stress test on Kyuubi and encountered intermittent 404 errors when fetching operation results. The issue appears to occur when calling get result row set for an operation that has already transitioned to CLOSED_STATE, even though the client did not explicitly close the operation. **Test Setup** Goal Validate the maximum concurrent Operation capacity supported by 4 Kyuubi nodes under the same environment configuration. Load Testing Method Using Locust to simulate client behavior with the following workflow: Create operation Poll operation status Fetch operation result Repeat continuously Test Parameters Concurrency: 60 Duration: 10 minutes continuous execution **Observed Behavior** Intermittent failures when fetching results: `Error getting result row set for operation handle <operation_id> ` Example error: `org.apache.kyuubi.KyuubiSQLException: Invalid OperationHandle [a0f5cbc3-08a4-4a54-8e38-e944174fa940] ` This corresponds to a 404-like scenario where the operation handle is no longer valid. ### Affects Version(s) 1.9.3 ### Kyuubi Server Log Output ```logtalk From Kyuubi server logs, the lifecycle of a failed operation is as follows: 2026-03-20 17:36:33.524 INFO OperationLog - Creating operation log file ... 2026-03-20 17:42:53.342 INFO ExecuteStatement - PENDING_STATE -> RUNNING_STATE 2026-03-20 17:42:54.572 INFO ExecuteStatement - RUNNING_STATE -> FINISHED_STATE 2026-03-20 17:44:42.814 INFO ExecuteStatement - FINISHED_STATE -> CLOSED_STATE 2026-03-20 17:44:42.814 ERROR ApiUtils - Error getting result row set for operation handle a0f5cbc3-08a4-4a54-8e38-e944174fa940 org.apache.kyuubi.KyuubiSQLException: Invalid OperationHandle [a0f5cbc3-08a4-4a54-8e38-e944174fa940] at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) at org.apache.kyuubi.operation.OperationManager.getOperation(OperationManager.scala:106) at org.apache.kyuubi.server.BackendServiceMetric.$anonfun$fetchResults$1(BackendServiceMetric.scala:211) at org.apache.kyuubi.metrics.MetricsSystem$.timerTracing(MetricsSystem.scala:112) at org.apache.kyuubi.server.BackendServiceMetric.fetchResults(BackendServiceMetric.scala:186) at org.apache.kyuubi.server.BackendServiceMetric.fetchResults$(BackendServiceMetric.scala:181) at org.apache.kyuubi.server.KyuubiServer$$anon$1.fetchResults(KyuubiServer.scala:179) at org.apache.kyuubi.server.api.v1.OperationsResource.getNextRowSet(OperationsResource.scala:186) ``` ### Kyuubi Engine Log Output ```logtalk ``` ### Kyuubi Server Configurations ```yaml ``` ### Kyuubi Engine Configurations ```yaml ``` ### Additional context Not all operations are affected。 <img width="2730" height="414" alt="Image" src="https://github.com/user-attachments/assets/bd168fff-0775-47a4-a015-9c7f3fe72771"; /> <img width="2746" height="1176" alt="Image" src="https://github.com/user-attachments/assets/eefe565b-0a44-45ac-8b2d-640396ba0db4"; /> <img width="1358" height="664" alt="Image" src="https://github.com/user-attachments/assets/b5284d6b-ee94-40d5-bdc3-4cb699c0d89c"; /> <img width="1364" height="522" alt="Image" src="https://github.com/user-attachments/assets/39d0f8fd-08d6-4af5-92db-8439218dfbab"; /> <img width="2664" height="512" alt="Image" src="https://github.com/user-attachments/assets/b2029896-1744-472d-8dea-b25721b444c4"; /> ### Are you willing to submit PR? - [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix. - [ ] No. I cannot submit a PR at this time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
