Rajshekhar Muchandi created LIVY-996:
----------------------------------------
Summary: Configure interval check of session garbage
Key: LIVY-996
URL: https://issues.apache.org/jira/browse/LIVY-996
Project: Livy
Issue Type: Improvement
Components: Server
Reporter: Rajshekhar Muchandi
Assignee: Rajshekhar Muchandi
We observed multiple deletion of the same session because a deletion of a
session could take more than a minute for various reasons: session not existing
anymore (especially true when restarting), session time to stop takes time,
kube api slowness, timeouts etc...
In case a session takes more than a minute to stop, the collectGarbage()
function, which is launched every minute, will relaunch the deletion of the
session.
Moreover, each deletion is calling a future, hence a new thread pops up, in a
cluster with 100s or 1000s retained, that could end up to high number of
threads. And if these threads are limited and queued for operations in a thread
pool, for sure these operations could take more than a minute.
To mitigate this effect and avoid warnings and relaunch of session's deletion
multiple times, that would be nice to be able to configure this interval check
session finished in state store instead of having it forced at 1 minute.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
