ashokkumarrathore commented on code in PR #462:
URL: https://github.com/apache/incubator-livy/pull/462#discussion_r1928059767
##########
server/src/main/scala/org/apache/livy/utils/SparkKubernetesApp.scala:
##########
@@ -138,6 +138,7 @@ object SparkKubernetesApp extends Logging {
private var sessionLeakageCheckInterval: Long = _
var kubernetesClient: DefaultKubernetesClient = _
+ var appNamespaces: mutable.Set[String] = mutable.Set("default")
Review Comment:
Thank you for inputs @jahstreet .
1. If user does not specify a namespace, the job goes to default namespace.
That's why we start with that. But we can remove it too. As it will anyway get
populated after first job submission. For thread safety, i will switch it to
java.util.concurrent.ConcurrentHashMap.
2. The configurable namespace allowlist would work but that would need
deployment everytime a new namespace is allowed. The rolebindings can be
created in new namespaces(to allow job submission from livy service account)
independently.
3. The reason we are maintaining this list of namespaces is for leaked apps.
The idea is, if we go through all namespaces where jobs were submitted, that
should be enough.
4. Livy monitoring all namespaces is not feasible due to security. In a
multi tenant cluster, due to security, livy service account might not have any
permissions on some namespaces.
5. For recovery, we are also saving the namespace now in
[server/src/main/scala/org/apache/livy/server/batch/BatchSession.scala] which
will be used to recover.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
