lhotari commented on code in PR #672: URL: https://github.com/apache/pulsar-helm-chart/pull/672#discussion_r3110344829
########## charts/pulsar/templates/jwt-secret-init.yaml: ########## @@ -0,0 +1,252 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +# When JWT authentication is enabled and generateSecrets is true, this pre-install/pre-upgrade +# hook job generates JWT signing keys and per-subject tokens, storing them as Kubernetes secrets. +# The job is skipped entirely (not rendered) if the signing key secret already exists. +# Individual token secrets that already exist are also left untouched. +{{- if and .Values.auth.authentication.enabled .Values.auth.authentication.jwt.enabled .Values.auth.authentication.jwt.generateSecrets.enabled }} +{{- $ns := include "pulsar.namespace" . }} +{{- $keySecret := ternary "symmetric" "asymmetric" .Values.auth.authentication.jwt.usingSecretKey }} +{{- if not (lookup "v1" "Secret" $ns (printf "%s-token-%s-key" .Release.Name $keySecret)) }} Review Comment: `lookup` returns empty during `helm template` and `helm install --dry-run`, so rendering them with the signing key already present will still emit the Job. user who runs `helm template | kubectl apply` or any GitOps tool that doesn't have cluster context will rerun the generator and may overwrite their token files in `emptyDir` even when they don't want to. The current `lookup`-only guard isn't sufficient as the sole idempotency mechanism — at minimum, document that the feature requires actual `helm install/upgrade` against a live cluster, and consider letting `create_secret` succeed on conflict for the signing key too (or fail loud) instead of relying on render-time lookup. ########## charts/pulsar/templates/jwt-secret-init.yaml: ########## @@ -0,0 +1,252 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +# When JWT authentication is enabled and generateSecrets is true, this pre-install/pre-upgrade +# hook job generates JWT signing keys and per-subject tokens, storing them as Kubernetes secrets. +# The job is skipped entirely (not rendered) if the signing key secret already exists. +# Individual token secrets that already exist are also left untouched. +{{- if and .Values.auth.authentication.enabled .Values.auth.authentication.jwt.enabled .Values.auth.authentication.jwt.generateSecrets.enabled }} +{{- $ns := include "pulsar.namespace" . }} +{{- $keySecret := ternary "symmetric" "asymmetric" .Values.auth.authentication.jwt.usingSecretKey }} +{{- if not (lookup "v1" "Secret" $ns (printf "%s-token-%s-key" .Release.Name $keySecret)) }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + namespace: {{ $ns }} + labels: + {{- include "pulsar.standardLabels" . | nindent 4 }} + component: jwt-secret-init + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-10" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + namespace: {{ $ns }} + labels: + {{- include "pulsar.standardLabels" . | nindent 4 }} + component: jwt-secret-init + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-10" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "create", "patch"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + namespace: {{ $ns }} + labels: + {{- include "pulsar.standardLabels" . | nindent 4 }} + component: jwt-secret-init + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-10" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed +subjects: + - kind: ServiceAccount + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" +roleRef: + kind: Role + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + namespace: {{ $ns }} + labels: + {{- include "pulsar.standardLabels" . | nindent 4 }} + component: jwt-secret-init + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "0" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded +spec: + backoffLimit: 1 + template: + metadata: + labels: + {{- include "pulsar.template.labels" . | nindent 8 }} + component: jwt-secret-init + spec: + {{- include "pulsar.imagePullSecrets" . | nindent 6 }} + serviceAccountName: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + restartPolicy: Never + {{- if .Values.auth.authentication.jwt.generateSecrets.nodeSelector }} + nodeSelector: +{{ toYaml .Values.auth.authentication.jwt.generateSecrets.nodeSelector | indent 8 }} + {{- end }} + {{- if .Values.auth.authentication.jwt.generateSecrets.tolerations }} + tolerations: +{{ toYaml .Values.auth.authentication.jwt.generateSecrets.tolerations | indent 8 }} + {{- end }} + affinity: + {{- if and .Values.affinity.anti_affinity .Values.auth.authentication.jwt.generateSecrets.affinity.anti_affinity }} + podAntiAffinity: + {{- if eq .Values.auth.authentication.jwt.generateSecrets.affinity.type "requiredDuringSchedulingIgnoredDuringExecution" }} + {{ .Values.auth.authentication.jwt.generateSecrets.affinity.type }}: + - labelSelector: + matchExpressions: + - key: "app" + operator: In + values: + - "{{ template "pulsar.name" . }}" + - key: "release" + operator: In + values: + - {{ .Release.Name }} + - key: "component" + operator: In + values: + - jwt-secret-init + topologyKey: {{ .Values.auth.authentication.jwt.generateSecrets.affinity.anti_affinity_topology_key }} + {{- else }} + {{ .Values.auth.authentication.jwt.generateSecrets.affinity.type }}: + - weight: 100 + podAffinityTerm: + labelSelector: + matchExpressions: + - key: "app" + operator: In + values: + - "{{ template "pulsar.name" . }}" + - key: "release" + operator: In + values: + - {{ .Release.Name }} + - key: "component" + operator: In + values: + - jwt-secret-init + topologyKey: {{ .Values.auth.authentication.jwt.generateSecrets.affinity.anti_affinity_topology_key }} + {{- end }} + {{- end }} + volumes: + - name: jwt-secrets + emptyDir: + sizeLimit: 1Mi + initContainers: + - name: generate-jwt-artifacts + image: "{{ template "pulsar.imageFullName" (dict "image" .Values.images.broker "root" .) }}" + imagePullPolicy: {{ .Values.images.broker.pullPolicy | default .Values.defaultPullPolicy }} + command: + - /bin/bash + - -c + - | + set -e + {{- if .Values.auth.authentication.jwt.usingSecretKey }} + echo "Generating symmetric secret key..." + bin/pulsar tokens create-secret-key --output /jwt-secrets/secret.key + {{- range $role, $subject := .Values.auth.superUsers }} + {{- if $subject }} + echo "Generating token for subject '{{ $subject }}'..." + bin/pulsar tokens create \ + --secret-key /jwt-secrets/secret.key \ + --subject "{{ $subject }}" | tr -d '\n' > "/jwt-secrets/{{ $subject }}.token" + {{- end }} + {{- end }} + {{- else }} + echo "Generating RSA key pair..." + bin/pulsar tokens create-key-pair \ + -a RS256 \ + --output-private-key=/jwt-secrets/private.key \ + --output-public-key=/jwt-secrets/public.key + {{- range $role, $subject := .Values.auth.superUsers }} + {{- if $subject }} + echo "Generating token for subject '{{ $subject }}'..." + bin/pulsar tokens create \ + -a RS256 \ + --private-key=/jwt-secrets/private.key \ + --subject "{{ $subject }}" | tr -d '\n' > "/jwt-secrets/{{ $subject }}.token" + {{- end }} + {{- end }} + {{- end }} + echo "JWT artifact generation complete" + volumeMounts: + - mountPath: /jwt-secrets + name: jwt-secrets + containers: + - name: create-secrets + image: "{{ template "pulsar.imageFullName" (dict "image" .Values.images.kubectl "root" .) }}" + env: + - name: NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + command: + - /bin/sh + - -c + - | + set -e + + create_secret() { + SECRET_NAME="$1" + shift + if output=$(kubectl create secret generic "$SECRET_NAME" -n "$NAMESPACE" "$@" 2>&1); then + echo "Created secret $SECRET_NAME" + return 0 + elif echo "$output" | grep -q "already exists"; then + echo "Secret $SECRET_NAME already exists, skipping" + return 1 Review Comment: together with `set -e`, this will currently cause the script to exit. Is that intentional? ########## charts/pulsar/templates/jwt-secret-init.yaml: ########## @@ -0,0 +1,252 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +# When JWT authentication is enabled and generateSecrets is true, this pre-install/pre-upgrade +# hook job generates JWT signing keys and per-subject tokens, storing them as Kubernetes secrets. +# The job is skipped entirely (not rendered) if the signing key secret already exists. +# Individual token secrets that already exist are also left untouched. +{{- if and .Values.auth.authentication.enabled .Values.auth.authentication.jwt.enabled .Values.auth.authentication.jwt.generateSecrets.enabled }} +{{- $ns := include "pulsar.namespace" . }} +{{- $keySecret := ternary "symmetric" "asymmetric" .Values.auth.authentication.jwt.usingSecretKey }} +{{- if not (lookup "v1" "Secret" $ns (printf "%s-token-%s-key" .Release.Name $keySecret)) }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + namespace: {{ $ns }} + labels: + {{- include "pulsar.standardLabels" . | nindent 4 }} + component: jwt-secret-init + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-10" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + namespace: {{ $ns }} + labels: + {{- include "pulsar.standardLabels" . | nindent 4 }} + component: jwt-secret-init + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-10" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["get", "create", "patch"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + namespace: {{ $ns }} + labels: + {{- include "pulsar.standardLabels" . | nindent 4 }} + component: jwt-secret-init + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-10" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded,hook-failed +subjects: + - kind: ServiceAccount + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" +roleRef: + kind: Role + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: "{{ template "pulsar.fullname" . }}-jwt-secret-init" + namespace: {{ $ns }} + labels: + {{- include "pulsar.standardLabels" . | nindent 4 }} + component: jwt-secret-init + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "0" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded Review Comment: Job `hook-delete-policy` is `before-hook-creation,hook-succeeded` for the Job but `before-hook-creation,hook-succeeded,hook-failed` for the SA/Role/RoleBinding. Asymmetric: if the Job fails, the SA/Role/RoleBinding are deleted but the failed Job pod sticks around for debugging — and the next attempt will recreate the SA but the Job remains, which on `before-hook-creation` will be replaced. It works, but it's worth either keeping `hook-failed` off the SA/Role/RoleBinding too (so the failed Job has the RBAC it needs for inspection) or adding it to the Job (so retries are clean). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
