Skip to content

Commit

Permalink
Support mechanism to provide external zookeeper-server list to build …
Browse files Browse the repository at this point in the history
…global/configuration zookeeper (#269)

* Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper

* Add external zk example

* add external zk list into values.yaml

Fixes #268

### Motivation
Right now, [chart dynamically](https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/templates/zookeeper-statefulset.yaml#L140) creates zk cluster with zk pods initialized in the same namespace. However, for global/configuration zookeeper, user requires to build zk clusters with pods deployed in different namespaces. Therefore, user needs a mechanism to pass an external list of zk-servers to the chart and build zk-cluster with pods across different namespaces.

### Modification
- Chart should be considering zk-value's configuration for external zookeeper and generate zk-configuration file with appropriate zk-server list and unique id of that zookeeper.

This PR sets `ZOOKEEPER_SERVERS` value provided by user and also sets override-value flag which will be used by [generate-zookeeper-config.sh](https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/generate-zookeeper-config.sh) to override external zk list in config file and assign appropriate id to the host.

apache/pulsar#15987 fixes [generate-zookeeper-config.sh](https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/generate-zookeeper-config.sh) changes.


### Result
- User can add `ZOOKEEPER_SERVERS` string into `zookeeper.configData` in [Values.yaml](https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/values.yaml#L385) file to override external zk-server list.
  • Loading branch information
rdhabalia committed Oct 18, 2022
1 parent 1bcf255 commit 89f28bc
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 4 deletions.
17 changes: 13 additions & 4 deletions charts/pulsar/templates/zookeeper-statefulset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -138,10 +138,19 @@ spec:
containerPort: {{ .Values.zookeeper.ports.clientTls }}
{{- end }}
env:
- name: ZOOKEEPER_SERVERS
value:
{{- $global := . }}
{{ range $i, $e := until (.Values.zookeeper.replicaCount | int) }}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{ $global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
- name: ZOOKEEPER_SERVERS
{{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
value: {{ .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
{{- else }}
{{- $global := . }}
value: {{ range $i, $e := until (.Values.zookeeper.replicaCount | int) }}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{ $global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
{{- end }}
- name: EXTERNAL_PROVIDED_SERVERS
{{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
value: "true"
{{- else }}
value: "false"
{{- end }}
envFrom:
- configMapRef:
name: "{{ template "pulsar.fullname" . }}-{{ .Values.zookeeper.component }}"
Expand Down
2 changes: 2 additions & 0 deletions charts/pulsar/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -397,6 +397,8 @@ zookeeper:
## templates/zookeeper-configmap.yaml
##
configData:
# External zookeeper server list in case of global-zk list to create zk cluster across zk deployed on different clusters/namespaces
# ZOOKEEPER_SERVERS: "us-east1-pulsar-zookeeper-0.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-1.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-2.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-0.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-1.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-2.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888"
PULSAR_MEM: >
-Xms64m -Xmx128m
PULSAR_GC: >
Expand Down
3 changes: 3 additions & 0 deletions examples/values-zookeeper-aws.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ monitoring:
node_exporter: false

zookeeper:
configData:
# External zookeeper server list in case of global-zk list to create zk cluster across zk deployed on different clusters/namespaces
# ZOOKEEPER_SERVERS: "us-east1-pulsar-zookeeper-0.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-1.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-2.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-0.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-1.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-2.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888"
volumes:
# use a persistent volume or emptyDir
persistence: true
Expand Down

0 comments on commit 89f28bc

Please sign in to comment.