Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper #269

Merged
merged 3 commits into from
Oct 18, 2022

Conversation

rdhabalia
Copy link
Contributor

@rdhabalia rdhabalia commented Jun 8, 2022

Fixes #268

Motivation

Right now, chart dynamically creates zk cluster with zk pods initialized in the same namespace. However, for global/configuration zookeeper, user requires to build zk clusters with pods deployed in different namespaces. Therefore, user needs a mechanism to pass an external list of zk-servers to the chart and build zk-cluster with pods across different namespaces.

Modification

  • Chart should be considering zk-value's configuration for external zookeeper and generate zk-configuration file with appropriate zk-server list and unique id of that zookeeper.

This PR sets ZOOKEEPER_SERVERS value provided by user and also sets override-value flag which will be used by generate-zookeeper-config.sh to override external zk list in config file and assign appropriate id to the host.

apache/pulsar#15987 fixes generate-zookeeper-config.sh changes.

Result

  • User can add ZOOKEEPER_SERVERS string into zookeeper.configData in Values.yaml file to override external zk-server list.

{{- $global := . }}
{{ range $i, $e := until (.Values.zookeeper.replicaCount | int) }}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{ $global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
- name: ZOOKEEPER_SERVERS
{{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably add it to value file with an example and comment it out? so user can know how to use this feature.
As from the script change, seems the provided server list need to contain election port and follower port.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rdhabalia - are you able to update the values.yaml file so that this configuration is documented? I plan to release an updated version of the helm chart next week, and it'd be great to include this.

Copy link
Contributor Author

@rdhabalia rdhabalia Jun 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I have added sample-example values in examples/values-zookeeper-aws.yaml and values.yaml

@rdhabalia
Copy link
Contributor Author

ping @MarvinCai @michaeljmarshall

Copy link
Member

@michaeljmarshall michaeljmarshall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for your contribution @rdhabalia

@michaeljmarshall
Copy link
Member

I missed a minor detail with how the feature is configured. I proposed a fix here #308.

@michaeljmarshall michaeljmarshall added this to the 3.0.0 milestone Oct 19, 2022
michaeljmarshall added a commit that referenced this pull request Oct 19, 2022
### Motivation

In #269, we added a way to configure external zookeeper servers. However, it was added to the wrong section of the zookeeper config. The `zookeeper.configData` section is mapped directly into the zookeeper configmap.

### Modifications

Move `zookeeper.configData.ZOOKEEPER_SERVERS` to `zookeeper.externalZookeeperServerList`

### Verifying this change
This is a cosmetic change on an unreleased feature.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper
3 participants