Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KafkaAdmin clusterId configuration is ignored when observability is enabled and bootstrap supplier is not set #3402

Closed
varmenise opened this issue Aug 2, 2024 · 0 comments

Comments

@varmenise
Copy link
Contributor

varmenise commented Aug 2, 2024

Version: org.springframework.kafka:spring-kafka:jar:3.2.0

Describe the bug

If you enable observability and define clutserId inside the KafkaAdmin configuration, unless you specify an observability supplier, which matches the producer bootstrap server, the code will re-create a KafkaAdmin passing the defined properties as well as the producer bootstrap config.

		if (this.kafkaAdmin != null) {
			Object producerServers = this.producerFactory.getConfigurationProperties()
					.get(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG);
			String adminServers = this.kafkaAdmin.getBootstrapServers();
			if (!producerServers.equals(adminServers)) {
				Map<String, Object> props = new HashMap<>(this.kafkaAdmin.getConfigurationProperties());
				props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, producerServers);
				int opTo = this.kafkaAdmin.getOperationTimeout();
				this.kafkaAdmin = new KafkaAdmin(props);
				this.kafkaAdmin.setOperationTimeout(opTo);
			}
		}

See https://github.com/spring-projects/spring-kafka/blob/main/spring-kafka/src/main/java/org/springframework/kafka/core/KafkaTemplate.java#L493

Unfortunately if you have specified a cluster id with

kafkaAdmin.setClusterId(clusterId);

this will not be part of the property map, so the code above will basically override this configuration and clusterId will be null.

To Reproduce

  1. enable observability

  2. create a KafkaAdminConfiguration and specify bootstrap property or no property at all, just a clusterId

    @bean

    public KafkaAdmin kafkaAdmin() {
        Map<String, Object> configs = new HashMap<>();
        configs.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServer);
        KafkaAdmin kafkaAdmin = new KafkaAdmin(configs);
        kafkaAdmin.setClusterId(clusterId);
        //kafkaAdmin.setBootstrapServersSupplier(() -> bootstrapServer);
        return kafkaAdmin;
    }
  1. send an event

Expected behavior

A kafka admin with properties, bootstrap and clusterId is set whereas clusterId is null.

@varmenise varmenise changed the title KafkaAdmin bootstrap configuration is ignored when observability is enabled KafkaAdmin clusterId configuration is ignored when observability is enabled Aug 2, 2024
@varmenise varmenise changed the title KafkaAdmin clusterId configuration is ignored when observability is enabled KafkaAdmin clusterId configuration is ignored when observability is enabled and bootstrap supplier is not set Aug 2, 2024
varmenise added a commit to varmenise/spring-kafka that referenced this issue Aug 2, 2024
… if observability is enabled and bootstrap supplier is not set

fixes spring-projectsGH-3402 (spring-projects#3402)

* Re-set clusterId after creating new KafkaAdmin
varmenise added a commit to varmenise/spring-kafka that referenced this issue Aug 2, 2024
… if observability is enabled and bootstrap supplier is not set

fixes spring-projectsGH-3402 (spring-projects#3402)

* Re-set clusterId after creating new KafkaAdmin
@sobychacko sobychacko added this to the 3.3.0-M2 milestone Aug 2, 2024
sobychacko pushed a commit that referenced this issue Aug 19, 2024
Fixes: #3402

Re-set clusterId after creating new KafkaAdmin to ensure proper
configuration when observability is enabled and bootstrap supplier
is not set.

This addresses the issue where kafkaAdmin clusterId configuration
was being ignored under specific conditions.

 **Auto-cherry-pick to `3.2.x` & `3.1.x`**
sobychacko pushed a commit that referenced this issue Aug 19, 2024
Fixes: #3402

Re-set clusterId after creating new KafkaAdmin to ensure proper
configuration when observability is enabled and bootstrap supplier
is not set.

This addresses the issue where kafkaAdmin clusterId configuration
was being ignored under specific conditions.

 **Auto-cherry-pick to `3.2.x` & `3.1.x`**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment