-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Schemas get deleted after compaction #1075
Comments
Are there still log files on the Kafka brokers for that topic? |
I have same issue. |
Hi sorry i had lost the logs. But i am trying to reproduce. I see _schemas logs and index delete message. Is it anything to do with log compact? Will not see messages on kafka topic when compact happened? |
Any updates on this. We have experienced the same issue. |
Any updates on this. We have experienced the same issue. It seems that our schemas are deleted from the _schemas topic after a period of time |
Any update? I have experienced this issue too. |
It baffles me that the is no response to such a serious issue.. |
Any updates on this? I think I experienced a similar issue while running version confluentinc/cp-schema-registry:7.0.1. I upgraded my cluster to larger machines, and everything was running fine for about 200 days, even after the upgrade. However, when I shut off the smaller pods, the schema registry pods relocated to the newer nodes. Somehow, the _schemas topic was the only one that appeared 'clean' after this. This caused my producer to start creating new messages with new IDs, leading to a very cumbersome process to restore the messages back to their original IDs. I had to publish directly on _schemas while all producers, consumers, and the schema registry were scaled to 0, insert the duplicate magic numbers, restart the cluster, and then process the messages. I was fortunate to encounter only one collision where a previous magic ID matched a newer one, and this happened during a maintenance window. I'm running 2 instances of the SR, without anything set besides SCHEMA_REGISTRY_HOST_NAME/SCHEMA_REGISTRY_LISTENERS/SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS so maybe some kind of race condition? maybe related to #3127 but I think that running in the same gruop is the default behavior, as I just one have 2 SR for HA |
Hi
I have a scenario where system is running for long time, but after somedays i run console consumer on _schemas and i see 0 msgs. But when i query subjects from SR I get all the registered schema. I am afraid now that when i restart SR all schemas will vanish as currenlty SR has schemas in cache.
What is the reason schemas got deleted from topic.
Below are the topics settings
Topic:_schemas PartitionCount:1 ReplicationFactor:3 Configs:cleanup.policy=compact Topic: _schemas Partition: 0 Leader: 1001 Replicas: 1001,1002,1003 Isr: 1001,1003,1002
The text was updated successfully, but these errors were encountered: