14

Below the steps I did to get this issue :

  1. Launch ZooKeeper
  2. Launch Kafka : .\bin\windows\kafka-server-start.bat .\config\server.properties

Also note that my OS is Windows

And at the second step the error happens :

ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.common.InconsistentClusterIdException: The Cluster ID Reu8ClK3TTywPiNLIQIm1w doesn't match stored clusterId Some(BaPSk1bCSsKFxQQ4717R6Q) in meta.properties. The broker is trying to join the wrong cluster. Configured zookeeper.connect may be wrong. at kafka.server.KafkaServer.startup(KafkaServer.scala:220) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44) at kafka.Kafka$.main(Kafka.scala:84) at kafka.Kafka.main(Kafka.scala)

When I trigger .\bin\windows\kafka-server-start.bat .\config\server.properties zookeeper console returns :

INFO [SyncThread:0:FileTxnLog@216] - Creating new log file: log.1

How to fix this issue to get kafka running ?

EDIT See this answer following those steps solved my problem

1
15

I solved this issue on my machine (Windows 10) by deleting the meta.properties file found in the log directory.

The issue itself occured after I had stopped both the zookeeper and server and then restarted them.

4
  • the logs are in /tmp directory
    – BJ5
    May 23 '20 at 18:03
  • Good for you @johnmin The file I removed still was in the logs directory.
    – Jakob
    May 25 '20 at 6:39
  • Yes, it will get generated again, but Kafka will start without issues
    – BJ5
    May 27 '20 at 14:29
  • @johnmin ok. I never claimed it wouldn't. Obviously the old file was the problem and the newly generated one worked fine. The removed file was in the logs directory.
    – Jakob
    Jun 24 '20 at 9:20
6

Usually, this problem arises when kafka logs are stored in the persistent folder and zookeeper data in the temporary, or vice-versa. Then, after system restart, files that are stored in the temporary directory get cleaned and regenerated leading to the configuration mismatch.

To prevent this situation to happen again revise log.dirs kafka parameter in the server.properties and dataDir zookeeper parameter in zookeeper.properties and ensure that both point to the same type of directory (e.g. temporary or persistent). Only after that reset inconsistent data (if any) using for example method from Jacob's answer, i.e. remove meta.properties file, and restart zookeeper and kafka.

1
  • hehe, second time I end up here. Thanks mate! Any advantages in choosing tmp over persistent?
    – DarkCygnus
    Oct 1 at 23:46
3

I encountered the same issue while running Kafka server on my Windows Machine. You can try the following to resolve this issue:

  1. Open server server.properties file which is located in your kafka folder kafka_2.11-2.4.0\config (considering your version of kafka, folder name could be kafka_<kafka_version>)
  2. Search for entry log.dirs
  3. If your log.dir path contains windows directory path like this E:\Shyam\Software\kafka_2.11-2.4.0\kafka-logs which has a single backslash i.e , change it to double back-slash i.e with \
0
1

Try the following...

  1. Enable following line in ./config/server.properties

    listeners=PLAINTEXT://:9092

  2. Modify default ZooKeeper dataDir

  3. Modify default Kafka log dir

0
1

On windows version 20H2 and OS build 19042.662 escaping the log.dirs path in <kafka_dir>\config directory worked for me.

it was previously

log.dirs=C:\*********\kafka\logs

changed to

log.dirs=C:\\*********\\kafka\\logs

Might need to restart zookeeper first and then kafka after the change.

1

No need to delete the log/data files on Kafka. Check the Kafka error logs and find the new cluster id. Update the meta.properties file with cluster-ID then restart the Kafka.

/home/kafka/logs/meta.properties

To resolve this issue permanently follow below.

Check your zookeeper.properties file and look for dataDirpath and change the path tmp location to any other location which should not be removed after server restart.

/home/kafka/kafka/config/zookeeper.properties

Copy the zookeeper folder and file to the new(below or non tmp) location then restart the zookeeper and Kafka.

cp -r /tmp/zookeeper /home/kafka/zookeeper

Now server restart won’t affect the Kafka startup.

0

If you do not mind losing your data than you can remove your kafka "data" folder and restart kafka. If you used the default location the folder named "data" is probably in /tmp/kafka-logs.

2
  • Thank you for your reply, the path of my log file in server.properties is log.dirs=C:\\Users\\Dorian\\kafka_2.12-2.4.0\\kafka-logs is it wrong ?
    – Dorian
    Jan 6 '20 at 10:30
  • I also have no data folder into kafka_2.12-2.4.0 so how should i set up the data folder path ?
    – Dorian
    Jan 6 '20 at 10:46
0

Try to delete kafka-logs from your tmp folder, which is located in C:/tmp folder by default on windows, and can be set in [your-kafka-folder]/config/server.properties file as log.dirs

0

Delete /tmp/kafka-logs/meta.properties which saved the wrong cluster.id of last/failed session:

#
broker.id=0
version=0
cluster.id=kzRHvVl-Rb2DZaHSqlTfow     <===

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.