Kafka Commands

Kafka package provides two different set of commands (.bat for windows) and .sh for Mac in the same distribution.

Windows user ? Run from \bin\windows

Need to run the kafka commans from this path.

c:\ngdeveloper\softwares\kafka_2.12-2.5.0\bin\windows

---

Mac user ? Run from \bin

Run the commands from below path

c:\ngdeveloper\softwares\kafka_2.12-2.5.0\bin\

I am listing down only the windows commands below, but if you need to run on Mac, just find the equivalent command yourself in the above mentioned folder.

Command to start Zookeeper

zookeeper-server-start.bat ..\..\config\zookeeper.properties

Not starting ? Make sure you are running it from this path: c:\ngdeveloper\softwares\kafka_2.12-2.5.0\bin\windows

Kafka command to start the kafka Broker:

kafka-server-start.bat ..\..\config\server.properties

Is Kafka broker started successfully? do checkout this and compare your output.

Need to start multiple instances of the kafka server, then do follow this link

Kafka command to create a kafka topic

• ngdev-topic: kafka topic name to be created
• zookeeper: we already started above with 2181 port, here linking the topic with zookeeper.
• replication-factor: 1 here, can be any number, its where distributed streaming comes into picture.
• partitions – Each kafka topic contains n number of partitions, here creating the ngdev-topic with 3 partition. Remember if consumer would like to receive the same order it is sent in the producer side, then all those messages must be handled in the single partition only. If messages are shared across the partitions then consumer can’t receive the messages in the exact order where producer sent it.

kafka-topics.bat --create --topic ngdev-topic --zookeeper localhost:2181 --replication-factor 1 --partitions 3

Is it created try listing down and verify it.

Kafka command to list all the kafka topics ?

kafka-topics.bat --zookeeper localhost:2181 --list

Kafka command to start/create a kafka producer

• kafka producer is created inside the broker, so localhost:9092 is given, this is where kafka broker is running (as above)
• key separator and all basically to retain the same order

kafka-console-producer.bat --broker-list localhost:9092 --topic ngdev-topic --property "key.separator=:" --property "parse.key=true"

Is Kafka producer created successfully ? do verify with this screenshot with yours.

Kafka command to start/create a kafka consumer

Same key separator mentioned here for ordering purpose and then mentioned the bootstrap server as kafka broker 9092 running instance.

kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic ngdev-topic --property "key.separator=:" --property "print.key=true"

Possible exceptions you may face:

---

Exception: Exception in thread “main” kafka.Zookeeper.ZooKeeperClientTimoutException: Timed out waiting for connection while in state: CONNECTING

at kafka.zookeeper.ZookeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:262)

Solution:

Zookeeper is not running in your local machine / server machine. Do start the zookeeper to resolve this exception.

Windows command to start up the zookeeper:

zookeeper-server-start.bat ....\config\zookeeper.properties