Monday, March 13, 2017

Apache Kafka Installation on Windows

  1. Make sure JDK and JRE are installed and path is set in environment variable
  2. Download Apache Kafka from https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.2.0/kafka_2.10-0.10.2.0.tgz
  3. Unpack kafka_2.10-0.10.2.0.tgz
  4. Copy extracted kafka_2.10-0.8.2.1 directory to C:\
  5. Open C:\kafka_2.10-0.10.2.0\config\server.properties and update existing log.dir entry as mentioned below :
    dirs=c:/kafka_2.10-0.10.2.0/kafka-logs
  6. Open C:\kafka_2.10-0.10.2.0\config\zookeeper.properties and update existing dataDir entry as mentioned below :
    dataDir=c:/kafka_2.10-0.10.2.0/zookeeper-data(Note : Make sure you use forward slash as depicted for step-5 and step-6)
  7. Installation of Single Node Single Broker Kafka is done on Windows
  8. Starting Zookeeper and Kafka server
    Open a command prompt and start Zookeeper server using following command :
    cd c:\kafka_2.10-0.10.2.0
    bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
  9. Now open another command prompt and start Kafka server using following command
    .\bin\windows\kafka-server-start.bat .\config\server.properties
    I hope both Zookeeper and Kafka server are executed successfully for you.
  10. Now let's create Kafka Topic using following command
    cd C:\kafka_2.10-0.10.2.0
    bin\windows\kafka-topics.bat ––create ––zookeeper localhost:2181 ––replication-factor 1 ––partition 1 ––topic test
  11. List the topics using following command
    cd C:\kafka_2.10-0.10.2.0
    bin\windows\kafka-topics.bat ––list  ––zookeeper localhost:2181
  12. Send message using Kafka console producer
    cd C:\kafka_2.10-0.10.2.0
    bin\windows\kafka-console-producer.bat ––broker-list localhost:9092 ––topic test
  13. Consume the above message using Kafka console consumer
    cd C:\kafka_2.10-0.10.2.0
    bin\windows\kafka-console-consumer.bat ––zookeeper localhost:2181 ––topic test ––from-beginning

No comments:

Post a Comment

Apache Spark – Catalyst Optimizer

Optimizer is the one that automatically finds out the most efficient plan to execute data operations specified in the user’s program. In...