Spring Boot with Apache Kafka

A hands-on Spring Boot demo that produces and consumes messages with Apache Kafka using the publish/subscribe model.

This tutorial covers the basics of Kafka and walks through a small Java app with a REST producer, a Kafka consumer, and an endpoint to inspect consumed messages.

Features

• Produce messages to a Kafka topic over HTTP
• Consume messages from the same topic with @KafkaListener
• List messages the consumer has received so far
• JSON serialization for producer payloads (User objects)

Tech stack

Component	Version / detail
Java	8
Spring Boot	2.2.4.RELEASE
Spring Kafka	(managed by Boot)
Kafka broker	localhost:9092
Topic	myTopic
Consumer group	kafka-sandbox
App port	8081

Prerequisites

• JDK 8+
• Maven 3.x
• Apache Kafka (classic Zookeeper mode — download from kafka.apache.org/downloads)

Project structure

spring-boot-with-apache-kafka/
└── spring-boot-kafka/
    ├── pom.xml
    └── src/main/java/com/chat/kafka/
        ├── SpringBootKafkaApplication.java
        ├── controller/KafkaController.java
        ├── producer/ProducerConfiguration.java
        ├── consume/ConsumerConfiguration.java
        ├── consume/MyTopicConsumer.java
        └── model/User.java

1. Install and start Kafka

Extract Kafka to a directory of your choice. Commands below assume you are inside the Kafka bin folder (use the scripts under bin/windows on Windows).

Start Zookeeper

./zookeeper-server-start.sh ../config/zookeeper.properties

Start the Kafka broker

In a second terminal:

./kafka-server-start.sh ../config/server.properties

Create the topic

In a third terminal:

./kafka-topics.sh --create --topic myTopic \
  --zookeeper localhost:2181 \
  --replication-factor 1 \
  --partitions 1

Note: Newer Kafka versions prefer --bootstrap-server localhost:9092 instead of --zookeeper. Use the flag that matches your Kafka install.

2. Run the Spring Boot app

cd spring-boot-kafka
mvn clean install
mvn spring-boot:run

Or import spring-boot-kafka into your IDE and run SpringBootKafkaApplication.

The app listens on http://localhost:8081.

3. Try the APIs

Action	Method	URL
Produce a message	GET	http://localhost:8081/kafka/produce?message=Hello%20Kafka
List consumed msgs	GET	http://localhost:8081/kafka/messages

Suggested flow

1. Open http://localhost:8081/kafka/messages — the list is empty until something is produced.
2. Produce a message:

   http://localhost:8081/kafka/produce?message=Message%20sent%20by%20my%20App!
   

3. Call /kafka/messages again — you should see the consumed payload(s).

You can use a browser or Postman for both requests.

How it works

1. Producer — GET /kafka/produce builds a User (name + message) and sends it to myTopic via KafkaTemplate.
2. Consumer — MyTopicConsumer listens on myTopic (group kafka-sandbox) and stores each message in memory.
3. Read API — GET /kafka/messages returns that in-memory list.

Broker address is configured in code as localhost:9092 (ProducerConfiguration / ConsumerConfiguration).

License

This project is provided as a learning sample. Feel free to fork and adapt it.