Connecting to Kafka Programmatically in Java

1. Clone the starter project from GitHub.

cd ~/
git clone https://github.com/linuxacademy/content-ccdak-kafka-simple-consumer.git

1. Perform a test to make sure the code is able to compile and run.

cd content-ccdak-kafka-simple-consumer/
./gradlew run

The output should contain the message printed by the Main class: Hello, world!.

1. Add the Kafka Client Libraries as a project dependency in build.gradle with:

vi build.gradle

1. In the dependencies { ... } block, add the following line:

implementation 'org.apache.kafka:kafka-clients:2.2.1'

1. Edit the Main class:

vi src/main/java/com/linuxacademy/ccdak/kafkaSimpleConsumer/Main.java

1. Implement a basic consumer that consumes messages from the topic and prints them to the screen:

package com.linuxacademy.ccdak.kafkaSimpleConsumer;

import org.apache.kafka.clients.consumer.*;
import java.util.Properties;
import java.util.Arrays;
import java.time.Duration;

public class Main {

    public static void main(String[] args) {
        Properties props = new Properties();
        props.setProperty("bootstrap.servers", "localhost:9092");
        props.setProperty("group.id", "my-group");
        props.setProperty("enable.auto.commit", "true");
        props.setProperty("auto.commit.interval.ms", "1000");
        props.setProperty("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.setProperty("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(props);
        consumer.subscribe(Arrays.asList("inventory_purchases"));
        while (true) {
            ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
            for (ConsumerRecord<String, String> record : records) {
                System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), record.key(), record.value());
            }
        }
    }

}

1. Run the code:

./gradlew run

The program should print a series of messages from the Kafka topic containing information about item purchases.