Set up Kafka¶ For Kafka inbound endpoint¶ To configure the Kafka inbound endpoint, copy the following orbit bundles to the <MI_HOME>/dropins directory. kafka-avro-serializer kafka-schema-serializer kafka-schema-registry-client Copy the following additional client libraries to the <MI_HOME>/lib directory. kafka-clients-3.7.0.jar avro-1.11.3.jar Note If you are also using the Kafka connector, copy the necessary dependencies as outlined in the following section. For connector version 3.2.0 and later¶ To use the Kafka connector, download and install Apache Kafka. Before you start configuring Kafka you also need the integration runtime and we refer to that location as <PRODUCT_HOME>. Note The Kafka connector has been tested with the Kafka versions listed below. To access all downloadable versions of Kafka, see the Kafka documentation. The recommended version of Java is 11. Kafka version 2.8.2¶ To configure the Kafka connector with Kafka version 2.8.2, copy the following client libraries from the <KAFKA_HOME>/lib directory to the <MI_HOME>/lib directory. kafka_2.12-2.8.2.jar kafka-clients-2.8.2.jar metrics-core-2.2.0.jar scala-library-2.12.13jar zkclient-0.10.jar zookeeper-3.5.9.jar Copy the following additional client libraries to the <MI_HOME>/lib directory (can be copied from the Confluent platform): avro-1.11.3.jar common-config-5.4.0.jar common-utils-5.4.0.jar kafka-avro-serializer-5.3.0.jar kafka-schema-registry-client-5.3.0.jar Kafka version 3.7.0¶ To configure the Kafka connector with Kafka version 3.7.0, copy the following orbit bundles to the <MI_HOME>/dropins directory. kafka-avro-serializer kafka-schema-serializer kafka-schema-registry-client Copy the following client libraries from the <KAFKA_HOME>/lib directory to the <MI_HOME>/lib directory. kafka_2.13-3.7.0.jar kafka-clients-3.7.0.jar metrics-core-2.2.0.jar scala-library-2.13.2jar zkclient-0.10.jar zookeeper-3.8.3.jar Copy the following additional client libraries to the <MI_HOME>/lib directory (can be copied from the Confluent platform): avro-1.11.3.jar common-config-7.6.0.jar common-utils-7.6.0.jar Navigate to <KAFKA_HOME> and run the following command to start the ZooKeeper server: bin/zookeeper-server-start.sh config/zookeeper.properties From the <KAFKA_HOME> directory, run the following command to start the Kafka server: bin/kafka-server-start.sh config/server.properties Now that you have connected to Kafka, you can start publishing and consuming messages using the Kafka brokers. For more information, see Publishing Messages using Kafka and Consuming Messages using Kafka. For connector version 3.1.x and below¶ To use the Kafka connector, download and install Apache Kafka. Before you start configuring the Kafka you also need the integration runtime and we refer to that location as <PRODUCT_HOME>. Note The recommended version of Kafka is Kafka_2.11-2.2.1. To access all downloadable versions of Kafka, see the Kafka documentation. The recommended version of Java is 1.8. To configure the Kafka connector, copy the following client libraries from the <KAFKA_HOME>/lib directory to the <MI_HOME>/lib directory. kafka_2.11-2.2.1.jar kafka-clients-1.0.0.jar metrics-core-2.2.0.jar scala-library-2.12.3.jar zkclient-0.10.jar zookeeper-3.4.10.jar Copy the following additional client libraries to the <MI_HOME>/lib directory (can be copied from the Confluent platform): avro-1.8.1.jar common-config-5.4.0.jar common-utils-5.4.0.jar kafka-avro-serializer-5.3.0.jar kafka-schema-registry-client-5.3.0.jar Navigate to <KAFKA_HOME> and run the following command to start the ZooKeeper server: bin/zookeeper-server-start.sh config/zookeeper.properties From the <KAFKA_HOME> directory, run the following command to start the Kafka server: bin/kafka-server-start.sh config/server.properties Now that you have connected to Kafka, you can start publishing and consuming messages using the Kafka brokers. For more information, see Publishing Messages using Kafka and Consuming Messages using Kafka.