1、准备
1.1、官网下载Kafka安装包
kafka_2.12-2.2.2.tgz
1.2、安装JDK，并配置环境变量
1.3、安装Zookeeper（详参照）
https://www.ibkun.cn/admin#/posts/edit?postId=132
2、环境搭建
2.1 解压Kafka安装包
tar -zvxf kafka_2.12-2.2.2.tgz
2.2 修改配置文件server.properties 点我下载
vim config/server.properties
主要调整内容如下：
节点1
broker.id=0
节点2
broker.id=1
节点3
broker.id=2
调整数据存储目录：
log.dirs=/app/kafka_2.12-2.2.2/data
listener IP 端口调整
advertised.listeners=PLAINTEXT://ip1:9092
调整示例如下：
############################# Server Basics #############################
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0
############################# Socket Server Settings #############################
listeners=PLAINTEXT://:9092
# The number of threads that the server uses for receiving requests from the network and sending responses to the network
num.network.threads=3
# The number of threads that the server uses for processing requests, which may include disk I/O
num.io.threads=8
# The send buffer (SO_SNDBUF) used by the socket server
socket.send.buffer.bytes=102400
# The receive buffer (SO_RCVBUF) used by the socket server
socket.receive.buffer.bytes=102400
# The maximum size of a request that the socket server will accept (protection against OOM)
socket.request.max.bytes=104857600
############################# Log Basics #############################
# A comma separated list of directories under which to store log files
log.dirs=/app/kafka_2.12-2.2.2/data
num.partitions=1
num.recovery.threads.per.data.dir=1
############################# Internal Topic Settings #############################
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
############################# Log Retention Policy #############################
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
############################# Zookeeper #############################
# Zookeeper connection string (see zookeeper docs for details).
zookeeper.connect=hostname1:2181,hostname2:2181,hostname3:2181
# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000
############################# Group Coordinator Settings #############################
group.initial.rebalance.delay.ms=0
2.3 复制到其他机器
scp -r ./apache-zookeeper-3.8.0-bin/ username@IP2:/app
scp -r ./apache-zookeeper-3.8.0-bin/ username@IP3:/app
Copy完成后，参照 “2.2 调整节点配置信息”
2.5 启动Kafka
nohup /app/kafka_2.12-2.2.2/bin/kafka-server-start.sh /app/kafka_2.12-2.2.2/config/server.properties > /dev/null 2>&1 &
2.5 检查Kafka启动状态
/app/kafka_2.12-2.2.2/bin/kafka-topics.sh --create --zookeeper hostname1:2181,hostname2:2181,hostname3:2181 --replication-factor 3 --partitions 6 --topic kfk_test
/app/kafka_2.12-2.2.2/bin/kafka-topics.sh --list --zookeeper hostname1:2181,hostname2:2181,hostname3:2181
/app/kafka_2.12-2.2.2/bin/kafka-console-producer.sh -broker-list hostname1:9092,hostname2:9092,hostname3:9092 --topic kfk_test
/app/kafka_2.12-2.2.2/bin/kafka-topics.sh --describe --zookeeper hostname1:2181,hostname2:2181,hostname3:2181 --topic kfk_test
/app/kafka_2.12-2.2.2/bin/kafka-topics.sh --delete --zookeeper hostname1:2181,hostname2:2181,hostname3:2181 --topic kfk_test
/export/deploy/kafka_2.12-2.2.2/bin/kafka-topics.sh --describe --zookeeper hostname1:2181,hostname2:2181,hostname3:2181 --topic kfk_test
/export/deploy/kafka_2.12-2.2.2/bin/kafka-topics.sh --list --zookeeper hostname1:2181,hostname2:2181,hostname3:2181
评论区