启动Kafka
#前台启动 bin/kafka-server-start.sh config/server.properties #后台启动 nohup bin/kafka-server-start.sh config/server.properties &创建一个topic
bin/kafka-topics.sh \ --create \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --replication-factor 3 \ --partitions 3 \ --topic pig查看所有的topic
bin/kafka-topics.sh --list \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka修改topic partition 分区
bin/kafka-topics.sh --alter \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --partitions 4 --topic pig查看topic详细信息
bin/kafka-topics.sh --describe \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --topic pig启动 console consumers,并从头开始消费
bin/kafka-console-consumer.sh \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --topic pig --from-beginning启动 console consumers,并从当前位置开始消费
bin/kafka-console-consumer.sh \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --topic pig --from-beginning启用 console producer
bin/kafka-console-producer.sh \ --broker-list hadoop001:9092,hadoop002:9092,hadoop003:9092 \ --topic pig1、创建的topic 副本数不能大于kafka集群中存活的broker数,否则或抛出如下错误:
如: replication factor: 3 larger than available brokers: 2
2、--zookeeper参数后面填写的参数一定要与连接的kafka-serve配置中的zookeeper.connect一致
如: [ -zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka
zookeeper.connect=hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka ]
3、当修改一个topic 分区时,分区只能增加,不能较小