Kafka常用脚本命令

    xiaoxiao2025-02-14  32

    启动Kafka

    #前台启动 bin/kafka-server-start.sh config/server.properties #后台启动 nohup bin/kafka-server-start.sh config/server.properties &

    创建一个topic

    bin/kafka-topics.sh \ --create \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --replication-factor 3 \ --partitions 3 \ --topic pig

    查看所有的topic

    bin/kafka-topics.sh --list \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka

     修改topic partition 分区

    bin/kafka-topics.sh --alter \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --partitions 4 --topic pig

    查看topic详细信息

    bin/kafka-topics.sh --describe \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --topic pig

    启动 console consumers,并从头开始消费

    bin/kafka-console-consumer.sh \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --topic pig --from-beginning

    启动 console consumers,并从当前位置开始消费

    bin/kafka-console-consumer.sh \ --zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka \ --topic pig --from-beginning

    启用 console producer

    bin/kafka-console-producer.sh \ --broker-list hadoop001:9092,hadoop002:9092,hadoop003:9092 \ --topic pig

    注意:

              1、创建的topic 副本数不能大于kafka集群中存活的broker数,否则或抛出如下错误:

                    如: replication factor: 3 larger than available brokers: 2

              2、--zookeeper参数后面填写的参数一定要与连接的kafka-serve配置中的zookeeper.connect一致

              如:  [   -zookeeper hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka

                           zookeeper.connect=hadoop001:2181,hadoop002:2181,hadoop003:2181/kafka    ]

              3、当修改一个topic 分区时,分区只能增加,不能较小

    最新回复(0)