0

As I know, 'num.partitions' in kafka server.properties will be worked for all topics. Now I want to set partitionNumber=1 for topicA and partitionNumber=2 for topicB. Is that possible to implementation with high level api?

fcbflying
  • 693
  • 1
  • 7
  • 23
  • So you're trying to create the topics through the API rather than through the command line scripts as shown in the quickstart? http://kafka.apache.org/documentation.html#quickstart – Morgan Kenyon Nov 20 '15 at 13:46
  • yes.I want to set the partitions for topic with api. Any idea for that? – fcbflying Nov 21 '15 at 13:10

2 Answers2

2

num.partitions is a value used when a topic is generated automatically. If you generate a topic yourself, you can set any number of partitions as you want.

You can generate a topic yourself with the following command. (replication factor 3 and the number of partitions 2. Capital words are what you have to replace.)

bin/kafka-topics.sh --create --zookeeper ZOOKEEPER_HOSTNAME:ZOOKEEPER_PORT \
--replication-factor 3 --partitions 2 --topic TOPIC_NAME
Heejin
  • 4,463
  • 3
  • 26
  • 30
0

There a configuration value that can be set on a Kafka Broker.

auto.create.topics.enable=true

True is actually the default setting,

Enable auto creation of topic on the server. If this is set to true then attempts to produce data or fetch metadata for a non-existent topic will automatically create it with the default replication factor and number of partitions.

So if you read or write from a non-existent partition as if it existed, if will automatically create one for you. I've never heard of using the high level api to automatically create one.

Looking over the Kafka Protocol Documentation, there doesn't seem to be a provided way to create topics.

Morgan Kenyon
  • 3,072
  • 1
  • 26
  • 38
  • Thanks.How about the low level api? Create topic with partitions manually? – fcbflying Nov 23 '15 at 01:40
  • So both the high level and low level api implement the Protocol Documentation that I listed above. So there doesn't seem a way to create a topic through either of the APIs. You've would either have to explicitly create it with a command like Heejin's answer, or setup your default configuration and enable auto topic creation like I said in my answer. – Morgan Kenyon Nov 23 '15 at 13:52
  • Thanks. That means if I want to create two topics with different partitions.I have to use command line script. – fcbflying Nov 24 '15 at 01:13