Skip to main content

Connect Using kaf CLI

kaf is a modern CLI for Apache Kafka. You can connect to your Upstash Kafka cluster using kaf.

info

If you do not have a Kafka cluster and/or topic already, follow these steps to create one.

In the cluster details section of the Upstash Console copy bootstrap endpoint, username and password. Then replace following parameters in the code snippets below with the actual values you copied earlier.

  • $BOOTSTRAP_ENDPOINT
  • $UPSTASH_KAFKA_USERNAME
  • $UPSTASH_KAFKA_PASSWORD
  • $GROUP_ID
  • $TOPIC_NAME

Initially we should add cluster configuration to kaf's config file, which should be located in ~/.kaf/config. Open config file if it exists or create an empty one and insert following config:

clusters:
- name: $CLUSTER_NAME
brokers:
- $BOOTSTRAP_ENDPOINT
SASL:
mechanism: SCRAM-SHA-512
username: $UPSTASH_KAFKA_USERNAME
password: $UPSTASH_KAFKA_PASSWORD
security-protocol: SASL_SSL
note

$CLUSTER_NAME is a logical name, which is used to identify different Kafka cluster. You can use your Upstash cluster name.

To select the cluster configuration to use, run:

kaf config use-cluster $CLUSTER_NAME  

At this point you should be able to connect to your Kafka cluster using kaf.

List Brokers and Topics:

kaf nodes
kaf topics

Produce a message:

echo "Hello Upstash!" | kaf produce $TOPIC_NAME

Fetch messages:

kaf consume $TOPIC_NAME

Consume messages using consumer groups:

kaf consume $TOPIC_NAME -g $GROUP_ID --offset oldest

For more information see kaf repository.