We use analytics and cookies to understand site traffic. Information about your use of our site is shared with Google for that purpose. Learn more.
Apache Kafka Sink
This page shows how to install and configure Apache Kafka Sink.
Prerequisites
Knative Eventing installation.
Installation
-
Install the Kafka controller:
kubectl apply --filename https://github.com/knative-sandbox/eventing-kafka-broker/releases/download/v0.19.0/eventing-kafka-controller.yaml
-
Install the Kafka Sink data plane:
kubectl apply --filename https://github.com/knative-sandbox/eventing-kafka-broker/releases/download/v0.19.0/eventing-kafka-sink.yaml
-
Verify that
kafka-controller
andkafka-sink-receiver
are running:kubectl get deployments.apps -n knative-eventing
Example output:
NAME READY UP-TO-DATE AVAILABLE AGE eventing-controller 1/1 1 1 10s eventing-webhook 1/1 1 1 9s kafka-controller 1/1 1 1 3s kafka-sink-receiver 1/1 1 1 5s
Kafka Sink
A KafkaSink
object looks like this:
apiVersion: eventing.knative.dev/v1alpha1
kind: KafkaSink
metadata:
name: my-kafka-sink
namespace: default
spec:
topic: mytopic
bootstrapServers:
- my-cluster-kafka-bootstrap.kafka:9092
Kafka Producer configurations
A Kafka Producer is the component responsible for sending events to the Apache Kafka cluster. Knative exposes all available Kafka Producer configurations that can be modified to suit your workloads.
You can change these configurations by modifying the config-kafka-sink-data-plane
config map in
the knative-eventing
namespace.
Documentation for the settings available in this config map is available on the Apache Kafka website, in particular, Producer configurations.
Enable debug logging for data plane components
To enable debug logging for data plane components change the logging level to DEBUG
in the kafka-config-logging
config map.
-
Apply the following
kafka-config-logging
config map:apiVersion: v1 kind: ConfigMap metadata: name: kafka-config-logging namespace: knative-eventing data: config.xml: | <configuration> <appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender"> <encoder class="net.logstash.logback.encoder.LogstashEncoder"/> </appender> <root level="DEBUG"> <appender-ref ref="jsonConsoleAppender"/> </root> </configuration>
-
Restart the
kafka-sink-receiver
:kubectl rollout restart deployment -n knative-eventing kafka-sink-receiver
Additional information
- To report bugs or add feature requests, open an issue in the eventing-kafka-broker repository.
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.