Kafka Month-to-month Digest – August 2021 – IBM Developer


That is the forty third version of the Kafka Month-to-month Digest. On this version, I’ll cowl what occurred within the Apache Kafka neighborhood in August 2021.

For final month’s digest, see Kafka Month-to-month Digest: July 2021.

Kafka Summit

The Kafka Summit Americas is on September 14 and 15. It’s a digital occasion with dozens of periods protecting all features of the Kafka ecosystem. You’ll be able to register without spending a dime on the Kafka Summit web site.



Kafka 3.0.0 is at present within the stabilization interval following code freeze. After just a few extra Blocker JIRAs (KAFKA-13198, KAFKA-13219), Konstantine revealed the primary Launch Candidate, 3.0.0 RC0, on August 27. A vote on this RC is at present on going.


On August 12, David Jacot volunteered to run this bugfix launch. The discharge date is at present targetted for September. The launch plan is on the market on the wiki.


Final month, the neighborhood submitted 5 KIPs (KIP-766 to KIP-770), and these are those that caught my eye.

  • KIP-768: Prolong SASL/OAUTHBEARER with Assist for OIDC. Kafka helps OAuth authentication since 2.0 (KIP-255). Nonetheless, KIP-255 solely launched a framework and organizations have to supply their very own implementations. This KIP proposes including a manufacturing prepared implementation of the interfaces outlined in KIP-255 to authenticate with Open ID identification suppliers.

  • KIP-770: Change “buffered.information.per.partition” with “enter.buffer.max.bytes”. This KIP goals at controlling the reminiscence utilized by a Kafka Streams utility. It proposes including a brand new configuration enter.buffer.max.bytes to specify the whole reminiscence that can be utilized to buffer information. The prevailing configuration buffered.information.per.partition which solely managed reminiscence per partition can be deprecated.

On this part, I’ll cowl releases of some neighborhood tasks. This solely consists of tasks which can be open supply.

  • kafka-connect-file-pulse 2.2.0. FilePulse is a connectors to stream any file into Kafka. This new model improves eventualities with a really giant variety of small recordsdata, improves assist for XML recordsdata and introduces a brand new pluggable Partitioner interface to distribute recordsdata throughout duties.

  • kcat 1.7.0. Beforehand known as kafkacat, kcat is a CLI shopper for Kafka. This launch improves multibytes delimiters and provides assist for incremental rebalancing.

  • node-rdkafka 2.11. This new launch of node-rdkafka is now based mostly on librdkafka 1.6.1. It now helps Nodejs v16 and in addition the idempotent and transactional producer.

  • kcctl 1.0.0-EA. kcctl is a command line shopper for Kafka Join. It permits to simply work together (deploy, cease, restart, describe, and many others) with connectors.


Get began with Kafka

IBM Occasion Streams for Cloud is Apache Kafka-as-a-Service for IBM Cloud. Get began with IBM Occasion Streams at this time.


Leave a Reply

Your email address will not be published. Required fields are marked *