A paper published by Databricks primarily focuses on why "Kafka sucks" and what if we make the source of truth a database and watch abstractions, then cases we miss with Kafka on resiliency and consistency can be solved.
It argues that modern pub-sub systems:
fail to achieve their goal of truly decoupling writes and readers
violate the end-to-end principle
expose an ad-hoc storage abstraction with a bespoke API and limited power
It argues that modern pub-sub systems:
fail to achieve their goal of truly decoupling writes and readers
violate the end-to-end principle
expose an ad-hoc storage abstraction with a bespoke API and limited power
relevant thread on X -https://x.com/BdKozlovski/status/1991892238335381524