DBs-ium, CDC and Streaming

An airhacks.fm conversation with Gunnar Morling (@gunnarmorling)
about: The first Debezium commit, Randal Hauch, DBs-iuim, Java
Content Repository JCR / modshape, exploring the Change Data
Capture (CDC), how Debezium started, the MySQL binlog, the logical
decoding in Postgres, Oracle Advanced Queuing, update triggers,
Java Message System (JMS), there is no read detection, switching
the current user at JDBC connection for audit purposes, helping
Debezium with additional metadata table, using Kafka Streams to
join the metadata and the payload, installing the logical decoding
plugins into PostgreSQL, logical decoding plugin exposes the data
from the write ahead log, decoding into protocol buffers with
decoderbufs, in cloud environments like e.g. Amazon RDS you are not
allowed to install any plugins, wal2json is verbose but comes
preinstalled on RDS, pgoutput is responsible for the actual
decoding of the events, debezium only sees committed transactions,
debezium is mainly written in Java, decoderbufs was written by
community and included to debezium, Debezium communicates with
Postgres via the JDBC / Postgres API, pgoutput format is converted
into Kafka Connector source format, Kafka Connect is a framework
for running connectors, Kafka Connect comes with sink and source
connectors, Kafka Connect comes with connector specific connectors
like e.g. StringConverter, Converters are not Serializers, Debezium
ships as Kafka Connect plugin, Kafka Connector runs as standalone
process, running Debezium in embedded mode, JPA cache invalidation
with Debezium, converting Debezium events into CDI events,
converting database changes to WebSockets events, database polling
vs the Debezium approach, DB2 will support Debezium, Oracle support
is "on the horizon", Oracle LogmMiner, Oracle XStream, Debezium
supports Microsoft SQL Server (starting with Enterprise license),
Apache Pulsar comes with Debezium out-of-the-box, Pulsar IO,
running Debezium as standalone service with outbounds APIs, MongoDB
supports the "Debezium Change Event Format", Kafka Sink connectors
are easy to implement, Debezium embedded mode and offsets, embedded
connector has to remember the offset, an offset API is available
for embedded Debezium connectors, combining CDC with Kafka Streams,
Quarkus supports Kafka Streams and Reactive Messaging, Quarkus and
Kafka Streams, Quarkus supports Kafka Streams in dev mode,
replacing Hibernate Envers with Debezium, Messaging vs. Streaming
or JMS vs. Kafka, Kafka is a database, the possible Debezium
features, Cassandra support is coming, Outbox pattern is going to
be better supported, transactional event grouping, dedicated topic
for transaction demarcations, commercial support for Debezium,
Debezium exposes JMX metrics, Five Advantages of Log-Based Change
Data Capture, Reliable Microservices Data Exchange With the Outbox
Pattern, Automating Cache Invalidation With Change Data Capture

Gunnar Morling on twitter: @gunnarmorling and github:
https://github.com/gunnarmorling. Gunnar's blog:
https://morling.dev/.