Kafka Streams Part 3: Making It Production-Ready

In Part 1 we built a working app. In Part 2 we mastered streams, tables, joins, and windows. Everything so far runs great — on your laptop, with clean data, at 11 AM on a Tuesday.

Production is different. Records arrive malformed. Clocks disagree. Instances crash mid-count. Traffic triples on Black Friday. This final part covers everything between "works on my machine" and "sleeps well at night".

Serdes for Your Own Classes: The JSON Serde

String serdes got us far, but real apps move objects: Order, Payment, User. Kafka stores bytes, so we need to teach it JSON. Here's a compact, reusable serde using Jackson:

import com.fasterxml.jackson.databind.ObjectMapper;

public class JsonSerde<T> implements Serde<T> {

    private final Class<T> type;
    private final ObjectMapper mapper = new ObjectMapper();

    public JsonSerde(Class<T> type) {
        this.type = type;
    }

    private final Serializer<T> serializer = new Serializer<>() {
        @Override
        public byte[] serialize(String topic, T data) {
            try {
                return data == null ? null : mapper.writeValueAsBytes(data);
            } catch (JsonProcessingException e) {
                throw new IllegalArgumentException(
                        "Cannot serialize record for topic " + topic, e);
            }
        }
    };

    private final Deserializer<T> deserializer = new Deserializer<>() {
        @Override
        public T deserialize(String topic, byte[] bytes) {
            try {
                return bytes == null ? null : mapper.readValue(bytes, type);
            } catch (IOException e) {
                throw new IllegalArgumentException(
                        "Cannot deserialize record from topic " + topic, e);
            }
        }
    };

    @Override public Serializer<T> serializer()   { return serializer; }
    @Override public Deserializer<T> deserializer() { return deserializer; }
}

Now use it wherever data crosses a boundary — in, out, or into state:

JsonSerde<Order> orderSerde = new JsonSerde<>(Order.class);

// Reading from a topic
KStream<String, Order> orders = builder.stream(
        "orders",
        Consumed.with(Serdes.String(), orderSerde));

// Writing to a topic
results.to("order-totals",
        Produced.with(Serdes.String(), Serdes.Double()));

// Storing in a state store
.aggregate(
        () -> 0.0,
        (key, order, total) -> total + order.amount(),
        Materialized.<String, Double, KeyValueStore<Bytes, byte[]>>as("order-totals")
                .withValueSerde(Serdes.Double()));

The 3 AM crash: forgetting the state store serde. When an aggregate or count writes to a state store, the default serde serializes the stored value. Your default is String; your aggregate produces a Double or a custom type → ClassCastException at runtime, hours after deploy, only for records that hit that path. If your aggregates store anything beyond simple types, always set Materialized...withValueSerde(...). This is the single most common Kafka Streams production bug.

For serious use, preferConfluent's kafka-json-schema-serde or Avro with a schema registry. Schemas become explicit, producers and consumers evolve safely, and incompatible changes get caught at write time instead of runtime.

Time: Which Clock Wins?

Every record has a timestamp, and which clock it comes from changes your results:

Event time — when the event actually happened (embedded in the data, e.g. order.eventTime). What really happened.
Processing time — when your app handled it. Includes every delay in between: producer buffering, network, retries, a rebalance. What your app saw.

Suppose a phone loses signal at 12:59 and uploads a health reading at 13:07. Event time says it belongs in the 12:00–13:00 window. Processing time puts it in the 13:00–14:00 one. Neither is "wrong" — but for correctness you almost always want event time.

Kafka Streams agrees: it uses event time by default. A record's event time comes from (in order) the producer's timestamp or, when the data knows better, a custom extractor:

public class OrderTimestampExtractor implements TimestampExtractor {
    @Override
    public long extract(ConsumerRecord<Object, Object> record, long partitionTime) {
        if (record.value() instanceof Order order) {
            return order.eventTime().toEpochMilli(); // the time that matters
        }
        return partitionTime; // fall back when the record is unusable
    }
}

props.put(StreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG,
        OrderTimestampExtractor.class);

Late events are part of the deal — that's what a window's grace period is for (from Part 2). A record arriving within the grace period after its window closed still lands in that window; beyond it, it's dropped. No extractor saves you from data that's hours late; for that, accept the loss or route late records to a side topic for batch reprocessing.

Exactly-Once: One Config Line

What happens if your app counts an order, crashes before committing the offset, and restarts? It counts the order again. That's at-least-once: no data lost, but duplicates possible. It's the default, and honestly fine for many workloads (a slightly-too-high page-view count hurts nobody).

For money, inventory, or anything where duplicates cause real damage, flip one switch:

props.put(StreamsConfig.PROCESSING_GUARANTEE_CONFIG,
        StreamsConfig.EXACTLY_ONCE_V2);

Now every step — consume, update state, produce output, commit offsets — happens atomically. A crash mid-flight rolls back everything; after restart, it's as if the interrupted work never happened. Records are counted exactly once, even across failures.

The trade-off is throughput (roughly 10–30% lower, thanks to transaction overhead) and slightly higher latency (results commit every 100 ms by default instead of every 30 s). Know which guarantee each of your apps actually needs — don't pay for exactly-once where a duplicate page-view doesn't matter.

Error Handling: Poison Pills and Dead Letters

Sooner or later, garbage arrives: a truncated JSON, a null where a number should be. One bad record that fails deserialization is called a poison pill, and by default it stops your whole app — forever retrying, stuck on the same record. Not acceptable in production.

Deserialization Errors: Skip or Stop

Implement DeserializationExceptionHandler to choose per record:

public class LogAndSkipHandler implements DeserializationExceptionHandler {

    @Override
    public DeserializationHandlerResponse handle(
            ProcessorContext context,
            ConsumerRecord<byte[], byte[]> record,
            Exception exception) {

        log.error("Skipping bad record: topic={} partition={} offset={}",
                record.topic(), record.partition(), record.offset(), exception);

        return DeserializationHandlerResponse.CONTINUE; // skip it, keep flowing
        // return DeserializationHandlerResponse.FAIL;  // or: die (the default)
    }

    @Override
    public void configure(Map<String, ?> configs) { }
}

props.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG,
        LogAndSkipHandler.class);

Skip with care. CONTINUE keeps your app alive but silently drops data. A good middle ground: count skips in a metric, and alert when the skip rate jumps — a sudden spike usually means an upstream producer changed its format.

Processing Errors: The Dead-Letter Queue

For errors inside your logic (a null field, a downstream hiccup), the standard pattern is a dead-letter queue (DLQ) — a side topic where failed records wait for inspection instead of blocking the main flow:

KStream<String, String> cleaned = sentences.process(() -> new Processor<String, String, String, String>() {

    private ProcessorContext<String, String> context;
    private KafkaProducer<byte[], byte[]> dlqProducer;

    @Override
    public void init(ProcessorContext<String, String> context) {
        this.context = context;
        this.dlqProducer = new KafkaProducer<>(Map.of(
                ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092",
                ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class,
                ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class));
    }

    @Override
    public void process(Record<String, String> record) {
        try {
            context.forward(record);              // happy path: pass it on
        } catch (Exception e) {
            dlqProducer.send(new ProducerRecord<>("sentences-dlq",   // quarantine it
                    record.key() == null ? null : record.key().getBytes(),
                    record.value().getBytes()));
            log.warn("Record sent to DLQ", e);
        }
    }

    @Override
    public void close() { dlqProducer.close(); }
});

Later — an hour, a day — a small job replays sentences-dlq after the bug is fixed. Nothing is lost, nothing blocks.

There's also ProductionExceptionHandler for the rare case that writing output records fails (record too large, authorization). Same shape as the handler above: return CONTINUE (drop and move on) or FAIL (default).

Scaling and Fault Tolerance

Remember from Part 1: tasks = partitions of the input topic. Tasks are the atoms of parallelism — they cannot be split further. You distribute them in two dimensions:

// More threads per instance (uses more cores):
props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, 4);

// Or more instances (uses more machines):
//   just start the app again with the same application.id

With a 6-partition input topic, 6 tasks exist. One instance with 6 threads, or 6 instances with 1 thread each — same total parallelism. Your ceiling is the partition count, decided when you created the topic:

Adding a third instance triggers a rebalance: tasks shuffle until they're spread evenly again. Same thing, in reverse, when an instance dies — its tasks move to survivors, and their state rebuilds by replaying changelog topics (Part 2).

Recovering from a crash means replaying the changelog — which can take a while for big state. Standby replicas fix that: each task gets a warm spare on another instance, continuously kept up to date. Failover then takes seconds instead of minutes. props.put(StreamsConfig.NUM_STANDBY_REPLICAS_CONFIG, 1);

Plan partitions before launch. You can add partitions to a topic later, but existing keys stick to their old partitions (partition = hash(key) / partitionCount), and active state won't automatically reshuffle. For stateful apps, resizing partitions cleanly usually means a careful migration. If there's any chance you'll need 12 tasks, create 12 partitions on day one.

Testing Without a Broker

Kafka Streams has a gift for testability: TopologyTestDriver runs your whole topology in memory, with no Kafka running. No Docker, no flaky integration setup, no excuses for skipping tests:

@Test
void countsWords() {
    Topology topology = WordCountTopology.build();   // extract your builder
                                                     // code into a testable method!
    Properties props = new Properties();
    props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
    props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "nowhere:9092"); // never contacted

    try (TopologyTestDriver driver = new TopologyTestDriver(topology, props)) {

        TestInputTopic<String, String> input = driver.createInputTopic(
                "sentences", new StringSerializer(), new StringSerializer());

        TestOutputTopic<String, Long> output = driver.createOutputTopic(
                "word-counts", new StringDeserializer(), new LongDeserializer());

        // act
        input.pipeValue("kafka streams is awesome kafka");

        // assert
        Map<String, Long> counts = output.readKeyValuesToMap();
        assertThat(counts).containsEntry("kafka", 2L);
    }
}

Notice the test even asserts kafka → 2 from a single sentence — real aggregate logic, verified in milliseconds.

Testing windows is where this shines even brighter: instead of waiting for wall-clock time to pass, you can control record timestamps directly:

// Feed events with explicit timestamps to test window boundaries
input.pipeKeyValueList(List.of(
        new KeyValue<>("user-1", "click"),
        new KeyValue<>("user-1", "click")));
// windowed output topics can also be read key-by-key:
// var windowed = output.readKeyValue(); // Windowed<String> key + Long count

A note on structure: notice WordCountTopology.build() — the topology lives in its own method, separate from main and config. Keep the what (topology) apart from the how (properties, lifecycle), and everything becomes testable.

Using Spring Boot? Spring Kafka wraps all this nicely: @EnableKafkaStreams plus a StreamsBuilderFactoryBean that manages the KafkaStreams lifecycle for you, with config from application.yml. The topology code stays exactly the same as what you learned in this series — it's the lifecycle plumbing that Spring takes over.

Kafka Streams vs the Alternatives

When is Kafka Streams the right tool at all?

	Kafka Streams	Flink	Spark Streaming	Plain consumers
What it is	Library in your app	Separate cluster	Separate cluster	DIY everything
Ops burden	None (it's your app)	Heavy (JobManager etc.)	Heavy	Low, but you build the features
Latency	Milliseconds	Milliseconds	Seconds (micro-batches)	Milliseconds
Event time + watermarks	Good	Best-in-class	Good	DIY
Exactly-once	✅	✅	✅	DIY (very hard)
Best when	Data lives in Kafka, team knows Java	Complex event-time logic, sources beyond Kafka	Already on Spark, big batch+stream jobs	Trivial pass-through only

The honest summary: if your data is in Kafka and you're a Java shop, Kafka Streams is the default choice — nothing to run, nothing extra to monitor, and it scales exactly like your other services do.

The Production Checklist

Everything that matters in one list. Before an app goes live, run through it:

application.id is deliberate and permanent — it names the consumer group, changelog topics, and repartition topics. Changing it orphans all of them.
State stores are named (Materialized.as("...")) — unnamed stores get auto-generated names that change with code edits and break recovery.
Serdes are explicit everywhere data crosses a boundary: Consumed.with(...), Produced.with(...), Materialized...withValueSerde(...).
A deserialization handler is set — decide skip-vs-fail on purpose, not by accident, and metric the skip rate.
A DLQ exists for processing errors, with an owner and a replay plan for its contents.
Grace periods are set on windows sized to your actual lateness, not the default.
Exactly-once is on — if and only if duplicates cost you something real.
Partitions cover your growth plan — task parallelism is capped by them, and changing them later is painful.
Standby replicas (NUM_STANDBY_REPLICAS_CONFIG = 1) if state is large and failover speed matters.
Topology extracted into its own method with TopologyTestDriver tests on the happy path, the weird records, and the window edges.
Monitoring on consumer lag and state store size — lag climbing means your app can't keep up with the input; store size climbing forever usually means windows or stores that never release old keys.
A deployment story for topology changes — adding steps mid-pipeline affects the numbered internal topics; test upgrades against a copy of real data before prod.

Wrap-Up: What You Now Know

Across this series, you went from "what is stream processing?" to being able to ship a production-grade pipeline:

Part 1 — Kafka basics, topologies, and your first running app
Part 2 — KStreams and KTables, stateful operations, joins, windows, and where state lives
Part 3 — Serdes, time, exactly-once, error handling, scaling, and testing

The best next step is a real one: take an actual topic in your world — clicks, orders, logs, sensor readings — and build a small pipeline against it with TopologyTestDriver tests, exactly-once, and a DLQ. Ship it small, then grow it.

When you're ready for more depth, the official Kafka Streams documentation and the Kafka: The Definitive Guide book are excellent companions.

Thanks for reading the whole series — now go build something that reacts in milliseconds. ⚡

Key Takeaways from Part 3:

Custom types need a serde — set it at every boundary: Consumed, Produced, and Materialized (forgetting the store serde is the classic runtime crash)

Kafka Streams uses event time by default; a custom TimestampExtractor reads the timestamp from inside your records

EXACTLY_ONCE_V2 makes consume→update→produce atomic for one config line — pay for it only where duplicates hurt

Poison pills kill unhandled apps: set a deserialization handler, and route processing errors to a DLQ with a replay plan

Parallelism = tasks = input partitions, spread across threads and instances; standby replicas make failover fast

TopologyTestDriver tests your full topology without a broker — extract the topology into its own method and test window edges with explicit timestamps

If your data is in Kafka and you write Java, Kafka Streams is the default answer

In Part 1 we built a working app. In Part 2 we mastered streams, tables, joins, and windows. Everything so far runs great — on your laptop, with clean data, at 11 AM on a Tuesday.

Serdes for Your Own Classes: The JSON Serde

String serdes got us far, but real apps move objects: Order, Payment, User. Kafka stores bytes, so we need to teach it JSON. Here's a compact, reusable serde using Jackson:

import com.fasterxml.jackson.databind.ObjectMapper;

public class JsonSerde<T> implements Serde<T> {

    private final Class<T> type;
    private final ObjectMapper mapper = new ObjectMapper();

    public JsonSerde(Class<T> type) {
        this.type = type;
    }

    private final Serializer<T> serializer = new Serializer<>() {
        @Override
        public byte[] serialize(String topic, T data) {
            try {
                return data == null ? null : mapper.writeValueAsBytes(data);
            } catch (JsonProcessingException e) {
                throw new IllegalArgumentException(
                        "Cannot serialize record for topic " + topic, e);
            }
        }
    };

    private final Deserializer<T> deserializer = new Deserializer<>() {
        @Override
        public T deserialize(String topic, byte[] bytes) {
            try {
                return bytes == null ? null : mapper.readValue(bytes, type);
            } catch (IOException e) {
                throw new IllegalArgumentException(
                        "Cannot deserialize record from topic " + topic, e);
            }
        }
    };

    @Override public Serializer<T> serializer()   { return serializer; }
    @Override public Deserializer<T> deserializer() { return deserializer; }
}

Now use it wherever data crosses a boundary — in, out, or into state:

JsonSerde<Order> orderSerde = new JsonSerde<>(Order.class);

// Reading from a topic
KStream<String, Order> orders = builder.stream(
        "orders",
        Consumed.with(Serdes.String(), orderSerde));

// Writing to a topic
results.to("order-totals",
        Produced.with(Serdes.String(), Serdes.Double()));

// Storing in a state store
.aggregate(
        () -> 0.0,
        (key, order, total) -> total + order.amount(),
        Materialized.<String, Double, KeyValueStore<Bytes, byte[]>>as("order-totals")
                .withValueSerde(Serdes.Double()));

Time: Which Clock Wins?

Every record has a timestamp, and which clock it comes from changes your results:

Event time — when the event actually happened (embedded in the data, e.g. order.eventTime). What really happened.
Processing time — when your app handled it. Includes every delay in between: producer buffering, network, retries, a rebalance. What your app saw.

Kafka Streams agrees: it uses event time by default. A record's event time comes from (in order) the producer's timestamp or, when the data knows better, a custom extractor:

public class OrderTimestampExtractor implements TimestampExtractor {
    @Override
    public long extract(ConsumerRecord<Object, Object> record, long partitionTime) {
        if (record.value() instanceof Order order) {
            return order.eventTime().toEpochMilli(); // the time that matters
        }
        return partitionTime; // fall back when the record is unusable
    }
}

props.put(StreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG,
        OrderTimestampExtractor.class);

Exactly-Once: One Config Line

For money, inventory, or anything where duplicates cause real damage, flip one switch:

props.put(StreamsConfig.PROCESSING_GUARANTEE_CONFIG,
        StreamsConfig.EXACTLY_ONCE_V2);

Error Handling: Poison Pills and Dead Letters

Deserialization Errors: Skip or Stop

Implement DeserializationExceptionHandler to choose per record:

public class LogAndSkipHandler implements DeserializationExceptionHandler {

    @Override
    public DeserializationHandlerResponse handle(
            ProcessorContext context,
            ConsumerRecord<byte[], byte[]> record,
            Exception exception) {

        log.error("Skipping bad record: topic={} partition={} offset={}",
                record.topic(), record.partition(), record.offset(), exception);

        return DeserializationHandlerResponse.CONTINUE; // skip it, keep flowing
        // return DeserializationHandlerResponse.FAIL;  // or: die (the default)
    }

    @Override
    public void configure(Map<String, ?> configs) { }
}

props.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG,
        LogAndSkipHandler.class);

Processing Errors: The Dead-Letter Queue

KStream<String, String> cleaned = sentences.process(() -> new Processor<String, String, String, String>() {

    private ProcessorContext<String, String> context;
    private KafkaProducer<byte[], byte[]> dlqProducer;

    @Override
    public void init(ProcessorContext<String, String> context) {
        this.context = context;
        this.dlqProducer = new KafkaProducer<>(Map.of(
                ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092",
                ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class,
                ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class));
    }

    @Override
    public void process(Record<String, String> record) {
        try {
            context.forward(record);              // happy path: pass it on
        } catch (Exception e) {
            dlqProducer.send(new ProducerRecord<>("sentences-dlq",   // quarantine it
                    record.key() == null ? null : record.key().getBytes(),
                    record.value().getBytes()));
            log.warn("Record sent to DLQ", e);
        }
    }

    @Override
    public void close() { dlqProducer.close(); }
});

Later — an hour, a day — a small job replays sentences-dlq after the bug is fixed. Nothing is lost, nothing blocks.

Scaling and Fault Tolerance

Remember from Part 1: tasks = partitions of the input topic. Tasks are the atoms of parallelism — they cannot be split further. You distribute them in two dimensions:

// More threads per instance (uses more cores):
props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, 4);

// Or more instances (uses more machines):
//   just start the app again with the same application.id

Testing Without a Broker

Kafka Streams has a gift for testability: TopologyTestDriver runs your whole topology in memory, with no Kafka running. No Docker, no flaky integration setup, no excuses for skipping tests:

@Test
void countsWords() {
    Topology topology = WordCountTopology.build();   // extract your builder
                                                     // code into a testable method!
    Properties props = new Properties();
    props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
    props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "nowhere:9092"); // never contacted

    try (TopologyTestDriver driver = new TopologyTestDriver(topology, props)) {

        TestInputTopic<String, String> input = driver.createInputTopic(
                "sentences", new StringSerializer(), new StringSerializer());

        TestOutputTopic<String, Long> output = driver.createOutputTopic(
                "word-counts", new StringDeserializer(), new LongDeserializer());

        // act
        input.pipeValue("kafka streams is awesome kafka");

        // assert
        Map<String, Long> counts = output.readKeyValuesToMap();
        assertThat(counts).containsEntry("kafka", 2L);
    }
}

Notice the test even asserts kafka → 2 from a single sentence — real aggregate logic, verified in milliseconds.

Testing windows is where this shines even brighter: instead of waiting for wall-clock time to pass, you can control record timestamps directly:

// Feed events with explicit timestamps to test window boundaries
input.pipeKeyValueList(List.of(
        new KeyValue<>("user-1", "click"),
        new KeyValue<>("user-1", "click")));
// windowed output topics can also be read key-by-key:
// var windowed = output.readKeyValue(); // Windowed<String> key + Long count

Kafka Streams vs the Alternatives

When is Kafka Streams the right tool at all?

	Kafka Streams	Flink	Spark Streaming	Plain consumers
What it is	Library in your app	Separate cluster	Separate cluster	DIY everything
Ops burden	None (it's your app)	Heavy (JobManager etc.)	Heavy	Low, but you build the features
Latency	Milliseconds	Milliseconds	Seconds (micro-batches)	Milliseconds
Event time + watermarks	Good	Best-in-class	Good	DIY
Exactly-once	✅	✅	✅	DIY (very hard)
Best when	Data lives in Kafka, team knows Java	Complex event-time logic, sources beyond Kafka	Already on Spark, big batch+stream jobs	Trivial pass-through only

The Production Checklist

Everything that matters in one list. Before an app goes live, run through it:

application.id is deliberate and permanent — it names the consumer group, changelog topics, and repartition topics. Changing it orphans all of them.
State stores are named (Materialized.as("...")) — unnamed stores get auto-generated names that change with code edits and break recovery.
Serdes are explicit everywhere data crosses a boundary: Consumed.with(...), Produced.with(...), Materialized...withValueSerde(...).
A deserialization handler is set — decide skip-vs-fail on purpose, not by accident, and metric the skip rate.
A DLQ exists for processing errors, with an owner and a replay plan for its contents.
Grace periods are set on windows sized to your actual lateness, not the default.
Exactly-once is on — if and only if duplicates cost you something real.
Partitions cover your growth plan — task parallelism is capped by them, and changing them later is painful.
Standby replicas (NUM_STANDBY_REPLICAS_CONFIG = 1) if state is large and failover speed matters.
Topology extracted into its own method with TopologyTestDriver tests on the happy path, the weird records, and the window edges.
Monitoring on consumer lag and state store size — lag climbing means your app can't keep up with the input; store size climbing forever usually means windows or stores that never release old keys.
A deployment story for topology changes — adding steps mid-pipeline affects the numbered internal topics; test upgrades against a copy of real data before prod.

Wrap-Up: What You Now Know

Across this series, you went from "what is stream processing?" to being able to ship a production-grade pipeline:

Part 1 — Kafka basics, topologies, and your first running app
Part 2 — KStreams and KTables, stateful operations, joins, windows, and where state lives
Part 3 — Serdes, time, exactly-once, error handling, scaling, and testing

When you're ready for more depth, the official Kafka Streams documentation and the Kafka: The Definitive Guide book are excellent companions.

Thanks for reading the whole series — now go build something that reacts in milliseconds. ⚡

Key Takeaways from Part 3:

Custom types need a serde — set it at every boundary: Consumed, Produced, and Materialized (forgetting the store serde is the classic runtime crash)

Kafka Streams uses event time by default; a custom TimestampExtractor reads the timestamp from inside your records

EXACTLY_ONCE_V2 makes consume→update→produce atomic for one config line — pay for it only where duplicates hurt

Poison pills kill unhandled apps: set a deserialization handler, and route processing errors to a DLQ with a replay plan

Parallelism = tasks = input partitions, spread across threads and instances; standby replicas make failover fast

TopologyTestDriver tests your full topology without a broker — extract the topology into its own method and test window edges with explicit timestamps

If your data is in Kafka and you write Java, Kafka Streams is the default answer

Kafka Streams Part 3: Making It Production-Ready

Kafka Streams: From Zero to Production(3-part series)

Serdes for Your Own Classes: The JSON Serde

Time: Which Clock Wins?

Exactly-Once: One Config Line

Error Handling: Poison Pills and Dead Letters

Deserialization Errors: Skip or Stop

Processing Errors: The Dead-Letter Queue

Scaling and Fault Tolerance

Testing Without a Broker

Kafka Streams vs the Alternatives

The Production Checklist

Wrap-Up: What You Now Know

Tags

Related Posts

Kafka Streams Part 1: Stream Processing Made Simple

Kafka Streams Part 2: KStreams, KTables, Joins and Windows

Kafka Streams with Spring Boot: Real-Time Pipelines the Spring Way

Kafka Streams Part 3: Making It Production-Ready

Kafka Streams: From Zero to Production(3-part series)

Serdes for Your Own Classes: The JSON Serde

Time: Which Clock Wins?

Exactly-Once: One Config Line

Error Handling: Poison Pills and Dead Letters

Deserialization Errors: Skip or Stop

Processing Errors: The Dead-Letter Queue

Scaling and Fault Tolerance

Testing Without a Broker

Kafka Streams vs the Alternatives

The Production Checklist

Wrap-Up: What You Now Know

Tags

Related Posts

Kafka Streams Part 1: Stream Processing Made Simple

Kafka Streams Part 2: KStreams, KTables, Joins and Windows

Kafka Streams with Spring Boot: Real-Time Pipelines the Spring Way