Transactions and exactly-once
Delivery over a network is at least once: if an answer is lost, the sender asks again. Lotse turns that into exactly once in two steps. Idempotent producers remove duplicates on the way in. Transactions make a read, a change and a write happen all together or not at all.
Idempotent producers
Every producer has an id from the cluster and numbers its batches. A leader that sees a number twice answers with the earlier result and writes nothing. The Java and Rust clients do this by default, so a retried produce does not duplicate. A producer with a name (producerName) keeps its identity across restarts, and a second instance with the same name fences the first (ADR 0028, ADR 0071).
When the outcome of a write is unknown and the client cannot find out, it throws ClientException.Unknown. That is the one case where you decide.
Consume, transform, produce
The example reads orders as a consumer group member and writes one invoice per order. Each poll’s invoices and the group’s offsets commit in one transaction. If the process stops in between, neither the invoices nor the new offsets exist, and the next instance reads the same orders again.
package dev.lotse.examples;
import dev.lotse.client.ClientException;
import dev.lotse.client.GroupConfig;
import dev.lotse.client.GroupConsumer;
import dev.lotse.client.GroupRecord;
import dev.lotse.client.LotseClient;
import dev.lotse.client.OutMessage;
import dev.lotse.client.Transaction;
import java.time.Duration;
import java.util.List;
/**
* Consume, transform, produce, exactly once: a group member reads orders, and each batch of invoices it writes
* commits in one transaction with the group's offsets. A crash between the two leaves neither.
*/
public final class ExactlyOnceExample {
private ExactlyOnceExample() {}
public static void main(String[] args) {
// One running client per transactional id: a second one fences the first.
try (LotseClient client = new LotseClient(Env.config().transactionalId("example-billing-1").build())) {
try {
client.createStream("example-invoices", 1, 3);
} catch (ClientException.ResourceExists e) {
// made by an earlier run
}
GroupConfig config = GroupConfig.builder().autoCommit(false).build(); // offsets go with the transaction
try (GroupConsumer group = client.joinGroup("example-billing", List.of("example-orders"), config)) {
long deadline = System.nanoTime() + Duration.ofSeconds(20).toNanos();
while (System.nanoTime() < deadline) {
List<GroupRecord> orders = group.poll(Duration.ofMillis(500)); // heartbeats too: call it often
if (orders.isEmpty()) {
continue;
}
try (Transaction txn = client.beginTransaction()) {
for (GroupRecord order : orders) {
txn.produce("example-invoices", 0,
OutMessage.value("invoice for " + order.record().value().toUtf8()));
}
txn.sendOffsets(group);
txn.commit();
System.out.printf("invoiced %d orders%n", orders.size());
} catch (ClientException e) {
// Not committed: read the same orders again from the group's committed offsets.
System.out.println("transaction failed, reading again: " + e.getMessage());
group.rewind();
}
}
}
}
}
}
Points to note:
- One client per transactional id.
transactionalId("example-billing-1")names the client’s identity. Starting a second client with the same id fences the first. - Turn auto-commit off for the group (
autoCommit(false)). The offsets belong to the transaction:txn.sendOffsets(group). - Call
group.polloften. It sends the group heartbeat. commit()ends the transaction.close()withoutcommit()orabort()leaves the decision to the coordinator, which aborts it, and the client’s next transaction starts with a fresh producer id; callabort()when you want it gone at once. OnClientExceptionthe example callsgroup.rewind()so the next poll reads from the committed offsets again.ClientException.MustAbortsays the transaction can no longer commit; abort it.
The same in Kotlin
transaction { } commits when its block returns and aborts on an exception or cancellation (ADR 0080).
package dev.lotse.examples
import dev.lotse.client.ClientException
import dev.lotse.client.GroupRecord
import dev.lotse.kotlin.groupConfig
import dev.lotse.kotlin.lotse
import dev.lotse.kotlin.message
import java.time.Duration
import kotlinx.coroutines.flow.take
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.withTimeoutOrNull
/**
* Writes orders, then invoices them exactly once from a consumer group, each poll's orders in one transaction with
* the group's offsets. Brokers from `LOTSE_BOOTSTRAP` (default `127.0.0.1:9092`), plain TCP on loopback.
*/
fun main(): Unit = runBlocking {
val bootstrap = (System.getenv("LOTSE_BOOTSTRAP") ?: "127.0.0.1:9092").split(",").toTypedArray()
lotse(*bootstrap) {
plaintextOnLoopback()
transactionalId("example-billing-kt")
}.use { lotse ->
for (stream in listOf("example-orders-kt", "example-invoices-kt")) {
try {
lotse.createStream(stream, partitions = 1, replication = 3)
} catch (e: ClientException.ResourceExists) {
// made by an earlier run
}
}
repeat(10) { lotse.produce("example-orders-kt", 0, message("order-$it", key = "customer-${it % 3}")) }
lotse.joinGroup("example-billing-kt", listOf("example-orders-kt"), groupConfig { autoCommit(false) }).use { group ->
withTimeoutOrNull(Duration.ofSeconds(20).toMillis()) {
var invoiced = 0
while (invoiced < 10) {
val orders: List<GroupRecord> = group.poll(Duration.ofMillis(500))
if (orders.isEmpty()) continue
try {
lotse.transaction {
for (order in orders) {
produce("example-invoices-kt", 0, message("invoice for ${order.record().value().toUtf8()}"))
}
sendOffsets(group)
}
invoiced += orders.size
println("invoiced $invoiced orders")
} catch (e: ClientException) {
group.rewind() // not committed: read them again
}
}
}
}
lotse.records("example-invoices-kt", 0, from = 0).take(10).collect { println(it.value().toUtf8()) }
}
}
What a transaction can hold
Inside one transaction you can:
produceto any number of streams and queues;publishthrough an exchange (the client routes, and the message reaches all its queues or none);sendOffsets(group)to commit a consumer group’s offsets;queue.ackIn(txn, delivery)to acknowledge queue messages.
A transaction that is neither committed nor aborted is aborted by the coordinator after txnTimeout.
Consume from a queue, produce, exactly once
ackIn ties a queue acknowledgement to the transaction. The commit settles the message; an abort gives it back to the queue.
// Not from the compiled examples.
Delivery job = jobs.poll(Duration.ofSeconds(5));
try (Transaction txn = client.beginTransaction()) {
txn.produce("results", 0, OutMessage.value(compute(job.value().toUtf8())));
jobs.ackIn(txn, job);
txn.commit();
}
Rules to know (ADR 0075):
- Ack on the connection the message came on; the client does this for you. After a leader change it subscribes again as the same consumer and sends the ack there, so the transaction goes on. A message that is not in flight with your subscription gets
MESSAGE_NOT_HELD; abort. - Of two transactions that ack the same message, the first in the log holds it; the other gets
MESSAGE_NOT_HELD. - A held message is not ready: purge leaves it, and expiry and the delivery limit do not touch it.
What readers see
A fetch is read_committed by default: records of open and aborted transactions are not returned. A reader never sees a half-written transaction. Messages in a queue that a transaction wrote become deliverable when it commits.
Limits of the guarantee
- Exactly once holds for effects inside Lotse: streams, queues, offsets, acks. A call to a database or an e-mail service inside your code is outside it; make that call idempotent.
- A plain
publishoutside a transaction that ends inUnknownis not retried by the client (see Exchanges and routing). - A stream consumer that commits offsets outside a transaction gives at-least-once.