The naive answer is “retry.” The dangerous truth is that retrying a non-idempotent operation can charge a card twice, create duplicate orders, or send the same email three times. This article walks through why retries break things, what an idempotency key actually buys you, and how to build a production-grade idempotency layer in Spring Boot 3.x backed by Redis — handling concurrent duplicates, conflicting bodies, and key expiry along the way.

Why retries break things

Consider a textbook failure. A client calls POST /payments. The server receives the request, charges the card, commits the transaction, and starts streaming the 201 Created response. Then the client's socket times out before the last byte arrives.

From the server’s perspective: success. From the client’s perspective: unknown. The HTTP spec gives the client no way to distinguish “the server never saw my request” from “the server processed it but I lost the reply.” This is the classic two-generals problem dressed up in TCP.

The client’s retry library — Resilience4j, an Axios interceptor, a mobile SDK with exponential backoff — does the only sensible thing it can: it sends the request again. Without server-side protection, the second request charges the card a second time.

The failure modes that produce duplicate effects:

• Read timeouts: the response is slow, the client gives up and retries while the original is still completing.
• Connection resets: a proxy or load balancer recycles a connection mid-flight.
• Client crashes and restarts: an at-least-once job queue redelivers a message.
• User double-clicks: the most boring and most common cause of duplicate orders.

The goal is not to prevent retries — retries are good, they are how distributed systems achieve reliability. The goal is to make retries safe by guaranteeing that the same logical operation, repeated any number of times, produces exactly one effect and returns the same response.

Idempotency, safety, and HTTP methods

Two related but distinct properties matter here.

• Safe: the method has no side effects on server state. Reading is safe; writing is not.
• Idempotent: making the request N times has the same effect on server state as making it once. The response may differ, but the resulting state does not.

HTTP gives us baseline guarantees per method:

Method Safe Idempotent Typical effect Naturally retry-safe? GET Yes Yes Read a resource Yes HEAD Yes Yes Read headers only Yes PUT No Yes Replace a resource at a known URI Yes DELETE No Yes Remove a resource Yes (second is no-op) POST No No Create a subordinate / trigger an effect No PATCH No No* Partial update Depends on payload

PUT is idempotent because it sets a resource to a fully specified state: PUT /users/42 with the same body twice leaves user 42 in the same state. DELETE /users/42 is idempotent because once it's gone, deleting it again changes nothing (you may return 404 the second time, but the state is identical).

POST is the problem child. POST /payments creates a new thing each time. There is no URI for the not-yet-created resource, so the method itself can't be idempotent. PATCH is conditional: PATCH {balance: 100} is idempotent, but PATCH {balance: balance + 10} is not.

The strategy, then, is to make the naturally-idempotent methods safe by design, and to add an explicit idempotency mechanism on top of POST (and any non-idempotent PATCH).

The idempotency key concept

The pattern, popularized by Stripe and now adopted across the industry, is simple: the client generates a unique key for each logical operation and sends it in a header.

POST /payments HTTP/1.1
Content-Type: application/json
Idempotency-Key: 8f14e45f-ceea-467f-9b3a-1d2f0e9c7a55

{ "amount": 4999, "currency": "eur", "source": "card_abc" }

The key is a client-generated, opaque, unique string — usually a UUID v4. The contract is:

• The client uses the same key for all retries of the same logical operation.
• The client uses a fresh key for each new operation, even if the body is identical.

The server’s job is to recognize repeated keys and ensure the underlying effect happens exactly once, replaying the original response for every retry.

Storing the request fingerprint and result

When the server sees an idempotency key for the first time, it:

1. Records the key together with a fingerprint of the request (a hash of method + path + body). The fingerprint guards against a client reusing a key with a different request — almost always a client bug, and one we must reject rather than silently corrupt.
2. Executes the operation.
3. Stores the resulting response (status, headers, body) against the key.

On any subsequent request with the same key:

• If the fingerprint matches and a stored response exists, replay the stored response without re-executing.
• If the fingerprint differs, return 409 Conflict — the key was reused for a different request.
• If the operation is still in flight (a concurrent duplicate arrived before the first finished), return 409 Conflict with a “retry later” semantic, so the caller doesn’t kick off a parallel charge.

┌────────────────────────────────────────────────────────┐
            │                    Idempotency Store (Redis)            │
            │   key -> { state, fingerprint, status, headers, body }  │
            └────────────────────────────────────────────────────────┘
                          ⚯                ▲                  ▲
        (1) reserve key  │   (3) save      │   (replay)       │
            SETNX        │   response      │   GET            │
                          │                │                  │
   client ──POST──▶ ┌──────────────────────────────────────────────┐
        retry ────▶ │  IdempotencyFilter (OncePerRequestFilter)      │
                    │                                                │
                    │  no key?  ──────────────▶ pass through         │
                    │  new key  ──reserve──▶ run controller ──save──▶ │
                    │  in-flight ─────────────▶ 409 (retry later)    │
                    │  completed + same body ─▶ replay stored 2xx    │
                    │  completed + diff body ─▶ 409 Conflict         │
                    └──────────────────────────────────────────────┘
                                       │
                                       ▼
                              ┌─────────────────┐
                              │  PaymentService │  (executed at most once)
                              └─────────────────┘

Why Redis

The idempotency store has demanding requirements: extremely low latency on the hot path, atomic “reserve-if-absent” semantics for concurrency control, and automatic expiry so keys don’t accumulate forever. Redis fits all three:

• SET key value NX PX ttl gives us atomic reservation in a single round trip.
• TTLs are first-class, so keys self-expire (typically 24h — long enough to cover any reasonable client retry window, short enough to bound storage).
• It’s shared across all application instances, so the guarantee holds in a horizontally-scaled deployment where retries may hit different pods.

A relational table works too, but you then need a unique constraint plus careful transaction isolation to avoid races, and a scheduled job to purge expired rows. Redis collapses that into the data model itself.

Implementation: Spring Boot 3.x + Redis

We’ll build the layer as an OncePerRequestFilter. A filter (rather than a HandlerInterceptor) lets us wrap the response and capture the bytes the controller writes, which we need in order to replay them later. We need three pieces: a record describing stored state, a repository over StringRedisTemplate, and the filter itself.

1. The stored record

We model two states explicitly: IN_PROGRESS (reserved, not yet completed) and COMPLETED (response captured).

package com.devdomain.idempotency;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;

import java.util.Map;

public record IdempotencyRecord(
        @JsonProperty("state") State state,
        @JsonProperty("fingerprint") String fingerprint,
        @JsonProperty("status") int status,
        @JsonProperty("headers") Map<String, String> headers,
        @JsonProperty("body") String body) {

    public enum State { IN_PROGRESS, COMPLETED }

    @JsonCreator
    public IdempotencyRecord {
        // Jackson canonical constructor; nothing extra to validate here.
    }

    static IdempotencyRecord inProgress(String fingerprint) {
        return new IdempotencyRecord(State.IN_PROGRESS, fingerprint, 0, Map.of(), null);
    }

    IdempotencyRecord completed(int status, Map<String, String> headers, String body) {
        return new IdempotencyRecord(State.COMPLETED, this.fingerprint, status, headers, body);
    }
}

2. The Redis repository

The repository owns serialization and the atomic reservation primitive. The key insight is setIfAbsent (which maps to Redis SET ... NX): it returns true only for the caller that wins the race to create the key. Everyone else gets false and must treat the request as a duplicate.

package com.devdomain.idempotency;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Repository;

import java.time.Duration;
import java.util.Optional;

@Repository
public class IdempotencyRepository {

    private static final String KEY_PREFIX = "idem:";
    private static final Duration TTL = Duration.ofHours(24);

    private final StringRedisTemplate redis;
    private final ObjectMapper objectMapper;

    public IdempotencyRepository(StringRedisTemplate redis, ObjectMapper objectMapper) {
        this.redis = redis;
        this.objectMapper = objectMapper;
    }

    /**
     * Atomically reserve the key. Returns true only if no record existed.
     * Backed by Redis SET key value NX PX ttl.
     */
    public boolean tryReserve(String key, IdempotencyRecord record) {
        Boolean reserved = redis.opsForValue()
                .setIfAbsent(redisKey(key), serialize(record), TTL);
        return Boolean.TRUE.equals(reserved);
    }

    public Optional<IdempotencyRecord> find(String key) {
        String raw = redis.opsForValue().get(redisKey(key));
        if (raw == null) {
            return Optional.empty();
        }
        return Optional.of(deserialize(raw));
    }

    /**
     * Overwrite the in-progress record with the completed response,
     * preserving the remaining TTL window for replay.
     */
    public void complete(String key, IdempotencyRecord record) {
        redis.opsForValue().set(redisKey(key), serialize(record), TTL);
    }

    /**
     * Release a reservation if the operation failed before completing,
     * so a genuine retry is allowed to proceed.
     */
    public void release(String key) {
        redis.delete(redisKey(key));
    }

    private String redisKey(String key) {
        return KEY_PREFIX + key;
    }

    private String serialize(IdempotencyRecord record) {
        try {
            return objectMapper.writeValueAsString(record);
        } catch (JsonProcessingException e) {
            throw new IllegalStateException("Failed to serialize idempotency record", e);
        }
    }

    private IdempotencyRecord deserialize(String raw) {
        try {
            return objectMapper.readValue(raw, IdempotencyRecord.class);
        } catch (JsonProcessingException e) {
            throw new IllegalStateException("Corrupt idempotency record in Redis", e);
        }
    }
}

3. The filter

The filter is where the state machine lives. It only engages for unsafe methods that carry an Idempotency-Key. For everything else it gets out of the way.

package com.devdomain.idempotency;

import com.fasterxml.jackson.databind.ObjectMapper;
import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Component;
import org.springframework.util.DigestUtils;
import org.springframework.web.filter.OncePerRequestFilter;
import org.springframework.web.util.ContentCachingRequestWrapper;
import org.springframework.web.util.ContentCachingResponseWrapper;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Optional;
import java.util.Set;

@Component
public class IdempotencyFilter extends OncePerRequestFilter {

    private static final String HEADER = "Idempotency-Key";
    private static final String REPLAYED_HEADER = "Idempotent-Replayed";
    private static final Set<String> GUARDED_METHODS = Set.of(
            HttpMethod.POST.name(), HttpMethod.PATCH.name());

    private final IdempotencyRepository repository;
    private final ObjectMapper objectMapper;

    public IdempotencyFilter(IdempotencyRepository repository, ObjectMapper objectMapper) {
        this.repository = repository;
        this.objectMapper = objectMapper;
    }

    @Override
    protected boolean shouldNotFilter(HttpServletRequest request) {
        boolean guardedMethod = GUARDED_METHODS.contains(request.getMethod());
        boolean hasKey = request.getHeader(HEADER) != null;
        return !(guardedMethod && hasKey);
    }

    @Override
    protected void doFilterInternal(HttpServletRequest request,
                                    HttpServletResponse response,
                                    FilterChain chain)
            throws ServletException, IOException {

        String key = request.getHeader(HEADER);
        if (key.isBlank() || key.length() > 200) {
            sendProblem(response, HttpServletResponse.SC_BAD_REQUEST,
                    "Idempotency-Key must be a non-empty string of at most 200 characters.");
            return;
        }

        // Wrap the request so we can read the body and still hand it to the controller.
        ContentCachingRequestWrapper req = new ContentCachingRequestWrapper(request);
        // Force the body to be read into the cache for fingerprinting.
        req.getInputStream().readAllBytes();
        String fingerprint = fingerprint(req);

        Optional<IdempotencyRecord> existing = repository.find(key);
        if (existing.isPresent()) {
            handleExisting(existing.get(), fingerprint, response);
            return;
        }

        // No record yet: try to atomically reserve the key.
        boolean reserved = repository.tryReserve(key, IdempotencyRecord.inProgress(fingerprint));
        if (!reserved) {
            // Lost the race to a concurrent duplicate; re-read and handle.
            Optional<IdempotencyRecord> raced = repository.find(key);
            if (raced.isPresent()) {
                handleExisting(raced.get(), fingerprint, response);
            } else {
                // Reserved-then-expired in a tiny window; treat as transient.
                sendProblem(response, HttpServletResponse.SC_CONFLICT,
                        "Concurrent request with the same Idempotency-Key; retry shortly.");
            }
            return;
        }

        // We own the key. Execute the operation and capture the response.
        executeAndStore(req, response, chain, key, fingerprint);
    }

    private void handleExisting(IdempotencyRecord record,
                                String fingerprint,
                                HttpServletResponse response) throws IOException {
        if (!record.fingerprint().equals(fingerprint)) {
            sendProblem(response, HttpServletResponse.SC_CONFLICT,
                    "Idempotency-Key was already used with a different request payload.");
            return;
        }
        if (record.state() == IdempotencyRecord.State.IN_PROGRESS) {
            response.setHeader(HttpHeaders.RETRY_AFTER, "1");
            sendProblem(response, HttpServletResponse.SC_CONFLICT,
                    "A request with this Idempotency-Key is still being processed.");
            return;
        }
        replay(record, response);
    }

    private void executeAndStore(ContentCachingRequestWrapper req,
                                 HttpServletResponse response,
                                 FilterChain chain,
                                 String key,
                                 String fingerprint) throws ServletException, IOException {
        ContentCachingResponseWrapper res = new ContentCachingResponseWrapper(response);
        boolean completed = false;
        try {
            chain.doFilter(req, res);

            int status = res.getStatus();
            // Only persist successful, deterministic responses for replay.
            if (status >= 200 && status < 300) {
                Map<String, String> headers = captureHeaders(res);
                String body = new String(res.getContentAsByteArray(), responseCharset(res));
                IdempotencyRecord done = IdempotencyRecord
                        .inProgress(fingerprint)
                        .completed(status, headers, body);
      2         repository.complete(key, done);
                completed = true;
            }
        } finally {
            if (!completed) {
                // 4xx/5xx or an exception: release so the client can retry cleanly.
                repository.release(key);
            }
            res.copyBodyToResponse();
        }
    }

    private void replay(IdempotencyRecord record, HttpServletResponse response) throws IOException {
        response.setStatus(record.status());
        record.headers().forEach(response::setHeader);
        response.setHeader(REPLAYED_HEADER, "true");
        if (record.body() != null) {
            byte[] bytes = record.body().getBytes(StandardCharsets.UTF_8);
            response.setContentLength(bytes.length);
            response.getOutputStream().write(bytes);
        }
        response.flushBuffer();
    }

    private String fingerprint(ContentCachingRequestWrapper req) {
        byte[] body = req.getContentAsByteArray();
        String material = req.getMethod() + '\n'
                + req.getRequestURI() + '\n'
                + (req.getQueryString() == null ? "" : req.getQueryString()) + '\n'
                + new String(body, StandardCharsets.UTF_8);
        return DigestUtils.md5DigestAsHex(material.getBytes(StandardCharsets.UTF_8));
    }

    private Map<String, String> captureHeaders(ContentCachingResponseWrapper res) {
        Map<String, String> headers = new LinkedHashMap<>();
        for (String name : res.getHeaderNames()) {
            // Skip hop-by-hop and length headers we recompute on replay.
            if (HttpHeaders.CONTENT_LENGTH.equalsIgnoreCase(name)) {
                continue;
            }
            headers.put(name, res.getHeader(name));
        }
        return headers;
    }

    private String responseCharset(ContentCachingResponseWrapper res) {
        return res.getCharacterEncoding() != null
                ? res.getCharacterEncoding()
                : StandardCharsets.UTF_8.name();
    }

    private void sendProblem(HttpServletResponse response, int status, String detail)
            throws IOException {
        response.setStatus(status);
        response.setContentType(MediaType.APPLICATION_PROBLEM_JSON_VALUE);
        String body = """
                {"type":"about:blank","status":%d,"title":"%s","detail":"%s"}"""
                .formatted(status, reason(status), detail);
        response.getWriter().write(body);
    }

    private String reason(int status) {
        return switch (status) {
            case 400 -> "Bad Request";
            case 409 -> "Conflict";
            default -> "Error";
        };
    }
}

4. Wiring and the protected endpoint

There’s nothing exotic to configure. Spring Boot autoconfigures StringRedisTemplate from spring-boot-starter-data-redis, and component-scanning registers the filter. The controller is written as if idempotency didn't exist — that separation of concerns is the whole point.

package com.devdomain.payments;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/payments")
public class PaymentController {

    private final PaymentService paymentService;

    public PaymentController(PaymentService paymentService) {
        this.paymentService = paymentService;
    }

    @PostMapping
    public ResponseEntity<PaymentResponse> charge(@RequestBody ChargeRequest request) {
        Payment payment = paymentService.charge(
                request.amount(), request.currency(), request.source());
        return ResponseEntity
                .status(HttpStatus.CREATED)
                .body(PaymentResponse.from(payment));
    }
}

# application.properties
spring.data.redis.host=localhost
spring.data.redis.port=6379
spring.data.redis.timeout=200ms

To make the filter run before Spring Security and other filters that read the body, register its order explicitly if needed:

package com.devdomain.idempotency;

import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.Ordered;

@Configuration
public class IdempotencyConfig {

    @Bean
    public FilterRegistrationBean<IdempotencyFilter> idempotencyRegistration(
            IdempotencyFilter filter) {
        FilterRegistrationBean<IdempotencyFilter> reg = new FilterRegistrationBean<>(filter);
        reg.setOrder(Ordered.HIGHEST_PRECEDENCE + 10);
        reg.addUrlPatterns("/payments/*", "/orders/*");
        return reg;
    }
}

Note: because the filter is also a @Component, you may want to mark the bean so Spring doesn't auto-register it twice. Annotate the filter class with @Component or register it via FilterRegistrationBean, not both — pick one. If you use FilterRegistrationBean, drop the @Component and inject dependencies through the config class.

The concurrency story

The subtle part is two duplicate requests arriving at nearly the same instant, possibly on different pods. The state machine handles it cleanly:

• Both call find(key) and see nothing.
• Both call tryReserve(key, IN_PROGRESS). Redis SET NX guarantees exactly one wins (true); the other gets false.
• The winner runs the controller. The loser re-reads the record, sees IN_PROGRESS, and returns 409 with Retry-After: 1.
• The client retries after the window; by then the record is COMPLETED and the stored response is replayed.

This is a deliberate design choice: rather than blocking the duplicate (holding a thread while the first request finishes), we fail fast with a retryable conflict. Blocking is also valid and feels nicer to callers, but it ties up server threads and risks cascading timeouts under load. Fail-fast-and-retry composes better with the backoff logic the client already has.

Failure handling and the “poison key” trap

A critical subtlety: what happens when the operation fails after the key is reserved? If we leave the IN_PROGRESS record in place, every retry sees an in-flight state forever (until TTL) and the client can never succeed — a "poison key."

The filter above handles this by only persisting 2xx responses. On a 4xx/5xx or a thrown exception, the finally block calls release(key), deleting the reservation so a genuine retry starts fresh.

There is a residual risk: if the process crashes between charging the card and writing the COMPLETED record, the reservation lingers until TTL expires and the response is lost. That window is unavoidable with a non-transactional store; the mitigation is to keep the operation and the store-write as close together as possible, and to make the underlying business operation itself idempotent at the domain level (e.g., the payment provider also accepts an idempotency key). Defense in depth — the API-layer key and a domain-layer key — is the gold standard for money movement.

Key expiry

TTL is a balance:

• Too short and a legitimate slow retry (a mobile client that comes back online an hour later) misses the window and re-executes.
• Too long and Redis fills with keys that will never be retried.

24 hours is the common default and matches Stripe’s behavior. Crucially, document it: tell clients that keys are remembered for 24 hours, so they understand that reusing a key the next day will not dedupe. The TTL is refreshed on complete() in the implementation above so the replay window starts from completion rather than reservation.

When to use / pitfalls

Use idempotency keys when:

• The endpoint is a non-idempotent POST/PATCH with real side effects: payments, orders, transfers, sending notifications.
• Clients are untrusted networks (mobile, third-party integrations) where retries are inevitable.
• The cost of a duplicate effect is high — money, inventory, customer trust.

Don’t bother when:

• The operation is already naturally idempotent (PUT, DELETE) — the method contract covers you.
• The endpoint is a pure read (GET).
• The “duplicate” is harmless (e.g., recording an analytics event where a few dupes don’t matter and the overhead isn’t worth it).

Pitfalls to watch:

• Caching error responses. Never store 4xx/5xx for replay. A transient 503 cached for 24h would block all retries. Only 2xx is replay-safe.
• Fingerprint scope. Hash method + path + body. Don’t include volatile headers (timestamps, auth tokens) or the fingerprint will never match on retry. Be deliberate about whether the query string is part of identity.
• Reading the body twice. A raw HttpServletRequest body is a one-shot stream. You must wrap it (ContentCachingRequestWrapper) so both the filter and the controller can read it.
• Streaming/large responses. Capturing the full response body to replay it doesn’t suit large downloads or streamed content — exclude those endpoints.
• Trusting the client to scope keys. A buggy client that reuses one key for different operations is caught by the fingerprint mismatch returning 409, but make sure your error message points to the bug.
• Non-deterministic responses. If the response embeds a server timestamp or a freshly generated ID, the replayed response will show the original values. That’s usually correct (it reflects the one effect that happened) — just be aware that “now” in a replayed response is the original “now.”
• Clock and TTL coupling. Don’t tie business logic to the key’s TTL. The TTL is a storage-cleanup concern, not a domain invariant.

Wrapping up

Idempotency is not a feature you bolt on after the first double-charge incident — it’s an architectural property you design in from the start. The recipe is small and durable: let clients carry an Idempotency-Key, reserve it atomically, fingerprint the request to catch misuse, execute the effect exactly once, and replay the stored response for every retry. Redis gives you the atomic reservation and free expiry; an OncePerRequestFilter keeps the concern out of your controllers entirely.

Get this right and retries stop being a liability and become what they were always meant to be: the mechanism that makes your system reliable.

If this helped you reason about safe retries, follow the devdomain publication for more deep dives on distributed systems and API design. Have you implemented idempotency differently — blocking instead of fail-fast, or a relational store instead of Redis? Leave a comment and tell us what worked (�nd what bit you) in production.

Designing Idempotent REST APIs: Safe Retries and Exactly-Once EffectsThe network is not your friend. was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Spring transactions are declarative: you annotate, Spring weaves the begin/commit/rollback around your method. But “around your method” hides a proxy, a transaction manager, a thread-bound connection, and a set of rules about propagation, isolation, and rollback that you must know to use it correctly. This article goes under the hood: the proxy mechanism and its infamous self-invocation trap, every propagation level with a concrete use case, isolation levels and the anomalies they prevent, rollback rules, readOnly, and the bugs that catch even experienced engineers.

How @Transactional actually works: proxies

When you annotate a bean method with @Transactional, Spring does not modify your method's bytecode. Instead, it wraps your bean in a proxy. Callers get the proxy; the proxy starts a transaction, calls your real method, and commits or rolls back.

caller ──► [ Spring proxy ] ──► your bean
                  │  begin tx
                  │  invoke real method ─────► business logic
                  │  commit / rollback
                  ▼
              returns

Two proxy strategies:

• JDK dynamic proxies — used when your bean implements an interface. The proxy implements the same interface.
• CGLIB proxies — used when there’s no interface (Spring Boot’s default). The proxy is a runtime subclass of your class.

The transaction logic lives in a TransactionInterceptor that the proxy invokes. It asks a PlatformTransactionManager (e.g. JpaTransactionManager, DataSourceTransactionManager) to begin/commit/rollback. The active transaction's resources (the JDBC connection / JPA EntityManager) are bound to the current thread via TransactionSynchronizationManager. This thread-binding is why the same connection is reused across all DAO calls within one transactional method.

This proxy model has two enormous consequences that cause most transaction bugs.

Pitfall #1: the self-invocation trap

Because the transaction is applied by the proxy, it only takes effect when the call goes through the proxy. A call from one method to another method of the same bean does not go through the proxy — it’s a plain this.method() call on the target object. The annotation is ignored.

@Service
public class OrderService {

    public void placeOrder(Order order) {
        // BUG: this.save(...) bypasses the proxy — NO transaction starts
        save(order);
    }

    @Transactional
    public void save(Order order) {
        repository.persist(order);
        repository.persist(order.lineItems()); // not atomic with the above!
    }
}

save is annotated, but placeOrder calls it directly on this. The proxy never sees the call, so no transaction begins. If the second persist fails, the first is not rolled back.

The same trap applies to @Transactional propagation: an inner method requesting REQUIRES_NEW via self-invocation will not get a new transaction.

Fixes for self-invocation

1. Move the transactional method to another bean. The cleanest fix — the call then crosses a proxy boundary.

@Service
public class OrderService {
    private final OrderPersister persister;
    public OrderService(OrderPersister persister) { this.persister = persister; }

    public void placeOrder(Order order) {
        persister.save(order); // goes through OrderPersister's proxy
    }
}

@Service
public class OrderPersister {
    @Transactional
    public void save(Order order) { /* ... */ }
}

1. Self-inject the proxy (a code smell, but sometimes pragmatic):

@Service
public class OrderService {
    @org.springframework.beans.factory.annotation.Autowired
    private OrderService self;

    public void placeOrder(Order order) { self.save(order); }

    @Transactional
    public void save(Order order) { /* ... */ }
}

1. Annotate the public entry point instead of the inner method, so the transaction starts at the proxy boundary you actually call.

Pitfall #2: @Transactional only works on public methods

With Spring’s default proxy-based AOP, @Transactional on private, protected, or package-private methods is silently ignored — the proxy can't intercept them. Always put @Transactional on public methods. (AspectJ load-time weaving can transactionalize non-public methods, but that's a different, less common setup.)

Propagation: how transactions compose

Propagation answers: when a transactional method is called, what happens relative to any transaction already in progress? Set it with @Transactional(propagation = ...).

Propagation If a transaction exists If none exists REQUIRED (default) Join it Start a new one REQUIRES_NEW Suspend it, start a new independent one Start a new one NESTED Create a savepoint within it Start a new one SUPPORTS Join it Run non-transactionally NOT_SUPPORTED Suspend it, run non-transactionally Run non-transactionally MANDATORY Join it Throw exception NEVER Throw exception Run non-transactionally

REQUIRED — the default you’ll use 90% of the time

The caller and callee share one transaction. If either fails, the whole thing rolls back. This is almost always what you want for a business operation that should be atomic.

REQUIRES_NEW — independent commit

The current transaction is suspended, a brand-new one starts (with its own connection), commits or rolls back independently, then the original resumes. Use it when an action must persist regardless of the outer transaction’s fate — the canonical example is an audit log:

@Service
public class AuditService {
    @Transactional(propagation = Propagation.REQUIRES_NEW)
    public void record(String event) {
        auditRepository.save(new AuditEntry(event));
    }
}

Even if the outer business transaction rolls back, the audit record stays. Caution: REQUIRES_NEW holds two connections at once (outer suspended, inner active). Under load this can exhaust the connection pool and even self-deadlock if the pool is too small. Use it deliberately.

NESTED — savepoints

NESTED runs inside the existing transaction but creates a JDBC savepoint. If the nested part fails, only its work rolls back (to the savepoint); the outer transaction can continue and still commit. Unlike REQUIRES_NEW, there's only one physical transaction and one connection. It requires a transaction manager and driver that support savepoints (most do for JDBC; JPA support is limited).

@Transactional(propagation = Propagation.NESTED)
public void importRow(Row row) {
    // failure here rolls back to the savepoint, not the whole batch
}

MANDATORY / NEVER / NOT_SUPPORTED

• MANDATORY — "I must run inside someone else's transaction; if there isn't one, that's a bug." Great for low-level helpers that should never start their own transaction.
• NEVER — "I must never run in a transaction." Throws if one exists.
• NOT_SUPPORTED — suspend any transaction and run without one. Useful for long read-only reporting that shouldn't hold a transaction open.

Isolation: which concurrency anomalies you tolerate

Isolation controls what one transaction can see of another’s uncommitted (or concurrently committing) work. The SQL standard defines three anomalies and four levels that prevent progressively more of them.

The anomalies:

• Dirty read — you read another transaction’s uncommitted change. If it rolls back, you read data that never existed.
• Non-repeatable read — you read a row twice in one transaction and get different values because another transaction committed an update in between.
• Phantom read — you run the same range query twice and get different rows because another transaction inserted/deleted matching rows.

Isolation level Dirty read Non-repeatable read Phantom read READ_UNCOMMITTED Possible Possible Possible READ_COMMITTED Prevented Possible Possible REPEATABLE_READ Prevented Prevented Possible* SERIALIZABLE Prevented Prevented Prevented

* Some databases (notably MySQL InnoDB) prevent phantoms at REPEATABLE_READ via next-key locks; the standard does not require it.

@Transactional(isolation = Isolation.REPEATABLE_READ)
public BigDecimal computeStatement(String accountId) { /* ... */ }

Practical notes:

• DEFAULT (the Spring default) defers to the database. PostgreSQL and Oracle default to READ_COMMITTED; MySQL InnoDB to REPEATABLE_READ. Know your DB's default rather than assuming.
• Higher isolation costs concurrency. SERIALIZABLE can serialize transactions with extra locking or optimistic abort-and-retry (PostgreSQL uses serializable snapshot isolation that throws serialization failures you must retry). Don't reach for it reflexively.
• REQUIRES_NEW is the only way to change isolation mid-flow. A REQUIRED method that joins an existing transaction cannot change its isolation level — Spring throws if you try, because the physical transaction already started. The isolation is fixed by whichever method opened the transaction.

Rollback rules: the most surprising default

This is where teams lose data. By default, Spring rolls back only on unchecked exceptions (RuntimeException and Error). It commits if a checked exception is thrown.

@Transactional
public void process() throws IOException {
    repository.save(entity);
    throw new IOException("boom"); // checked → transaction COMMITS the save!
}

That IOException does not trigger a rollback. The save is committed. This trips up everyone at least once.

Control it explicitly:

// Roll back on a checked exception too
@Transactional(rollbackFor = IOException.class)
public void process() throws IOException { /* ... */ }

// Or never roll back on a specific runtime exception
@Transactional(noRollbackFor = BusinessValidationException.class)
public void validate() { /* ... */ }

Many teams standardize on @Transactional(rollbackFor = Exception.class) to make rollback the default for any exception, eliminating the checked/unchecked surprise.

The “marked rollback-only” trap

A second rollback gotcha: when an inner REQUIRED method catches an exception thrown by a deeper call, the transaction may already be marked rollback-only. Spring marks the shared transaction for rollback as soon as an exception escapes a transactional boundary. If the outer method then catches it and tries to commit, you get:

org.springframework.transaction.UnexpectedRollbackException:
   Transaction silently rolled back because it has been marked as rollback-only

The fix is usually to use REQUIRES_NEW for the inner operation (so its failure doesn't poison the outer transaction) or to not swallow the exception. Understand that a REQUIRED inner failure dooms the entire shared transaction even if you catch the exception.

readOnly: more than a hint

@Transactional(readOnly = true) declares the transaction won't modify data. It's not just documentation:

@Transactional(readOnly = true)
public List<Order> recentOrders(String customerId) { /* queries only */ }

What it actually does:

• Hibernate sets FlushMode.MANUAL, skipping dirty-checking and the automatic flush before queries. This is a real performance win for read-heavy methods — Hibernate doesn't scan the persistence context for changes.
• The connection is marked read-only, which some drivers and databases use to route to read replicas or to optimize.
• It signals intent: accidental writes in a read-only method may fail or be silently discarded (because there’s no flush), which surfaces design mistakes.

Use readOnly = true on all query-only service methods. It's free performance and clearer intent. But remember: it does not make the method side-effect-free — it only governs the persistence layer.

The “no open EntityManager” / LazyInitializationException angle

Because the persistence context lives for the duration of the transaction (thread-bound), accessing a lazy association after the transactional method returns throws LazyInitializationException — the session is closed. This is the most common JPA-with-Spring error.

Don’t fix it by widening transaction boundaries into the web layer (the “Open Session in View” anti-pattern, on by default in Spring Boot — consider setting spring.jpa.open-in-view=false). Instead, fetch what you need inside the transaction: use JOIN FETCH, entity graphs, or map to a DTO before returning.

@Transactional(readOnly = true)
public OrderDto load(String id) {
    Order order = repository.findWithItems(id); // JOIN FETCH inside the tx
    return OrderDto.from(order);               // map while session is open
}

Transaction timeout

Long-running transactions hold connections and locks. Bound them:

@Transactional(timeout = 10) // seconds
public void reconcile() { /* ... */ }

If the method exceeds the timeout, Spring rolls back and throws. Set sensible timeouts on anything that touches external systems or large datasets so a stuck transaction can’t pin a connection forever.

Putting it together: a worked service

import org.springframework.transaction.annotation.*;
import org.springframework.stereotype.Service;

@Service
public class CheckoutService {

    private final OrderRepository orders;
    private final InventoryService inventory;
    private final AuditService audit;

    public CheckoutService(OrderRepository orders, InventoryService inventory,
                           AuditService audit) {
        this.orders = orders;
        this.inventory = inventory;
        this.audit = audit;
    }

    @Transactional(rollbackFor = Exception.class, timeout = 15)
    public Order checkout(Cart cart) throws InsufficientStockException {
        Order order = orders.save(Order.from(cart)); // REQUIRED: shares this tx
        inventory.reserve(cart);                     // REQUIRED: same tx, atomic
        audit.record("checkout:" + order.id());      // REQUIRES_NEW: survives rollback
        return order;
    }
}

If inventory.reserve throws InsufficientStockException (checked), rollbackFor = Exception.class ensures the order save rolls back — but the audit record, written in its own REQUIRES_NEW transaction, persists for forensics.

Common pitfalls

• Self-invocation. Calling a @Transactional method from the same class bypasses the proxy. Move it to another bean or self-inject.
• Non-public methods. Proxy-based transactions ignore non-public methods entirely.
• Checked exceptions don’t roll back by default. A thrown checked exception commits your changes. Use rollbackFor = Exception.class.
• UnexpectedRollbackException. Catching an exception from an inner REQUIRED call doesn't save the shared transaction — it's already marked rollback-only. Use REQUIRES_NEW for isolation.
• Changing isolation on a joined transaction. Only the method that starts the physical transaction sets isolation; REQUIRED joins can't override it.
• REQUIRES_NEW connection exhaustion. It holds two connections; overuse plus a small pool can self-deadlock.
• LazyInitializationException. Accessing lazy fields after the transaction closed. Fetch eagerly inside the transaction or map to DTOs; prefer disabling Open Session in View.
• No timeout. Unbounded transactions pin connections and locks under failure.
• Assuming readOnly prevents writes. It changes flush behavior and connection hints; it's not a security boundary.

Wrapping up

@Transactional is declarative on the surface and mechanical underneath: a proxy, a thread-bound connection, and a transaction manager applying begin/commit/rollback. Almost every transaction bug comes from forgetting the proxy (self-invocation, non-public methods), misunderstanding rollback rules (checked exceptions commit by default), or composing propagation incorrectly (REQUIRES_NEW vs NESTED, joined-transaction isolation). Internalize the proxy model, choose propagation and isolation deliberately, make rollback explicit, and use readOnly for queries — and your data integrity stops depending on luck.

If this saved you a future incident, follow devdomain for more deep Spring internals, and drop your worst @Transactional debugging war story in the comments — the self-invocation ones are always painfully relatable.

Mastering Spring Transactions: Propagation, Isolation, and the Pitfalls@Transactional is the… was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is the central lie of in-memory database testing: you are not testing your application against the thing it runs against in production. You are testing it against an emulator that is close enough to give you confidence — right up until it isn’t.

Testcontainers fixes this by spinning up real databases, message brokers, and any other infrastructure inside throwaway Docker containers, scoped to your test lifecycle. In this article we’ll go deep: why mocks and H2 fail, the core Testcontainers programming model, a complete PostgreSQL + Spring Boot test, the magic of @ServiceConnection in Spring Boot 3.1+, container reuse for fast feedback, and how to extend the same pattern to Kafka and Redis.

The Problem with H2 and Mocks

Let’s be precise about what goes wrong, because “use a real database” is easy advice that needs justification.

H2 (and other in-memory DBs) lie about dialect. Even in “PostgreSQL compatibility mode,” H2 does not implement the full PostgreSQL feature set. Things that silently differ or break:

• JSON/JSONB columns and operators (->, ->>, @>)
• Array column types (text[], int[])
• ON CONFLICT (...) DO UPDATE upsert semantics
• Window functions, LATERAL joins, full-text search (tsvector)
• Vendor-specific functions, sequences, and RETURNING behavior
• Lock semantics, isolation levels, and SELECT ... FOR UPDATE SKIP LOCKED

Mocks test your assumptions, not reality. When you mock a repository, you assert that you think it returns a List<Order>. You never verify the SQL is valid, the mapping is correct, the migration applies, or the transaction boundary behaves. A mock-based repository test is a tautology: it passes because you told it to.

The result is a test suite that is green and worthless for the layer that talks to the database. Here’s the comparison that matters:

Concern Mocks H2 (compat mode) Testcontainers (real PG) SQL dialect fidelity None Partial Full Migrations actually run No Sometimes Yes JSONB / arrays / upserts No No Yes Isolation & locking behavior No No Yes Startup cost ~0 ms ~50 ms ~1–3 s (cached image) Requires Docker No No Yes

The startup cost is the only real tradeoff — and as we’ll see, container reuse and singleton patterns shrink it dramatically.

What Testcontainers Actually Is

Testcontainers is a Java library (with ports for many languages) that provides a programmatic API to define, start, and stop Docker containers as part of your tests. It talks to a Docker daemon (local Docker Desktop, Colima, Podman, or a remote daemon) via the Docker API.

The lifecycle is simple:

1. Before your test (class or method), Testcontainers pulls the image if needed and starts a container.
2. It waits — using a configurable wait strategy — until the container is genuinely ready (port listening, log line printed, healthcheck passing).
3. It exposes the container’s internal port on a random free host port, so parallel runs never collide.
4. After your test, it stops and removes the container.

Cleanup is guaranteed even if your JVM crashes, thanks to a companion container called Ryuk (the “reaper”) that watches the test session and kills orphaned containers.

Add the BOM and dependencies (Maven):

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.testcontainers</groupId>
      <artifactId>testcontainers-bom</artifactId>
      <version>1.20.4</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>postgresql</artifactId>
    <scope>test</scope>
  </dependency>
  <!-- Spring Boot brings its own testcontainers integration -->
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-testcontainers</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>

The Core Annotations: @Testcontainers and @Container

The JUnit 5 integration centers on two annotations from org.testcontainers.junit.jupiter:

• @Testcontainers — a class-level annotation that activates the Testcontainers JUnit 5 extension. It scans the class for @Container fields and manages their lifecycle.
• @Container — marks a container field. A static @Container field is started once before all tests in the class and stopped after all of them (container-per-class). A non-static (instance) field is started and stopped around each test method (container-per-method).

In practice you almost always want static containers: a fresh container per method is slow, and you can isolate tests via transactions or truncation instead.

Here is the minimal shape:

import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

@Testcontainers
class RawContainerTest {

    @Container
    static PostgreSQLContainer<?> postgres =
            new PostgreSQLContainer<>("postgres:16-alpine")
                    .withDatabaseName("shop")
                    .withUsername("test")
                    .withPassword("test");

    @Test
    void containerIsRunning() {
        assertThat(postgres.isRunning()).isTrue();
        String jdbcUrl = postgres.getJdbcUrl(); // jdbc:postgresql://localhost:54213/shop
        assertThat(jdbcUrl).contains("postgresql");
    }
}

Note the version pin: postgres:16-alpine. Never use latest — it makes your tests non-reproducible and can break overnight when the upstream tag moves.

A Real PostgreSQL Test with Spring Boot (the old way)

Before @ServiceConnection existed, you wired the container's connection details into Spring's Environment using @DynamicPropertySource. This is still worth understanding because it works for any property, not just well-known ones.

Suppose we have an entity, a Spring Data repository, and a Flyway migration that creates a products table with a JSONB column.

@Entity
@Table(name = "products")
public class Product {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String name;

    @Column(columnDefinition = "jsonb")
    @JdbcTypeCode(SqlTypes.JSON)
    private Map<String, Object> attributes;

    // getters / setters omitted
}

public interface ProductRepository extends JpaRepository<Product, Long> {

    @Query(value = """
            SELECT * FROM products
            WHERE attributes ->> 'color' = :color
            """, nativeQuery = true)
    List<Product> findByColor(@Param("color") String color);
}

That ->> 'color' operator is pure PostgreSQL — H2 cannot run this query. Here's the test:

import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.PostgreSQLContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;

import static org.assertj.core.api.Assertions.assertThat;

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Testcontainers
class ProductRepositoryTest {

    @Container
    static PostgreSQLContainer<?> postgres =
            new PostgreSQLContainer<>("postgres:16-alpine");

    @DynamicPropertySource
    static void datasourceProps(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
    }

    @Autowired
    private ProductRepository repository;

    @Test
    void findsProductsByJsonbAttribute() {
        Product red = new Product();
        red.setName("Red Mug");
        red.setAttributes(Map.of("color", "red", "size", "M"));

        Product blue = new Product();
        blue.setName("Blue Mug");
        blue.setAttributes(Map.of("color", "blue"));

        repository.saveAll(List.of(red, blue));

        List<Product> result = repository.findByColor("red");

        assertThat(result).hasSize(1);
        assertThat(result.get(0).getName()).isEqualTo("Red Mug");
    }
}

Two critical details:

• @AutoConfigureTestDatabase(replace = Replace.NONE) — by default @DataJpaTest tries to swap in an embedded database. This annotation tells it to keep the real datasource. Forget this and Spring quietly replaces your PostgreSQL with H2 and the JSONB query explodes.
• The lambdas in @DynamicPropertySource are suppliers (postgres::getJdbcUrl), evaluated lazily after the container starts. Pass the value eagerly and you'll capture a null URL.

@ServiceConnection: The Spring Boot 3.1+ Way

Spring Boot 3.1 introduced @ServiceConnection, which eliminates the @DynamicPropertySource boilerplate entirely. Spring recognizes the container type (PostgreSQLContainer, KafkaContainer, GenericContainer with a hint, etc.) and auto-configures the matching connection details.

The same test becomes:

import org.springframework.boot.testcontainers.service.connection.ServiceConnection;

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Testcontainers
class ProductRepositoryServiceConnectionTest {

    @Container
    @ServiceConnection
    static PostgreSQLContainer<?> postgres =
            new PostgreSQLContainer<>("postgres:16-alpine");

    @Autowired
    private ProductRepository repository;

    @Test
    void contextLoadsAgainstRealPostgres() {
        assertThat(repository.count()).isZero();
    }
}

No URL wiring. No username/password plumbing. @ServiceConnection reads the container metadata and registers a JdbcConnectionDetails bean. It supports PostgreSQL, MySQL, MariaDB, MongoDB, Redis, Kafka, RabbitMQ, Elasticsearch, Cassandra, and more out of the box.

Sharing containers across many test classes

For an integration suite, you don’t want each test class to start its own database. Define a base class with a shared static container and have your tests extend it. Spring’s test context caching keeps the application context warm too.

@SpringBootTest
@Testcontainers
public abstract class AbstractIntegrationTest {

    @Container
    @ServiceConnection
    static PostgreSQLContainer<?> postgres =
            new PostgreSQLContainer<>("postgres:16-alpine");
}

Because the field is static and lives on the shared base class, the container is started exactly once across all subclasses in the same JVM run.

The @TestConfiguration / @Bean style

There’s a third, increasingly popular idiom: define containers as Spring beans annotated with @ServiceConnection. This integrates with Spring Boot's Testcontainers support and even lets you run your main() application locally against containers via SpringApplication.from(...).with(...).

@TestConfiguration(proxyBeanMethods = false)
public class ContainersConfig {

    @Bean
    @ServiceConnection
    PostgreSQLContainer<?> postgresContainer() {
        return new PostgreSQLContainer<>("postgres:16-alpine");
    }
}

@SpringBootTest
@Import(ContainersConfig.class)
class OrderServiceIntegrationTest {
    // containers managed as beans, started with the context
}

Reusing Containers for Fast Feedback

The container start cost is paid once per test run. For tight local TDD loops you can do better with reusable containers, a Testcontainers feature that keeps the container alive between test runs on your machine.

Enable it on the container and opt in globally:

@Container
static PostgreSQLContainer<?> postgres =
        new PostgreSQLContainer<>("postgres:16-alpine")
                .withReuse(true);

Then create ~/.testcontainers.properties with:

testcontainers.reuse.enable=true

With reuse on, Testcontainers computes a hash of the container configuration; if a matching container is already running, it attaches to it instead of starting a new one. Ryuk is skipped for reusable containers (otherwise the reaper would kill them). The tradeoff: you are responsible for state — reuse a database long enough and old test data accumulates. Use transactional rollback or TRUNCATE between tests rather than relying on a fresh container.

A note on CI: reuse is a developer-machine optimization. On CI, each run is a fresh agent, so reuse buys nothing; rely on the Docker layer cache for the image pull instead. Don’t enable reuse expecting CI speedups.

Singleton container pattern

If you want a single container shared across the entire test suite without inheritance, start it manually in a static initializer and never stop it (the JVM exit cleans it up via Ryuk):

public abstract class DatabaseSuite {

    static final PostgreSQLContainer<?> POSTGRES =
            new PostgreSQLContainer<>("postgres:16-alpine");

    static {
        POSTGRES.start(); // started once, on first class load
    }

    @DynamicPropertySource
    static void props(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", POSTGRES::getJdbcUrl);
        registry.add("spring.datasource.username", POSTGRES::getUsername);
        registry.add("spring.datasource.password", POSTGRES::getPassword);
    }
}

Notice there is no @Container and no @Testcontainers here — we manage the lifecycle ourselves precisely so JUnit doesn't stop it after each class.

Testing Kafka and Redis Too

The same model extends to any infrastructure. The point of Testcontainers is that all your external dependencies become first-class, real test fixtures.

Kafka

Use org.testcontainers:kafka with the org.testcontainers.kafka.KafkaContainer (the modern Apache-Kafka-native image, no ZooKeeper):

import org.testcontainers.kafka.KafkaContainer;

@SpringBootTest
@Testcontainers
class OrderEventsTest {

    @Container
    @ServiceConnection
    static KafkaContainer kafka =
            new KafkaContainer("apache/kafka-native:3.8.0");

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    @Autowired
    private OrderEventListener listener; // accumulates received events

    @Test
    void publishesAndConsumesOrderEvent() {
        kafkaTemplate.send("orders", "order-1", "{\"id\":1}");

        await().atMost(Duration.ofSeconds(10))
               .untilAsserted(() ->
                   assertThat(listener.received()).contains("order-1"));
    }
}

@ServiceConnection auto-configures spring.kafka.bootstrap-servers. Use Awaitility (org.awaitility:awaitility) for the async assertion — never Thread.sleep.

Redis

Redis has no dedicated module, so use a GenericContainer and tell @ServiceConnection what it is via the name attribute (matching the redis connection detail), or wire properties manually:

import org.testcontainers.containers.GenericContainer;

@SpringBootTest
@Testcontainers
class CacheIntegrationTest {

    @Container
    @ServiceConnection(name = "redis")
    static GenericContainer<?> redis =
            new GenericContainer<>("redis:7-alpine").withExposedPorts(6379);

    @Autowired
    private StringRedisTemplate redisTemplate;

    @Test
    void readsBackWhatItWrote() {
        redisTemplate.opsForValue().set("k", "v");
        assertThat(redisTemplate.opsForValue().get("k")).isEqualTo("v");
    }
}

The name = "redis" hint matches Spring Boot's RedisConnectionDetails factory. For containers Spring doesn't recognize by image, this hint is how you bridge the gap.

Pitfalls

Real-world Testcontainers usage has sharp edges. Here are the ones that bite teams repeatedly.

• Instance vs static @Container. A non-static @Container field restarts the container for every test method. On a class with 30 tests, that's 30 PostgreSQL boots. Make it static unless you have a specific isolation reason.
• Forgetting @AutoConfigureTestDatabase(replace = NONE). With @DataJpaTest, Spring replaces your datasource with an embedded one by default, silently bypassing your container. Your "PostgreSQL test" secretly runs on H2.
• Using :latest image tags. Non-reproducible builds. A new upstream release can break your suite with zero code changes. Pin exact versions.
• No Docker in CI. Testcontainers needs a Docker daemon. On CI you must provide one (Docker-in-Docker, a mounted socket, or a Testcontainers Cloud token). A suite that’s green locally and red in CI is usually a missing daemon.
• Thread.sleep for async assertions. Kafka/Redis tests are inherently asynchronous. Sleeping is flaky and slow. Use Awaitility's await().untilAsserted(...).
• State leaking between tests. With shared/reused containers, data persists across tests. Wrap each test in a transaction that rolls back (@DataJpaTest does this automatically), or TRUNCATE in a @BeforeEach/@AfterEach. Order-dependent tests are a symptom of leaked state.
• Weak wait strategies. A container reporting “running” isn’t the same as “ready.” The bundled modules ship sensible wait strategies, but for a raw GenericContainer add .waitingFor(Wait.forLogMessage(...)) or .waitingFor(Wait.forListeningPort()) so you don't connect to a half-booted service.
• Ryuk disabled but containers leak. Some locked-down CI environments disable Ryuk (TESTCONTAINERS_RYUK_DISABLED=true). If you do this, you own cleanup. Orphaned containers will pile up and exhaust disk/ports.
• Pulling images on the critical path. First run pulls images and is slow. Pre-pull in CI (docker pull postgres:16-alpine) before the test stage, or warm the cache, so the timing isn't attributed to flaky tests.

Wrapping Up

The argument for Testcontainers is simple: test against what you ship. H2 and mocks give you a green suite that doesn’t actually exercise your SQL, your migrations, your serialization, or your locking. Testcontainers gives you a real PostgreSQL, a real Kafka, a real Redis — disposable, isolated, and fast enough with image caching and reuse.

The progression to adopt: start with @Testcontainers + a static @Container, switch to @ServiceConnection to delete the wiring boilerplate, share containers via a base class, and turn on local reuse for tight feedback loops. Once the pattern clicks, you'll wonder how you ever trusted a green H2 build.

Found this useful? Follow devdomain for more deep-dive Java and testing articles, and drop a comment with the trickiest integration-testing problem you’ve hit — real-world war stories make the best follow-up posts.

Integration Testing with Testcontainers: Real Databases in Your Test SuiteYou wrote a repository… was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.

NIO has a reputation for being fiddly, and the ByteBuffer API in particular trips people up. This article builds the model carefully: why blocking I/O doesn't scale, exactly how a buffer's position/limit/capacity work, what channels and selectors give you, and how it all assembles into the reactor pattern that powers Netty and the reactive stack.

The Limits of Classic Blocking I/O

The traditional java.io model is stream-based and blocking. A read blocks the calling thread until data arrives:

// Classic blocking server — one thread per connection
try (ServerSocket server = new ServerSocket(8080)) {
    while (true) {
        Socket socket = server.accept();        // blocks until a client connects
        new Thread(() -> handle(socket)).start(); // dedicate a thread to it
    }
}

static void handle(Socket socket) {
    try (var in = socket.getInputStream()) {
        byte[] buf = new byte[1024];
        int n = in.read(buf);   // BLOCKS until bytes arrive — thread is stuck
        // ... process ...
    } catch (IOException e) { /* ... */ }
}

This is beautifully simple but doesn’t scale, for two reasons:

• Thread cost. Each thread costs ~512 KB–1 MB of stack plus kernel scheduling overhead. Ten thousand connections means ten thousand threads — gigabytes of stack and brutal context-switching.
• Idle waste. Most connections are idle most of the time (think chat, keep-alive HTTP, websockets). A blocked thread waiting on a quiet socket does nothing but consume resources.

The fundamental problem: blocking ties a thread to a connection for the connection’s entire lifetime, whether or not data is flowing. NIO decouples them: one thread can watch many connections and act only on the ones that are ready.

Buffers: The Heart of NIO

In NIO you don’t read/write bytes directly — you move them through a Buffer. A buffer is a fixed-size container backed by an array (or off-heap memory). The most important is ByteBuffer; there are typed variants (IntBuffer, CharBuffer, etc.).

A buffer tracks three indices, and understanding them is the whole game:

Property Meaning Invariant capacity total size, fixed at creation constant position index of the next byte to read/write 0 <= position <= limit limit first index you must NOT touch position <= limit <= capacity

There’s also mark, an optional remembered position you can reset() to.

The fill / drain cycle

A buffer is always in one of two phases — write mode (you put data in) or read mode (you take data out). You switch between them with flip(). Picture an empty 8-byte buffer:

After allocate(8) — write mode:
  pos=0                          limit=cap=8
   v                              v
  [ . . . . . . . . ]
   ^ next write goes here

After writing "Hi" (2 bytes):
       pos=2                     limit=cap=8
        v                         v
  [ H i . . . . . . ]

After flip() — read mode (limit=old pos, pos=0):
  pos=0   limit=2
   v       v
  [ H i . . . . . . ]
   ^ next read         ^ stop reading here

After reading both bytes:
       pos=2,limit=2
        v
  [ H i . . . . . . ]

After clear() — back to write mode (pos=0, limit=cap):
  pos=0                          limit=cap=8

The canonical loop:

ByteBuffer buf = ByteBuffer.allocate(1024);

int bytesRead = channel.read(buf);   // channel WRITES into buffer (fill)
buf.flip();                          // switch to read mode
while (buf.hasRemaining()) {
    byte b = buf.get();              // drain
    // ... process b ...
}
buf.clear();                         // reset for the next fill

The four buffer “verbs” you must know cold:

• flip() — finish writing, prepare to read: limit = position; position = 0.
• clear() — discard contents, prepare to write again: position = 0; limit = capacity. (Doesn't erase data; just resets indices.)
• rewind() — re-read from the start: position = 0, limit unchanged.
• compact() — for partial reads: move unread bytes to the front, then position after them, ready to write more.

Heap vs direct buffers

ByteBuffer heap   = ByteBuffer.allocate(1024);       // lives in JVM heap
ByteBuffer direct = ByteBuffer.allocateDirect(1024); // off-heap, native memory

A direct buffer lives outside the Java heap. The OS can perform I/O on it without an extra copy to/from the heap, which makes large or frequent transfers faster. The trade-offs: allocation is more expensive, it’s not subject to normal GC (freed via a Cleaner), and it costs native memory. Rule of thumb: use direct buffers for long-lived, large, I/O-heavy buffers; heap buffers for small, short-lived ones.

Channels: Bidirectional Conduits

A Channel represents an open connection to something that does I/O — a file, a socket, a pipe. Unlike streams, channels are bidirectional (a socket channel both reads and writes) and always operate through buffers.

Key channel types:

• FileChannel — file I/O, memory-mapping, zero-copy transfers.
• SocketChannel — a TCP connection (client side).
• ServerSocketChannel — accepts incoming TCP connections.
• DatagramChannel — UDP.

File I/O with a channel

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class FileCopy {
    public static void main(String[] args) throws IOException {
        Path src = Path.of("input.txt");
        Path dst = Path.of("output.txt");

        try (FileChannel in  = FileChannel.open(src, StandardOpenOption.READ);
             FileChannel out = FileChannel.open(dst,
                     StandardOpenOption.CREATE,
                     StandardOpenOption.WRITE,
                     StandardOpenOption.TRUNCATE_EXISTING)) {

            ByteBuffer buf = ByteBuffer.allocateDirect(8192);
            while (in.read(buf) != -1) {   // fill
                buf.flip();                // switch to read
                out.write(buf);            // drain into the other channel
                buf.compact();             // keep any unwritten bytes
            }
        }
    }
}

Zero-copy transfer

FileChannel can transfer bytes directly between channels in the kernel, skipping user space entirely — the basis of efficient file serving:

try (FileChannel in = FileChannel.open(Path.of("big.iso"), StandardOpenOption.READ)) {
    // hands the bytes straight to the socket without copying through the JVM
    in.transferTo(0, in.size(), socketChannel);
}

Non-Blocking Mode and Selectors

This is where NIO earns its keep. A SelectableChannel (sockets, not files) can be put into non-blocking mode:

SocketChannel ch = SocketChannel.open();
ch.configureBlocking(false);  // reads/writes return immediately

In non-blocking mode, read() returns whatever bytes are available right now (possibly zero) and never blocks. By itself that would force you to busy-poll. The solution is the Selector — an object that lets one thread register many channels and ask the OS, "which of these are ready for I/O?" This maps directly to the kernel's epoll/kqueue mechanisms.

You register a channel with a selector for a set of interest operations:

• OP_ACCEPT — a server socket has an incoming connection
• OP_CONNECT — a client connect completed
• OP_READ — data is available to read
• OP_WRITE — the channel can accept writes

selector.select() blocks until at least one registered channel is ready, then hands you the ready set. One thread, many connections.

A complete non-blocking echo server

import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.*;
import java.util.Iterator;

public class NioEchoServer {

    public static void main(String[] args) throws IOException {
        Selector selector = Selector.open();

        ServerSocketChannel server = ServerSocketChannel.open();
        server.bind(new InetSocketAddress(8080));
        server.configureBlocking(false);
        server.register(selector, SelectionKey.OP_ACCEPT);

        System.out.println("Echo server listening on :8080");

        while (true) {
            selector.select();                     // blocks until something is ready
            Iterator<SelectionKey> it = selector.selectedKeys().iterator();

            while (it.hasNext()) {
                SelectionKey key = it.next();
                it.remove();                       // MUST remove — selector won't

                try {
                    if (key.isAcceptable()) {
                        accept(server, selector);
                    } else if (key.isReadable()) {
                        echo(key);
                    }
                } catch (IOException e) {
                    key.cancel();
                    key.channel().close();
                }
            }
        }
    }

    private static void accept(ServerSocketChannel server, Selector selector)
            throws IOException {
        SocketChannel client = server.accept();    // never null here
        client.configureBlocking(false);
        // attach a per-connection buffer so each client has its own state
        client.register(selector, SelectionKey.OP_READ, ByteBuffer.allocate(1024));
        System.out.println("Accepted " + client.getRemoteAddress());
    }

    private static void echo(SelectionKey key) throws IOException {
        SocketChannel client = (SocketChannel) key.channel();
        ByteBuffer buf = (ByteBuffer) key.attachment();

        int n = client.read(buf);                  // non-blocking read
        if (n == -1) {                             // client closed
            key.cancel();
            client.close();
            return;
        }
        buf.flip();
        client.write(buf);                         // echo back
        buf.compact();                             // retain unwritten bytes
    }
}

The whole server runs on one thread yet handles many simultaneous clients. Note the critical detail: you must call it.remove() on each selected key. The selector adds keys to the selected set but never removes them; forget this and you'll reprocess stale keys forever.

The Reactor Pattern

That event loop is the reactor pattern, a foundational design for scalable network servers:

┌──────────────────────────────────────────┐
        │              Reactor (1 thread)           │
        │                                           │
        │   ┌─────────────┐    selector.select()    │
        │   │  Selector   │◄──────────┐             │
        │   └─────────────┘           │ ready keys  │
        │         │ dispatch          │             │
        │         ▼                   │             │
        │   ┌──────────┬──────────┬──────────┐      │
        │   │ ACCEPT   │  READ    │  WRITE   │      │
        │   │ handler  │ handler  │ handler  │      │
        │   └──────────┴──────────┴──────────┘      │
        └──────────────────────────────────────────┘
                 │            │            │
              client A     client B     client C

The reactor blocks once in select(), then dispatches ready events to handlers. Real systems extend this with a multi-reactor design: one acceptor thread plus a pool of worker reactors (often one per CPU core), each owning a selector and a slice of the connections. CPU-bound or blocking work is offloaded to a separate thread pool so the event loop never stalls.

When NIO Actually Matters

Be honest about whether you need it. NIO’s complexity pays off when:

• You have many concurrent, mostly-idle connections (chat, websockets, IoT, long-polling, push). This is the C10k problem NIO was built for.
• You’re building a protocol server or proxy where connection count dwarfs CPU work per request.
• You need zero-copy file serving or memory-mapped files (FileChannel.map).

It’s usually not worth it when:

• Connections are few or short-lived — thread-per-connection is simpler and plenty fast.
• Each request is CPU-heavy — you’re bound by compute, not by waiting on I/O.

Modern alternatives also change the calculus. Virtual threads (Project Loom, Java 21) let you write simple blocking-style code that scales to millions of “threads,” reclaiming much of NIO’s benefit without the buffer gymnastics. But virtual threads run on top of the JDK’s NIO machinery under the hood — so understanding NIO still matters.

Relationship to Netty and Reactive Stacks

Almost nobody writes raw selector loops in production. Netty wraps NIO in a battle-tested, ergonomic framework: it manages the reactor (its EventLoopGroup), pools direct buffers (ByteBuf with reference counting and pooling), and gives you a ChannelPipeline of composable handlers for encoding, decoding, TLS, and business logic. Netty fixes the rough edges of ByteBuffer (no manual flip/clip) and gracefully handles partial reads, backpressure, and write batching.

On top of Netty sit the reactive frameworks — Spring WebFlux, Project Reactor, Vert.x. They expose the non-blocking event loop through Mono/Flux (or futures) so you compose asynchronous pipelines without ever touching a selector. The chain looks like this:

Your code (WebFlux / Vert.x)
        │  reactive streams, backpressure
        ▼
   Netty (event loop, ByteBuf, pipeline)
        │
        ▼
   Java NIO (Selector, Channel, ByteBuffer)
        │
        ▼
   OS (epoll / kqueue / IOCP)

Understanding NIO is understanding the bottom of that stack — why event loops must never block, why buffers are pooled, why backpressure exists.

Pitfalls and Gotchas

• Forgetting flip(). Writing into a buffer then trying to read without flipping reads garbage (you read from the wrong position). The fill→flip()→drain rhythm is non-negotiable.
• Forgetting selectedKeys().remove(). The selector never clears the selected set; omitting it.remove() causes endless reprocessing of stale keys.
• clear() doesn't erase data. It only resets the indices. The old bytes are still there until overwritten — never assume a "cleared" buffer is zeroed.
• Blocking inside the event loop. A single slow database call or Thread.sleep on the reactor thread stalls every connection it manages. Offload blocking work to a separate executor.
• Partial reads and writes. Non-blocking read()/write() transfer as much as possible right now, not everything. Use compact() and re-register OP_WRITE to drain the rest; never assume one call moved all the bytes.
• Leaking direct buffers. They’re freed by a Cleaner, not promptly by clear(). Allocating many short-lived direct buffers causes native-memory bloat and OutOfMemoryError: Direct buffer memory. Pool them or use heap buffers for transient data.
• Reusing a buffer across connections without isolation. Each connection needs its own buffer state — attach a buffer per SelectionKey (as in the example) rather than sharing one.
• Not handling read() == -1. A return of -1 means the peer closed the connection; you must cancel the key and close the channel, or you'll spin on a dead socket.

Wrapping Up

Java NIO replaces the thread-per-connection model with an event-driven one: buffers shuttle bytes, channels are the bidirectional conduits, and selectors let a single thread watch thousands of connections and act only on the ready ones — the reactor pattern.

Keep these anchors:

• Master the ByteBuffer lifecycle — position/limit/capacity and the fill→flip()→drain→clear() cycle.
• Non-blocking mode + a Selector is what makes one thread scale to many idle connections.
• You’ll usually consume NIO through Netty or a reactive framework — but those just wrap exactly the machinery shown here.

Reach for NIO (or its descendants) when connection count, not CPU, is your bottleneck. For everything else, simpler blocking code — or virtual threads — may serve you better.

If this clarified NIO for you, follow devdomain for more JVM and networking deep dives. Building something on Netty or virtual threads? Share your war stories in the comments.

Java NIO: Channels, Buffers, and Non-Blocking I/OIf you’ve ever written a server that spins up a… was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Eighteen months later you have forty services, four environments, and a database password copy-pasted into nineteen of them. Someone rotates that password on a Friday. You spend the weekend grepping YAML.

This is config sprawl, and it is one of the quietest ways a microservices platform rots. Spring Cloud Config exists to fix it.

The problem we are actually solving

In a fleet of N services across M environments, naive configuration creates N × M files that all drift independently. The symptoms are predictable:

• No single source of truth. The “real” value of feature.x.enabled lives in whichever pod last restarted.
• No audit trail. “Who changed the connection pool size and when?” has no answer. There is no diff, no blame, no review.
• Redeploy to change a value. Tweaking a timeout means a new image build, a CI pipeline run, and a rolling restart. Configuration and code share a release cadence they should not.
• Secrets scattered everywhere. Plaintext credentials end up baked into images, committed to repos, and pasted into Slack.

The goal is a configuration control plane: one place that is versioned, reviewable, environment-aware, secure, and changeable at runtime. Spring Cloud Config gives you exactly that, with Git as the storage and history engine.

Architecture at a glance

+------------------------------+
                       |        Git repository        |
                       |  config-repo/                |
                       |    application.yml           |
                       |    order-service.yml         |
                       |    order-service-prod.yml    |
                       |    payment-service-dev.yml   |
                       |  (branches = labels)         |
                       +---------------+--------------+
                                       | clone / pull
                                       v
                       +------------------------------+
                       |     Config Server :8888      |
                       |   @EnableConfigServer        |
                       |   /{app}/{profile}/{label}   |
                       |   /encrypt  /decrypt         |
                       +------+---------------+-------+
                              |               |
              spring.config.import           |
              (on startup + on refresh)      |
                              |               |
        +---------------------+----+   +------+-------------------+
        |   order-service          |   |   payment-service        |
        |   @RefreshScope beans     |   |   @RefreshScope beans     |
        |   /actuator/refresh       |   |   /actuator/refresh       |
        |   /actuator/busrefresh    |   |   /actuator/busrefresh    |
        +-------------+-------------+   +------------+-------------+
                      ^                              ^
                      |                              |
                      +-------------+----------------+
                                    |
                          +---------+----------+
                          |   Spring Cloud Bus  |
                          |   (RabbitMQ / Kafka) |
                          |  fan-out RefreshEvent |
                          +----------------------+

The flow: services pull config from the server at startup. The server reads it from Git. When config changes, you can fan out a single busrefresh and every subscribed client reloads without a restart.

Prerequisites

Spring Boot 3.x requires Java 17+. The examples below target the Spring Cloud 2023.x / 2024.x release train, which is the line that pairs with Spring Boot 3.2–3.3.

Pin the train with a BOM so every Spring Cloud artifact lines up:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-dependencies</artifactId>
      <version>2023.0.3</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

The Config Server

The server is a thin Spring Boot app that serves configuration over HTTP. Its dependency:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-config-server</artifactId>
</dependency>

Enable it with one annotation:

package com.devdomain.configserver;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.config.server.EnableConfigServer;

@EnableConfigServer
@SpringBootApplication
public class ConfigServerApplication {

    public static void main(String[] args) {
        SpringApplication.run(ConfigServerApplication.class, args);
    }
}

Point it at a Git repo in application.yml:

server:
  port: 8888

spring:
  application:
    name: config-server
  cloud:
    config:
      server:
        git:
          uri: https://github.com/acme/config-repo.git
          default-label: main
          clone-on-start: true
          timeout: 10
          # Only scan these folders for {application}-{profile}.yml files
          search-paths:
            - services/{application}
            - shared

A few production-relevant knobs:

• clone-on-start: true clones the repo at boot so the first client request is fast and you fail loud at startup if the repo is unreachable.
• default-label is the branch used when a client does not request one. main for most teams.
• search-paths lets you organize the repo into subfolders. The {application} placeholder is expanded per request, so services/order-service/order-service-prod.yml resolves correctly.

How the server maps requests to files

Spring Cloud Config exposes a small, well-defined REST API. The canonical endpoint is:

/{application}/{profile}[/{label}]

• application = the requesting service's spring.application.name.
• profile = the active Spring profile(s), e.g. dev, prod. Comma-separated for multiple.
• label = a Git ref: a branch, tag, or commit. Optional; defaults to default-label.

The server resolves files in the repo using the {application}-{profile}.yml (or .properties) naming convention, and layers them in a defined order:

Repo file When it applies Precedence (high → low) order-service-prod.yml app = order-service, profile = prod Highest order-service.yml app = order-service, any profile Middle application-prod.yml any app, profile = prod Lower application.yml global defaults for every app Lowest

More specific wins. A property in order-service-prod.yml overrides the same key in application.yml. This lets you keep shared defaults in application.yml and override only what differs per service and environment.

You can hit the API directly to debug:

# order-service, prod profile, main branch
curl http://localhost:8888/order-service/prod

# pin to a specific Git tag (label)
curl http://localhost:8888/order-service/prod/v2.4.0

# raw rendered YAML as a service would consume it
curl http://localhost:8888/order-service-prod.yml

The JSON response includes a propertySources array, ordered exactly as the server layered them — invaluable when a value is not what you expect.

The client side

Each microservice becomes a Config client with:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Goodbye bootstrap.yml, hello spring.config.import

If you learned Spring Cloud before 2020, you remember bootstrap.yml and a separate "bootstrap context" that ran before the main application context to fetch remote config. That is legacy.

Since Spring Boot 2.4 (and unchanged in 3.x), the bootstrap context is disabled by default. The modern, recommended approach is the unified spring.config.import mechanism, which loads remote config as a first-class part of the normal application.yml processing:

spring:
  application:
    name: order-service
  profiles:
    active: prod
  config:
    import: "optional:configserver:http://localhost:8888"
  cloud:
    config:
      # request a specific Git label; defaults to server's default-label
      label: main
      # how the client behaves if the server is down at startup
      fail-fast: true
      retry:
        max-attempts: 6
        initial-interval: 1000
        max-interval: 2000
        multiplier: 1.1

Two details that bite people:

• The optional: prefix means "do not crash at startup if the config server is unreachable." Drop optional: (or set fail-fast: true) when config is mandatory and you would rather fail loudly than boot with defaults. The two are subtly different: optional: controls whether a missing import is fatal; fail-fast controls whether connection/retry exhaustion is fatal. In production you usually want config to be mandatory.
• You only need spring-cloud-starter-bootstrap if you are deliberately keeping the old bootstrap flow alive (or pulling in a library that still requires it). For new services, do not add it.

Consuming config in code

Inject values like any other property. The cleanest pattern is @ConfigurationProperties, which binds a whole tree and refreshes cleanly:

package com.devdomain.order.config;

import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "order")
public class OrderProperties {

    private int maxItemsPerOrder = 50;
    private Pricing pricing = new Pricing();

    public int getMaxItemsPerOrder() { return maxItemsPerOrder; }
    public void setMaxItemsPerOrder(int v) { this.maxItemsPerOrder = v; }
    public Pricing getPricing() { return pricing; }
    public void setPricing(Pricing p) { this.pricing = p; }

    public static class Pricing {
        private double taxRate = 0.0;
        public double getTaxRate() { return taxRate; }
        public void setTaxRate(double v) { this.taxRate = v; }
    }
}

Backed by this in order-service-prod.yml:

order:
  max-items-per-order: 200
  pricing:
    tax-rate: 0.21

Enable it on a config class:

package com.devdomain.order.config;

import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableConfigurationProperties(OrderProperties.class)
public class OrderConfig {
}

Refreshing config at runtime

The headline feature: change a value in Git, and update running services without a redeploy.

@RefreshScope and /actuator/refresh

Expose the refresh endpoint:

management:
  endpoints:
    web:
      exposure:
        include: refresh,busrefresh,health,info

Annotate beans whose state depends on config that you want to be re-read:

package com.devdomain.order.service;

import com.devdomain.order.config.OrderProperties;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.context.config.annotation.RefreshScope;
import org.springframework.stereotype.Service;

@RefreshScope
@Service
public class PricingService {

    private final OrderProperties props;

    @Value("${order.pricing.surcharge:0.0}")
    private double surcharge;

    public PricingService(OrderProperties props) {
        this.props = props;
    }

    public double quote(double base) {
        return base * (1 + props.getPricing().getTaxRate()) + surcharge;
    }
}

How @RefreshScope actually works: it wraps the bean in a proxy. When a RefreshEvent fires, the cached instance is discarded and recreated lazily on the next call — picking up new config in the process.

Trigger it for a single instance:

curl -X POST http://localhost:9001/actuator/refresh

The response lists the property keys that changed:

["order.pricing.tax-rate","order.pricing.surcharge"]

@ConfigurationProperties beans are refreshed automatically by the refresh machinery — you do not even need @RefreshScope on them. That is a strong reason to prefer them over scattered @Value fields.

Spring Cloud Bus and /actuator/busrefresh — fan-out

Calling /actuator/refresh on one pod is fine for a single instance. With dozens of replicas across services, you do not want to script a loop over every instance.

Spring Cloud Bus connects all instances to a shared message broker (RabbitMQ or Kafka) and broadcasts a single RefreshRemoteApplicationEvent. Hit /actuator/busrefresh on any one instance and every subscribed instance refreshes.

Add the AMQP bus starter to each client (and optionally the server, so it can originate the event):

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-bus-amqp</artifactId>
</dependency>

Point at the broker:

spring:
  rabbitmq:
    host: rabbitmq.internal
    port: 5672
    username: bus
    password: '{cipher}AQB7c2f...'   # encrypted, see below

Trigger a fleet-wide refresh:

# any single instance broadcasts to all
curl -X POST http://localhost:9001/actuator/busrefresh

You can even scope it to a subset by destination:

``bbash

only order-service instances refresh

curl -X POST “http://localhost:9001/actuator/busrefresh/order-service:**"

The clean production pattern: a Git push triggers a webhook → the Config Server's `/monitor` endpoint (from `spring-cloud-config-monitor`) publishes the bus event → all affected services refresh. No human runs curl at all.

git push → webhook → Config Server /monitor → Bus event → all clients refresh

## Encrypting secrets

Plaintext secrets in a Git repo are a non-starter. Spring Cloud Config encrypts values at rest in the repo and decrypts them when serving config. Encrypted values carry a **`{cipher}`** prefix.

### Symmetric vs asymmetric

| Aspect            | Symmetric (`encrypt.key`)            | Asymmetric (RSA keystore)                 |
|-------------------|--------------------------------------|-------------------------------------------|
| Key material      | One shared secret                    | Public/private key pair                    |
| Encrypt           | Needs the shared key                 | Needs only the public key                  |
| Decrypt           | Needs the shared key                 | Needs the private key                      |
| Rotation          | Re-encrypt everything at once         | Can hand out public key freely             |
| Setup complexity  | Trivial                              | Requires a keystore (JKS/PKCS12)           |
| Best for          | Small teams, getting started         | Larger orgs, separation of duties          |

**Symmetric** is the quickest start. Set a key on the server (via environment variable, never hard-coded):

```yaml
encrypt:
  key: ${CONFIG_ENCRYPT_KEY}

Asymmetric uses a keystore. Generate and configure:

keytool -genkeypair -alias configkey -keyalg RSA -keysize 2048 \
  -dname "CN=Config Server,OU=Platform,O=Acme,L=Lisbon,C=PT" \
  -keystore config-server.jks -storepass changeit -keypass changeit

encrypt:
  key-store:
    location: file:/etc/config-server/config-server.jks
    password: ${KEYSTORE_PASSWORD}
    alias: configkey
    secret: ${KEY_PASSWORD}

/encrypt and /decrypt endpoints

With a key configured, the server exposes encryption endpoints:

# encrypt a secret
curl -X POST --data-urlencode "s3cr3t-db-password" \
  http://localhost:8888/encrypt
# -> 682bc583f4641835fa2db009355293665d2647dade3375c0ee201de2a49f7bda

# sanity-check by decrypting
curl -X POST --data-urlencode "682bc583f4641835fa2db009355293665d2647dade3375c0ee201de2a49f7bda" \
  http://localhost:8888/decrypt
# -> s3cr3t-db-password

Take the ciphertext, prefix it with {cipher}, and commit that to Git:

spring:
  datasource:
    url: jdbc:postgresql://db.internal:5432/orders
    username: orders_app
    password: '{cipher}682bc583f4641835fa2db009355293665d2647dade3375c0ee201de2a49f7bda'

Quote the value in YAML — the leading { otherwise reads as a JSON flow map. By default the server decrypts before serving, so clients receive plaintext over the (TLS-secured) wire. If you would rather ship ciphertext to clients and decrypt there, set spring.cloud.config.server.encrypt.enabled: false on the server and give clients the key.

Profiles per environment

Profiles are how one repo serves every environment. A client declares its active profile:

``Fyaml spring: profiles: active: prod

…and the server layers `application.yml` → `application-prod.yml` → `order-service.yml` → `order-service-prod.yml`. Keep environment-invariant settings (logging format, JSON serialization rules) in the unprofiled files and let profiled files carry only the deltas (URLs, pool sizes, feature flags).

A practical repo layout:

config-repo/ application.yml # global defaults, all services application-prod.yml # global prod overrides order-service.yml # order defaults order-service-dev.yml order-service-prod.yml payment-service.yml payment-service-prod.yml

Profiles also map cleanly onto Git **labels**: a long-lived `main` branch for stable config, short-lived branches for proposed changes that go through pull-request review, and tags (`v2.4.0`) for pinning a service to a known-good config snapshot during an incident.

## Pitfalls

Centralized config removes one class of pain and quietly introduces another. The ones that actually cause incidents:

- **`@RefreshScope` does not magically refresh everything.** A plain `@Value` field on a bean that is *not* in refresh scope keeps its startup value forever. Worse, `@Value` injected into singleton beans created before the refresh machinery (or into static contexts) silently goes stale. Prefer `@ConfigurationProperties` beans, which the refresh endpoint rebinds reliably.

- **Some beans need a real restart.** Connection pools (`DataSource` / HikariCP), thread pools, embedded servers, and Kafka listeners do not gracefully rebuild from a `RefreshEvent`. Changing `spring.datasource.url` and calling `/actuator/refresh` may leave you on the old connection — or in a half-migrated state. Treat infrastructure endpoints as restart-only, and dynamic flags/timeouts as refreshable.

- **Refresh ordering is not guaranteed.** A `RefreshEvent` recreates beans, but the order across many `@RefreshScope` beans is undefined. Avoid refresh-time logic that assumes bean B already saw the new value when bean A reconstructs.

- **Plaintext secrets live forever in Git history.** Encrypting a value *today* does nothing about the commit last month where you pasted it raw. You must rewrite history (`git filter-repo`) and rotate the leaked secret. Encrypt from day one.

- **An unsecured config server is a credential dump.** By default the server will happily serve every secret to anyone who can reach `:8888`. Put it behind authentication (Spring Security / mTLS), restrict it to the internal network, and never expose `/encrypt` or `/decrypt` publicly — those endpoints are an oracle.

- **`optional:` can hide a real outage.** `optional:configserver:` is great for local dev but in production it lets a service boot with stale or default config when the server is down — masking the failure. Make config mandatory in prod and let `fail-fast` plus retry surface the problem.

- **Profile precedence surprises.** Remember "more specific wins." If `application-prod.yml` and `order-service.yml` both set a key, the *service-specific* file wins even though it is not profiled. When in doubt, read the `propertySources` order in the raw `/{app}/{profile}` response.

- **Label and caching confusion.** The server caches the cloned repo. If you push to `main` but a client requests an old `label` (tag/commit), it will get the old values — by design. And a client that never refreshes keeps whatever it loaded at startup regardless of what Git now says. "I changed it in Git" is not the same as "it took effect."

## Wrapping up

Spring Cloud Config turns configuration into something you actually manage: versioned in Git, reviewed in pull requests, layered by profile, encrypted at rest, and changeable at runtime via a single `busrefresh`. The hard parts are not the annotations — they are knowing which values are safe to hot-reload, keeping secrets out of history, and locking down the server itself. Get those right and "redeploy to change a value" becomes a story you tell new hires.

---

If this saved you a weekend of grepping YAML, **follow devdomain** and drop a comment with the configuration war story that made you adopt a config server. What is the one value you wish you could change without a redeploy? I read and reply to every comment.

Spring Cloud Config: Centralized, Versioned Configuration for Microservices was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Picture a checkout service that calls a payment service. Payment starts responding in 30 seconds instead of 50ms. Every checkout request now holds a thread for 30 seconds. Your thread pool — say 200 threads — fills in seconds. Now checkout can’t serve any request, including ones that don’t even touch payment. The failure propagates upstream to whatever calls checkout, and so on. This is a cascading failure, and it’s the single most common way distributed systems fall over.

Resilience patterns exist to contain this blast radius. Resilience4j is a lightweight, functional, Spring-friendly library that implements the canonical patterns — circuit breaker, retry, rate limiter, bulkhead, time limiter — as composable decorators. Unlike the now-defunct Hystrix, it has no thread-pool baggage by default, integrates with Micrometer for metrics, and works beautifully with Spring Boot annotations. This article walks through each pattern with real code, then shows how to combine them correctly.

The mental model: decorate the call, don’t trust the network

Every Resilience4j pattern wraps a unit of work (a Supplier, Function, CompletableFuture, etc.) in a decorator that adds a guarantee. You can compose decorators functionally, or — far more common in Spring — apply them with annotations and configure them in application.yml.

The patterns answer different questions:

Pattern Question it answers Protects against Circuit Breaker Should I even try calling this right now? A dependency that’s down/slow — stop hammering it Retry Should I try again after a transient blip? Brief, recoverable failures Rate Limiter Am I allowed to call this many times? Overwhelming a dependency or hitting a quota Bulkhead How many concurrent calls are allowed? One dependency exhausting all your threads Time Limiter How long will I wait before giving up? Unbounded latency / hung calls

They are independent and stackable. A single outbound call might be wrapped in all five at once.

Setup

Spring Boot 3.x with the AOP-based Resilience4j starter and Actuator for metrics:

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.boot:spring-boot-starter-aop'
    implementation 'io.github.resilience4j:resilience4j-spring-boot3:2.2.0'
    implementation 'org.springframework.boot:spring-boot-starter-actuator'
    implementation 'io.micrometer:micrometer-registry-prometheus'
}

The spring-boot-starter-aop dependency is not optional — the annotations (@CircuitBreaker, @Retry, etc.) are implemented as AOP aspects. Forget it and your annotations silently do nothing, which is a deeply confusing failure mode.

Circuit Breaker: the centerpiece

The circuit breaker borrows its metaphor from electrical engineering: when something goes wrong, it trips and stops the flow rather than letting damage spread. It is a state machine with three primary states.

failures exceed threshold
  CLOSED ───────────────────────────►  OPEN
   ▲  │                                  │
   │  │ calls pass through               │ wait duration elapses
   │  └─ record success/failure          ▼
   │                              HALF_OPEN
   │     probe calls succeed          │
   └──────────────────────────────────┘
              probe calls fail → back to OPEN

• CLOSED — normal operation. Calls pass through; the breaker records outcomes in a sliding window.
• OPEN — the failure rate crossed the threshold. Calls are rejected immediately with CallNotPermittedException (no waiting, no thread held). The fallback runs instantly.
• HALF_OPEN — after a wait duration, the breaker allows a limited number of probe calls. If they succeed, it closes; if they fail, it re-opens.

The genius is the OPEN state: instead of letting 200 threads each wait 30 seconds on a dead service, the breaker fails them in microseconds and frees those threads. That’s what stops the cascade.

Configuration

resilience4j:
  circuitbreaker:
    instances:
      paymentService:
        sliding-window-type: COUNT_BASED
        sliding-window-size: 20
        minimum-number-of-calls: 10
        failure-rate-threshold: 50
        slow-call-rate-threshold: 80
        slow-call-duration-threshold: 2s
        wait-duration-in-open-state: 10s
        permitted-number-of-calls-in-half-open-state: 3
        automatic-transition-from-open-to-half-open-enabled: true
        record-exceptions:
          - java.io.IOException
          - java.util.concurrent.TimeoutException
        ignore-exceptions:
          - com.devdomain.payment.PaymentDeclinedException

Each knob matters:

• sliding-window-size: 20 + minimum-number-of-calls: 10 — the breaker evaluates the failure rate over the last 20 calls but won't trip until it has seen at least 10. This prevents one failure out of two calls (50%!) from instantly opening the circuit on startup.
• failure-rate-threshold: 50 — opens when ≥50% of windowed calls fail.
• slow-call-* — crucially, a call that succeeds slowly is still a problem. If 80% of calls take longer than 2s, the breaker opens even though nothing technically failed. Slow calls are the real cascade trigger, so configuring this is essential.
• ignore-exceptions — a declined payment is a business outcome, not an infrastructure failure. It must not count toward tripping the breaker. Getting this list right is the difference between a breaker that protects you and one that flaps on normal business errors.

Usage with annotations and a fallback

package com.devdomain.payment;

import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClient;

@Service
public class PaymentClient {

    private final RestClient restClient;

    public PaymentClient(RestClient.Builder builder) {
        this.restClient = builder.baseUrl("http://payment-service").build();
    }

    @CircuitBreaker(name = "paymentService", fallbackMethod = "chargeFallback")
    public PaymentResult charge(ChargeRequest request) {
        return restClient.post()
                .uri("/charges")
                .body(request)
                .retrieve()
                .body(PaymentResult.class);
    }

    // Fallback signature: same args + a trailing Throwable.
    private PaymentResult chargeFallback(ChargeRequest request, Throwable t) {
        if (t instanceof io.github.resilience4j.circuitbreaker.CallNotPermittedException) {
            // Circuit is OPEN — fail fast, queue for async retry, etc.
            return PaymentResult.deferred(request.orderId());
        }
        // Some other failure leaked through.
        return PaymentResult.failed(request.orderId(), t.getMessage());
    }
}

The fallback method must have a matching signature: the same parameters as the protected method, plus a trailing Throwable (or a more specific exception type for finer-grained handling). A mismatched signature throws NoSuchMethodException at runtime — another silent-looking footgun. You can define multiple overloaded fallbacks keyed by exception type; Resilience4j picks the most specific match.

Retry: for transient blips only

Retry re-invokes a failed call a bounded number of times. It is the right tool only for genuinely transient, idempotent operations: a momentary connection reset, a brief 503. Retrying a non-idempotent operation (like a payment charge) can double-charge a customer. Retrying a deterministic failure (a 400 Bad Request) just wastes time and amplifies load.

resilience4j:
  retry:
    instances:
      inventoryService:
        max-attempts: 3
        wait-duration: 500ms
        enable-exponential-backoff: true
        exponential-backoff-multiplier: 2
        enable-randomized-wait: true       # jitter
        randomized-wait-factor: 0.5
        retry-exceptions:
          - java.io.IOException
          - org.springframework.web.client.HttpServerErrorException
        ignore-exceptions:
          - org.springframework.web.client.HttpClientErrorException

Two non-negotiable details:

1. Exponential backoff. Retrying immediately hammers a struggling service and can push it from “slow” to “dead.” Back off: 500ms, 1s, 2s.
2. Jitter (enable-randomized-wait). Without it, if 500 clients all fail at the same instant, they all retry at exactly 500ms, then 1s, in synchronized waves — a thundering herd that keeps the dependency pinned down. Jitter spreads the retries out. This is the most overlooked retry setting and one of the most important.

@Retry(name = "inventoryService")
@CircuitBreaker(name = "inventoryService", fallbackMethod = "reserveFallback")
public Reservation reserve(String sku, int qty) {
    return restClient.post().uri("/reserve")
            .body(new ReserveRequest(sku, qty))
            .retrieve().body(Reservation.class);
}

Note retry-exceptions vs ignore-exceptions: 5xx errors are server-side and often transient (retry), while 4xx errors are caused by your request and won't improve on retry (ignore).

Rate Limiter: stay within bounds

A rate limiter caps how many calls are permitted per time window. Use it to respect a third-party API quota, or to protect a fragile internal dependency from your own traffic spikes.

resilience4j:
  ratelimiter:
    instances:
      geocodingApi:
        limit-for-period: 100        # 100 permits...
        limit-refresh-period: 1s     # ...refreshed every second
        timeout-duration: 250ms      # how long a call waits for a permit

@RateLimiter(name = "geocodingApi", fallbackMethod = "geocodeFallback")
public Coordinates geocode(String address) {
    return restClient.get().uri("/geocode?q={a}", address)
            .retrieve().body(Coordinates.class);
}

private Coordinates geocodeFallback(String address, RequestNotPermitted ex) {
    // Over the rate limit — degrade gracefully.
    return Coordinates.unknown();
}

If a permit isn’t available within timeout-duration, the call throws RequestNotPermitted and the fallback fires. Set timeout-duration: 0 to fail instantly rather than block a thread waiting for a permit.

Bulkhead: isolate the damage

The bulkhead — named after the watertight compartments in a ship — limits concurrent calls to a dependency, so one slow dependency can’t consume all your threads. Resilience4j offers two flavors:

• SemaphoreBulkhead (default) — a simple counting semaphore on the caller's thread. Lightweight, no extra threads.
• ThreadPoolBulkhead — runs calls on a dedicated, bounded thread pool with its own queue. Provides true isolation (the calling thread is freed) and only works with CompletableFuture return types.

resilience4j:
  bulkhead:                       # semaphore
    instances:
      reportService:
        max-concurrent-calls: 10
        max-wait-duration: 100ms
  thread-pool-bulkhead:           # thread-pool
    instances:
      heavyReport:
        core-thread-pool-size: 5
        max-thread-pool-size: 10
        queue-capacity: 20

// Semaphore bulkhead: at most 10 concurrent calls; the 11th waits up to 100ms then fails.
@Bulkhead(name = "reportService", type = Bulkhead.Type.SEMAPHORE)
public Report generate(ReportRequest req) { ... }

// Thread-pool bulkhead: must return CompletableFuture.
@Bulkhead(name = "heavyReport", type = Bulkhead.Type.THREADPOOL)
public CompletableFuture<Report> generateHeavy(ReportRequest req) {
    return CompletableFuture.supplyAsync(() -> doWork(req));
}

The point: even if reportService hangs completely, at most 10 of your threads are stuck. The other 190 keep serving unrelated traffic. That containment is what bulkheads buy you.

Time Limiter: bound the wait

A TimeLimiter caps how long an asynchronous call may run before being cancelled. It operates on CompletableFuture / reactive types — it cannot interrupt a blocking synchronous call (the underlying thread can't be force-killed safely), which is exactly why you pair it with the thread-pool bulkhead.

resilience4j:
  timelimiter:
    instances:
      pricingService:
        timeout-duration: 2s
        cancel-running-future: true

@TimeLimiter(name = "pricingService", fallbackMethod = "priceFallback")
@Bulkhead(name = "pricingService", type = Bulkhead.Type.THREADPOOL)
public CompletableFuture<Price> price(String sku) {
    return CompletableFuture.supplyAsync(() ->
            restClient.get().uri("/price/{sku}", sku).retrieve().body(Price.class));
}

private CompletableFuture<Price> priceFallback(String sku, TimeoutException ex) {
    return CompletableFuture.completedFuture(Price.cached(sku));
}

Combining patterns: order matters

Stacking annotations on one method is common, but the order in which Resilience4j applies the decorators is fixed and important. From outermost to innermost the default aspect order is:

Retry ( Circuit Breaker ( Rate Limiter ( Time Limiter ( Bulkhead ( call ) ) ) ) )

Read it inside-out and the logic is sound:

1. Bulkhead admits the call (concurrency check) — innermost.
2. Time Limiter bounds its duration.
3. Rate Limiter checks you’re within quota.
4. Circuit Breaker records the outcome and may short-circuit.
5. Retry wraps everything — so a retry re-enters the breaker, gets a fresh rate-limit permit, etc. — outermost.

This ordering is deliberate and usually what you want. A critical implication: Retry is outermost, so it counts circuit-breaker rejections as failures unless told otherwise. When the breaker is OPEN, each retry attempt gets a CallNotPermittedException instantly — retrying a fast-failing open breaker is pointless churn. Configure retry to not retry on CallNotPermittedException:

resilience4j:
  retry:
    instances:
      paymentService:
        max-attempts: 3
        ignore-exceptions:
          - io.github.resilience4j.circuitbreaker.CallNotPermittedException

A full real-world combination:

@Retry(name = "payment")
@CircuitBreaker(name = "payment", fallbackMethod = "fallback")
@RateLimiter(name = "payment")
@TimeLimiter(name = "payment")
@Bulkhead(name = "payment", type = Bulkhead.Type.THREADPOOL)
public CompletableFuture<PaymentResult> charge(ChargeRequest req) {
    return CompletableFuture.supplyAsync(() -> paymentClient.doCharge(req));
}

private CompletableFuture<PaymentResult> fallback(ChargeRequest req, Throwable t) {
    return CompletableFuture.completedFuture(PaymentResult.deferred(req.orderId()));
}

You can override the aspect order globally with properties like resilience4j.retry.retryAspectOrder, but the defaults are sensible — change them only with a clear reason.

Functional API: when annotations aren’t enough

Annotations cover most cases, but sometimes you need to decorate a call dynamically — for example, picking a circuit breaker instance by tenant, or wrapping a lambda you build at runtime. Resilience4j’s functional API does exactly that, and understanding it also demystifies what the annotations do under the hood.

import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry;
import io.github.resilience4j.retry.Retry;
import io.github.resilience4j.retry.RetryRegistry;
import io.github.resilience4j.decorators.Decorators;

import java.util.function.Supplier;

public class FunctionalExample {

    private final CircuitBreaker breaker;
    private final Retry retry;

    public FunctionalExample(CircuitBreakerRegistry cbRegistry, RetryRegistry retryRegistry) {
        this.breaker = cbRegistry.circuitBreaker("paymentService");
        this.retry = retryRegistry.retry("paymentService");
    }

    public PaymentResult charge(ChargeRequest req) {
        Supplier<PaymentResult> supplier = () -> paymentClient.doCharge(req);

        // Decorators apply in a controlled, explicit order.
        Supplier<PaymentResult> decorated = Decorators.ofSupplier(supplier)
                .withCircuitBreaker(breaker)
                .withRetry(retry)
                .withFallback(
                        java.util.List.of(Exception.class),
                        ex -> PaymentResult.deferred(req.orderId()))
                .decorate();

        return decorated.get();
    }
}

The Decorators builder makes composition explicit and is the cleanest way to wrap CompletableFuture chains or third-party clients you can't annotate. The registries (CircuitBreakerRegistry, RetryRegistry, etc.) are Spring beans auto-configured from your application.yml, so functional and annotation styles share the same configuration and metrics.

A common real use is per-tenant or per-endpoint isolation: cbRegistry.circuitBreaker("payment-" + tenantId) lazily creates and reuses a breaker keyed by tenant, so one noisy tenant's failures don't trip the breaker for everyone else.

Observability

Resilience4j publishes everything through Micrometer. Expose the actuator endpoints:

management:
  endpoints:
    web:
      exposure:
        include: health, circuitbreakers, metrics, prometheus
  health:
    circuitbreakers:
      enabled: true

You then get metrics like resilience4j_circuitbreaker_state, resilience4j_circuitbreaker_failure_rate, and resilience4j_retry_calls. Alert on circuit-breaker state transitions to OPEN — that's an early warning that a dependency is degrading, often before users notice. You can also register event listeners programmatically:

circuitBreakerRegistry.circuitBreaker("paymentService").getEventPublisher()
    .onStateTransition(e -> log.warn("CB {} -> {}",
        e.getCircuitBreakerName(), e.getStateTransition()));

Pitfalls and gotchas

• Missing spring-boot-starter-aop. Without it the annotations are no-ops and do nothing — and nothing tells you. Always confirm the AOP starter is present.
• Self-invocation. Annotations are applied via proxies. If method a() calls annotated method b() within the same bean, the proxy is bypassed and resilience is skipped. Put protected calls in a separate bean.
• Counting business errors as failures. A 404 or a PaymentDeclined is a normal outcome. If it counts toward the failure rate, your breaker trips on healthy traffic. Curate ignore-exceptions deliberately.
• Retrying non-idempotent operations. Retrying a charge or an order-create without an idempotency key can double-execute the side effect. Only retry idempotent calls, or pass an idempotency key the server dedupes on.
• No jitter on retries. Synchronized retries create thundering herds that keep a dependency down. Always enable randomized wait.
• Fallback signature mismatch. The fallback must match the args plus a trailing Throwable; otherwise it's never found at runtime.
• TimeLimiter on synchronous code. It only cancels futures. A blocking JDBC call won’t be interrupted — set the underlying client’s own connect/read timeouts too.
• Fallbacks that hide outages. A fallback returning empty/cached data can mask a total dependency failure so nobody notices for days. Always emit a metric/log from fallback paths.
• Retrying an open circuit. Add CallNotPermittedException to retry's ignore list, or you waste cycles retrying instant rejections.
• One shared instance name for everything. Each dependency should have its own named instance so one flaky service’s failures don’t trip the breaker for an unrelated one.

Wrapping up

Resilience isn’t a single feature you bolt on — it’s a layered defense. The circuit breaker stops you from hammering a dead dependency and frees threads instantly. Retry with backoff and jitter recovers from transient blips without creating herds. Rate limiters keep you within quota, bulkheads contain concurrency damage, and time limiters bound the wait. Composed in the right order, with business errors carefully excluded and every fallback observable, they turn a brittle service mesh into one that degrades gracefully instead of collapsing.

Start with a circuit breaker and a time limiter on every outbound call. Add the rest where your traffic profile demands it. And test your failure paths deliberately — resilience you’ve never exercised is resilience you don’t actually have.

Found this useful? Clap and follow devdomain for more hands-on distributed-systems engineering. How are you tuning your circuit breakers and bulkheads in production? Share your war stories in the comments.

Resilient Microservices with Resilience4j: Circuit Breakers, Retries, and Bulkheads was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.

Hibernate ships with three distinct caching layers, and they solve different problems with different lifetimes, different scopes, and very different failure modes. If you treat them as one big “cache” switch you will eventually get burned. This article unpacks all three: the first-level (persistence-context) cache that you already use whether you know it or not, the optional second-level cache and its providers, and the query cache that almost everyone misconfigures.

The three caches at a glance

Cache Scope Lifetime Default Stores First-level (L1) Single EntityManager/Session One persistence context (usually one transaction) Always on, cannot disable Managed entity instances Second-level (L2) EntityManagerFactory (whole app, per JVM) Until evicted/expired Off Entity state (dehydrated), collections Query cache EntityManagerFactory Until evicted/expired Off Query result identifiers

The key mental model: L1 caches entity object references; L2 caches dehydrated entity state; the query cache caches the IDs a query returned, not the entities themselves.

First-level cache: the persistence context

The L1 cache is the persistence context. Every EntityManager (Hibernate Session) maintains a map of every managed entity it has loaded or persisted, keyed by entity type and primary key. You don't configure it, you can't turn it off, and it is scoped to a single persistence context — typically a single transaction.

import jakarta.persistence.EntityManager;
import jakarta.persistence.PersistenceContext;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class ProductService {

    @PersistenceContext
    private EntityManager em;

    @Transactional
    public void demonstrateL1() {
        Product p1 = em.find(Product.class, 1L); // SELECT hits the database
        Product p2 = em.find(Product.class, 1L); // no SQL — served from L1

        // Same managed instance, guaranteed identity within the context:
        assert p1 == p2;
    }
}

The second find issues no SQL. More importantly, p1 == p2 is true: within one persistence context Hibernate guarantees a single in-memory instance per database row. This is the identity guarantee, and it is why dirty checking and lazy loading work at all.

Why L1 sometimes “lies”

Because L1 is bound to the persistence context, two transactions see two separate caches. This is correct, but it surprises people:

@Transactional
public void update(Long id, String name) {
    Product p = em.find(Product.class, id);
    p.setName(name);
    // Auto dirty-checked and flushed at commit — `no explicit save needed.
}

If another thread loaded the same product into its persistence context before your commit, that thread still sees the old name. That’s not a cache bug — it’s transaction isolation. L1 never crosses transaction boundaries, so it cannot serve stale data across requests. The dangerous one is the next layer.

Second-level cache: shared across the application

The L2 cache lives on the EntityManagerFactory and is shared by every session in the JVM. It survives transactions. When enabled and an entity is marked cacheable, Hibernate stores the entity's dehydrated state (a flat array of column values, not the object graph) in a region, keyed by ID.

L2 stores state, not instances. When you read a cached entity, Hibernate rehydrates a fresh object from the stored array. That’s why L2 entries can be shared safely across threads — “nobody hands out a mutable shared object.

Enabling L2 in Spring Boot 3.x

You need a provider on the classpath plus configuration. Hibernate uses JCache (JSR-107) as the standard bridge, with EhCache 3 or Caffeine as common backends.

<!-- pom.xml -->
<dependency>
    <groupId>org.hibernate.orm</groupId>
    <artifactId>hibernate-jcache</artifactId>
</dependency>
<dependency>
    <groupId>org.ehcache</groupId>
    <artifactId>ehcache</artifactId>
    <version>3.10.8</version>
    <classifier>jakarta</classifier>
</dependency>

# application.properties
spring.jpa.properties.hibernate.cache.use_second_level_cache=true
spring.jpa.properties.hibernate.cache.region.factory_class=jcache
spring.jpa.properties.hibernate.javax.cache.provider=org.ehcache.jsr107.EhcacheCachingProvider
spring.jpa.properties.hibernate.javax.cache.uri=classpath:ehcache.xml
# Surface what gets cached during development:
spring.jpa.properties.hibernate.generate_statistics=true

Note the jakarta classifier on EhCache 3.10+— without it you'll pull the javax-namespaced jars and Hibernate 6 (Jakarta Persistence) will not wire up.

Marking entities cacheable

L2 is opt-in per entity. Use Jakarta’s @Cacheable plus Hibernate's @Cache to pick a concurrency strategy:

import jakarta.persistence.Cacheable;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import org.hibernate.annotations.Cache;
import org.hibernate.annotations.CacheConcurrencyStrategy;

@Entity
@Cacheable
@Cache(usage = CacheConcurrencyStrategy.READ_WRITE, region = "products")
public class Product {

    @Id
    private Long id;

    private String name;
    private java.math.BigDecimal price;
    // getters/setters
}

For collections, you must annotate the association itself — caching the owning entity does not cache its collections:

@OneToMany(mappedBy = "product")
@Cache(usage = CacheConcurrencyStrategy.READ_WRITE, region = "product.reviews")
private List<Review> reviews = new ArrayList<>();

Concurrency strategies

The strategy controls how Hibernate keeps the cache coherent with the database under concurrent writes.

Strategy Use when Behavior READ_ONLY Reference data that never changes Fastest; throws if you try to update NONSTRICT_READ_WRITE Rare writes, brief staleness OK Evicts (not updates) on commit; small stale window READ_WRITE Frequent writes, need consistency Uses soft locks during write; safe but more overhead TRANSACTIONAL JTA environments with XA Cache participates in the transaction

For most Spring Boot apps on a single datasource, READ_WRITE is the safe default and READ_ONLY is ideal for lookup tables (countries, currencies, config).

A minimal ehcache.xml

<config xmlns='http://www.ehcache.org/v3'>
  <cache alias="products">
    <expiry><ttl unit="minutes">30</ttl></expiry>
    <heap unit="entries">10000</heap>
  </cache>
  <cache alias="default-query-results-region">
    <expiry><ttl unit="minutes">5</ttl></expiry>
    <heap unit="entries">2000</heap>
  </cache>
</config>

Using Caffeine or Redis instead

Caffeine is a drop-in JCache provider for a fast local cache — swap the provider class and dependency:

spring.jpa.properties.hibernate.javax.cache.provider=com.github.benmanes.caffeine.jcache.spi.CaffeineCachingProvider

Redis gives you a distributed L2 so multiple app instances share (and invalidate) the same cache. The common choice is Redisson, which ships a Hibernate region factory:

spring.jpa.properties.hibernate.cache.region.factory_class=org.redisson.hibernate.RedissonRegionFactory
spring.jpa.properties.hibernate.cache.redisson.config=classpath:redisson.yaml

The trade-off is fundamental: local caches (EhCache/Caffeine) are faster but go stale across nodes unless every write evicts on every node; distributed caches (Redis) stay coherent but add a network hop on every cache hit. Don’t reach for Redis L2 until you’ve proven a local cache causes cross-node staleness problems.

The query cache

The query cache solves a problem L2 doesn’t: caching the results of a query. But it does so in a subtle way — it stores only the primary keys the query returned, plus a timestamp. The entities themselves are then fetched from L2 (or the database if not cached there).

This means the query cache is nearly useless on its own. If you cache a query but the matched entities aren’t in L2, Hibernate re-fetches every entity by ID — sometimes turning one query into N. Always enable L2 for the entities a cached query returns.

spring.jpa.properties.hibernate.cache.use_query_cache=true

import org.hibernate.jpa.AvailableHints;
import jakarta.persistence.TypedQuery;

public List<Product> findActive() {
    TypedQuery<Product> q = em.createQuery(
        "select p from Product p where p.active = true", Product.class);
    q.setHint(AvailableHints.HINT_CACHEABLE, true); // opt-in per query
    return q.getResultList();
}

Query cache invalidation and the timestamp region

Hibernate maintains a special UpdateTimestampsCache region. Every time a table is modified, Hibernate records the timestamp of the last update for that table. When it considers serving a cached query result, it checks: was any queried table modified after this result was cached? If so, the cached result is discarded.

This makes the query cache automatically invalidated on writes to the affected tables — which is correct, but also means a write-heavy table will constantly bust its query cache, giving you all the overhead and none of the benefit. The query cache pays off only for queries over tables that change rarely relative to how often the query runs.

Invalidation: how each layer stays fresh

Event L1 L2 Query cache Entity updated via Hibernate Updated in context Region entry updated/evicted Table timestamp bumped → results busted em.clear() / context closed Cleared Untouched Untouched Native SQL UPDATE/DELETE Not aware Stale! Stale unless you tell Hibernate TTL expiry (provider) n/a Entry evicted Entry evicted

The killer is the third row. Hibernate only invalidates L2 for changes it performs through the ORM. Native queries, JDBC, Flyway/Liquibase migrations, and direct DB edits bypass L2 entirely, leaving stale entries behind. If you run native bulk updates, tell Hibernate which spaces (tables) they touch:

import org.hibernate.query.NativeQuery;

NativeQuery<?> q = em.createNativeQuery("UPDATE product SET price = price * 1.1")
        .unwrap(NativeQuery.class);
q.addSynchronizedEntityClass(Product.class); // invalidate Product's L2 region
q.executeUpdate();

You can also evict programmatically through the JPA Cache API:

import org.springframework.beans.factory.annotation.Autowired;
import jakarta.persistence.EntityManagerFactory;

@Autowired
EntityManagerFactory emf;

public void evict() {
    emf.getCache().evict(Product.class, 1L); // one entity
    emf.getCache().evict(Product.class);      // whole Product region
    emf.getCache().evictAll();                // nuke everything
}

How L2 stores data: hydration and dehydration

It’s worth dwelling on what L2 actually keeps, because it explains several behaviors that otherwise look arbitrary.

When Hibernate puts an entity into L2, it does not store your object. It stores a dehydrated form: a flat Object[] of the entity's column values, indexed by property, plus the version and the discriminator if applicable. Associations are stored as foreign-key identifiers, not as resolved object graphs.

When you read from L2, Hibernate hydrates a brand-new entity instance from that array, associates it with your current persistence context, and hands it back. Two important results follow:

• L2 entries are immutable snapshots of state. Two threads reading the same cached entity each get their own fresh instance — there’s no shared mutable object to corrupt. This is what makes L2 thread-safe.
• Lazy associations are not “pre-loaded” by L2. A cached entity’s @OneToMany is still a lazy proxy unless that collection has its own cached region. L2 caching the parent does nothing for the children.

This dehydration model is also why L2 cannot help with arbitrary query results — a query returns rows that may not map one-to-one to a cacheable entity. That’s the gap the query cache fills, and the reason it stores IDs rather than state.

Putting it together: a worked configuration

Here’s a coherent setup for a catalog service: products are read constantly and updated occasionally, categories are effectively static, and a “find active products” query runs on every page load.

@Entity
@Cacheable
@Cache(usage = CacheConcurrencyStrategy.READ_WRITE, region = "products")
public class Product { /* ... */ }

@Entity
@Cacheable
@Cache(usage = CacheConcurrencyStrategy.READ_ONLY, region = "categories")
public class Category { /* ... never changes at runtime */ }

spring.jpa.properties.hibernate.cache.use_second_level_cache=true
spring.jpa.properties.hibernate.cache.use_query_cache=true
spring.jpa.properties.hibernate.cache.region.factory_class=jcache
spring.jpa.properties.hibernate.javax.cache.provider=org.ehcache.jsr107.EhcacheCachingProvider
spring.jpa.properties.hibernate.javax.cache.uri=classpath:ehcache.xml

Category uses READ_ONLY (cheapest, no soft-locking) because it's immutable at runtime. Product uses READ_WRITE because it's updated. The active-products query is cacheable and is backed by the cached Product region, so a cache hit returns IDs and then resolves them straight from L2 — no database round trip at all on the happy path. When any product is written, the timestamp region busts the query result and the next page load rebuilds it.

When caching helps — and when it hurts

L2 and the query cache pay off when reads massively outnumber writes and the data is small enough to fit comfortably in memory: reference tables, configuration, product catalogs, rarely-changing aggregates. The hit ratio is what matters; check it with statistics:

import org.hibernate.stat.Statistics;

Statistics stats = emf.unwrap(org.hibernate.SessionFactory.class).getStatistics();
long hits = stats.getSecondLevelCacheHitCount();
long misses = stats.getSecondLevelCacheMissCount();
double ratio = (double) hits / (hits + misses);

A hit ratio below ~80% usually means the cache costs more than it saves: you pay the put/evict overhead on writes and the memory footprint without recovering it on reads.

Caching hurts when:

• Writes are frequent. Every write evicts/updates entries and busts query results. You add overhead and gain little.
• Data must be strictly fresh (prices, inventory, balances). Even a 5-minute TTL is a 5-minute window for wrong answers.
• You run multiple nodes with a local L2. Node A’s write doesn’t evict Node B’s cache. You now have per-node stale data that’s maddening to reproduce.
• Cardinality is huge. Caching millions of rarely-reread rows just evicts useful entries.

Common pitfalls

• Enabling the query cache without L2. The query cache stores IDs; without L2 each cached query re-hydrates every row by ID — often slower than no cache.
• Forgetting @Cache on collections. Caching the parent entity does not cache its @OneToMany/@ManyToMany. The association needs its own @Cache.
• Native/bulk writes silently going stale. em.createQuery("delete ...").executeUpdate() (JPQL bulk) and native SQL bypass L1 and, for native, L2 unless you add synchronized spaces. Always evict or synchronize.
• Mixing the javax and jakarta EhCache jars in Hibernate 6 — use the jakarta classifier or(the cache region factory silently won't initialize.
• Caching mutable reference data as READ_ONLY. If it ever changes, you'll get an exception on write or a permanently stale value. Use READ_WRITE if updates are possible.
• Local L2 in a clustered deployment. This is the single most common stale-data incident. Use Redis (Redisson) or accept TTL-bounded staleness deliberately.
• No TTL / unbounded heap. Without expiry and size limits you cache forever and risk OutOfMemoryError.
• Trusting demo numbers. L1 makes a single transaction look cache-friendly. Real gains come only from cross-transaction L2 with a measured high hit ratio.

Wrapping up

Hibernate’s three caches are not interchangeable. L1 is always there, scoped to your transaction, and gives you the identity guarantee that makes the ORM work. L2 is an opt-in, JVM-wide store of dehydrated state that pays off for read-heavy reference data — if you pick the right concurrency strategy and respect invalidation. The query cache is a narrow tool that only helps over rarely-changing tables and only when L2 backs it up.

Measure your hit ratios, be honest about your read/write mix, and never assume a write you made outside the ORM updated the cache. Used deliberately, caching is a scalpel. Used as a magic flag, it’s a stale-data generator.

If this saved you a production incident, follow devdomain for more deep-dives on Spring Boot, JPA, and Hibernate internals. Got a caching war story or a config that bit you? Drop it in the comments — I read and reply to every one.

Hibernate Caching Explained: First-Level, Second-Level, and Query Cache was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.

This guide separates the two cleanly, shows real Spring Boot 3.x code for each, builds a worked example that combines them, and — most importantly — is blunt about when they help and when they’ll wreck your delivery timeline.

CQRS: One Model Was Two Models All Along

CQRS stands for Command Query Responsibility Segregation. The idea, distilled: the model you use to change state should be separate from the model you use to read state.

In a traditional CRUD app, one object — usually a JPA entity — serves both jobs. You load Order, mutate it, save it; you also load Order to display it. The two jobs pull in opposite directions. Writes want normalized data, invariants, and small transactional units. Reads want denormalized, query-shaped data optimized for the screen that displays it. Serving both from one model means compromises on both.

CQRS splits them:

• Commands express intent to change state (PlaceOrderCommand, CancelOrderCommand). They return nothing meaningful (maybe an ID or void). They go through a model rich in business rules.
• Queries ask for data (GetOrderSummary). They return DTOs. They never change state and can read from a model shaped exactly for the answer.

COMMAND SIDE (write)                      QUERY SIDE (read)
   ┌────────────────────┐                    ┌───────────────────┐
   │ PlaceOrderCommand   │                    │ GetOrderSummary     │
   │  -> CommandHandler  │                    │  -> QueryHandler    │
   │  -> Aggregate       │                    │  -> Read model DTO  │
   │  -> Write store     │ ── propagate ─ₔ�▶  │  -> Read store      │
   They return DTOs. └───────────────────┘                   └───────────────────┘

At its simplest, CQRS needs no special infrastructure — just discipline. Two sets of handlers, two sets of objects, one database:

// --- Command side ---
public record PlaceOrderCommand(UUID customerId, List<LineItem> items) {}

@Service
class PlaceOrderHandler {
    private final OrderRepository orders;     // write model
    PlaceOrderHandler(OrderRepository orders) { this.orders = orders; }

    @Transactional
    public UUID handle(PlaceOrderCommand cmd) {
        Order order = Order.place(cmd.customerId(), cmd.items()); // enforces invariants
        orders.save(order);
        return order.id().value();
    }
}

// --- Query side ---
public record OrderSummaryDto(UUID id, String customerName, BigDecimal total, String status) {}

@Service
class OrderQueryService {
    private final JdbcTemplate jdbc;          // read model, hand-tuned SQL, no JPA
    OrderQueryService(JdbcTemplate jdbc) { this.jdbc = jdbc; }

    public OrderSummaryDto getSummary(UUID id) {
        return jdbc.queryForObject("""
            SELECT o.id, c.name AS customer_name, o.total, o.status
            FROM orders o JOIN customers c ON c.id = o.customer_id
            WHERE o.id = ?""",
            (rs, n) -> new OrderSummaryDto(
                rs.getObject("id", UUID.class), rs.getString("customer_name"),
                rs.getBigDecimal("total"), rs.getString("status")),
            id);
    }
}

That’s “level 1” CQRS — same database, separate code paths. It’s cheap, it’s useful, and it’s where most teams should stop. You only escalate to separate read/write databases when read and write scaling profiles genuinely diverge.

Event Sourcing: Store the Facts, Not the State

Event Sourcing is a different idea entirely. Instead of storing the current state of an entity, you store the sequence of events that led to it. The current state is a left-fold over those events.

Traditional persistence:

orders table:  id=42, status=SHIPPED, total=120.00

Event-sourced persistence:

event stream for order 42:
  1. OrderPlaced     {customerId, items, total: 120.00}
  2. OrderPaid       {paymentId, amount: 120.00}
  3. OrderShipped    {trackingNumber, carrier}

To get the current state, you replay the events from the beginning. The events are the source of truth; current state is derived and disposable.

Why would anyone do this? Because the events are an immutable, complete audit log by construction. You never lose history — you know not just that an order is shipped, but when it was placed, paid, and shipped, by whom, and in what order. You can answer questions you didn’t know to ask when you designed the system, replay events to debug production, and build entirely new read models retroactively from history.

The Aggregate as an Event Folder

An event-sourced aggregate applies commands to produce events, and applies events to mutate its own state:

public class Order {

    private OrderId id;
    private OrderStatus status;
    private Money total;
    private final List<Object> uncommittedEvents = new ArrayList<>();

    // ----- COMMANDS: validate, then emit events (do NOT mutate directly) -----
    public static Order place(OrderId id, CustomerId customer, List<OrderLine> lines) {
        if (lines.isEmpty()) throw new IllegalArgumentException("Empty order");
        Order order = new Order();
        Money total = lines.stream().map(OrderLine::subtotal).reduce(Money.ZERO, Money::add);
        order.raise(new OrderPlaced(id, customer, lines, total, Instant.now()));
        return order;
    }

    public void ship(String trackingNumber) {
        if (status != OrderStatus.PAID) {
            throw new IllegalStateException("Cannot ship an unpaid order");
        }
        raise(new OrderShipped(id, trackingNumber, Instant.now()));
    }

    // ----- APPLY: state changes happen ONLY here, driven by events -----
    private void apply(OrderPlaced e) {
        this.id = e.orderId(); this.status = OrderStatus.PLACED; this.total = e.total();
    }
    private void apply(OrderShipped e) { this.status = OrderStatus.SHIPPED; }

    private void raise(Object event) {
        dispatch(event);
        uncommittedEvents.add(event);
    }

    // Replay constructor: rebuild from history
    public static Order rehydrate(List<Object> history) {
        Order order = new Order();
        history.forEach(order::dispatch);   // apply without re-recording
        return order;
    }

    private void dispatch(Object event) {
        switch (event) {
            case OrderPlaced e  -> apply(e);
            case OrderShipped e -> apply(e);
            default -> throw new IllegalStateException("Unknown event " + event);
        }
    }

    public List<Object> uncommittedEvents() { return List.copyOf(uncommittedEvents); }
    public void markCommitted() { uncommittedEvents.clear(); }
}

A Minimal Event Store on Spring + JDBC

You don’t need a specialized event-store database to start. A single append-only table with optimistic concurrency on a version column is enough:

CREATE TABLE event_store (
    aggregate_id   UUID        NOT NULL,
    version        INT         NOT NULL,
    event_type     VARCHAR(200) NOT NULL,
    payload        JSONB       NOT NULL,
    occurred_at    TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (aggregate_id, version)   -- enforces no two events at same version
);

@Repository
class JdbcEventStore {

    private final JdbcTemplate jdbc;
    private final ObjectMapper mapper;
    // constructor omitted

    @Transactional
    public void append(UUID aggregateId, int expectedVersion, List<Object> events) {
        int version = expectedVersion;
        for (Object event : events) {
            version++;
            try {
                jdbc.update("""
                    INSERT INTO event_store (aggregate_id, version, event_type, payload)
                    VALUES (?, ?, ?, ?::jsonb)""",
                    aggregateId, version, event.getClass().getName(), toJson(event));
            } catch (DuplicateKeyException dup) {
                throw new ConcurrencyException(aggregateId, version); // someone else wrote
            }
        }
    }

    public List<Object> load(UUID aggregateId) {
        return jdbc.query(
            "SELECT event_type, payload FROM event_store WHERE aggregate_id = ? ORDER BY version",
            (rs, n) -> fromJson(rs.getString("event_type"), rs.getString("payload")),
            aggregateId);
    }
}

The PRIMARY KEY (aggregate_id, version) is doing the heavy lifting: two concurrent commands that both think they(re at version 3 can't both insert version 4 — one wins, the other gets a ConcurrencyException and retries. This is optimistic concurrency for free.

Projections: Building Read Models from Events

The query side of an event-sourced CQRS system is built by projections — handlers that consume events and write to read-optimized tables. This is where CQRS and Event Sourcing fit together so naturally: the write side emits events, the read side projects them into whatever shape the UI needs.

@Component
class OrderSummaryProjection {

    private final JdbcTemplate jdbc;
    // constructor omitted

    @EventListener
    void on(OrderPlaced e) {
        jdbc.update("""
            INSERT INTO order_summary (id, customer_id, total, status)
            VALUES (?, ?, ?, 'PLACED')""",
            e.orderId().value(), e.customerId().value(), e.total().amount());
    }

    @EventListener
    void on(OrderShipped e) {
        jdbc.update("UPDATE order_summary SET status = 'SHIPPED' WHERE id = ?",
            e.orderId().value());
    }
}

Rebuilding State and Replaying

The superpower of event sourcing: a read model is disposable. Need a new report? Write a new projection and replay the entire event log through it. Found a projection bug? Truncate the read table, replay, done. Want to debug a production incident? Replay events into a test environment to reproduce the exact state.

// Rebuild a projection from scratch
public void rebuildOrderSummary() {
    jdbc.update("TRUNCATE order_summary");
    eventStore.streamAllInOrder().forEach(eventPublisher::publishEvent);
}

This is impossible with state-based storage — you only ever had “the current value,” and the path to it is gone.

Snapshots — when replay gets slow

Replaying 50,000 events to load one aggregate is slow. Snapshots store the materialized state at version N; on load you start from the snapshot and replay only events after it.

public Order load(UUID id) {
    Snapshot snap = snapshots.latest(id).orElse(null);
    if (snap == null) return Order.rehydrate(eventStore.load(id));
    Order order = Order.fromSnapshot(snap.state());
    order.replay(eventStore.loadAfter(id, snap.version()));
    return order;
}

Eventual Consistency: The Tax You Must Pay

Here is the part the conference talks gloss over. The moment you have a separate read store updated by projections, the read side lags the write side. A user places an order (command succeeds), immediately navigates to “my orders,” and… it isn’t there yet, because the projection hasn’t run.

This is eventual consistency, and it is not a bug you can fix — it is the defining trade-off. You manage it, you don’t eliminate it:

• Return the new ID/state from the command so the UI can render optimistically without a round trip to the read model.
• Show “processing” states in the UI for operations the user just triggered.
• Keep the projection lag small (synchronous in-process projections lag by milliseconds; async cross-service projections by seconds).
• Never enforce a write-side invariant by querying the read model — it may be stale. Invariants live in the aggregate, always.

Aspect Plain CRUD CQRS (1 DB) CQRS + Event Sourcing Read/write models Shared Separate Separate Source of truth Current state Current state Event log Audit / history Manual Manual Built-in Read consistency Strong Strong or eventual Eventual Operational complexity Low Medium High Time-travel / replay No No Yes Good default? Yes Sometimes Rarely

When CQRS / Event Sourcing Help — and When They Hurt

Reach for CQRS when:

• Read and write workloads scale very differently (e.g., 1000:1 read:write).
• Read queries are complex and would force ugly compromises on the write model.
• Different teams own reads and writes.

Reach for Event Sourcing when:

• A complete, immutable audit trail is a hard requirement (finance, healthcare, compliance).
• Temporal questions matter (“what did the balance look like on March 3rd?”).
• You genuinely need to build new read models from historical data after the fact.

Avoid both when:

• Your domain is simple CRUD. The overhead buys you nothing and costs you everything.
• Your team is new to the patterns and under deadline. The learning curve is real and the failure modes are subtle.
• Strong read-after-write consistency is a product requirement you can’t design around.
• You’d be adopting them for résumé reasons. (Be honest.)

Common Pitfalls

• Adopting both because they’re “always together.” They’re independent. Start with CQRS-lite if you need it; add event sourcing only for a specific, justified reason.
• Mutable or “fixable” events. Events are immutable historical facts. You never edit a stored event. To correct a mistake, append a compensating event (OrderCorrected), preserving the true history.
• Event schema evolution ignored. Your stored events from 2023 must still deserialize in 2026. Version events, use upcasters, and never remove fields blindly. This is the single most underestimated long-term cost.
• Giant aggregates / huge streams. A stream with millions of events is slow to replay and a snapshot maintenance burden. Keep aggregates small.
• Leaking eventual consistency to users without UX design. “Where did my order go?” support tickets are an architecture smell that reached the customer.
• Querying the read model to make write decisions. Stale data plus business invariant equals data corruption. Invariants belong in the aggregate.
• No idempotent projections. Events can be redelivered. A projection that does balance += amount without idempotency double-counts. Make handlers idempotent (e.g., track last processed version).

Closing Thoughts

CQRS and Event Sourcing are sharp tools. CQRS — even the lightweight, single-database flavor — is broadly useful and low-risk; separating the intent to change from the act of reading clarifies most non-trivial domains. Event Sourcing is powerful but heavy: it pays off spectacularly where audit, temporality, and replay are first-class needs, and it punishes teams who adopt it for novelty. Treat eventual consistency as a design constraint, not a surprise. And when in doubt, start simple — you can evolve a clean CRUD app toward CQRS far more easily than you can dig out of an event-sourcing system you didn’t actually need.

If this clarified the difference, follow devdomain for more pragmatic architecture writing. Have you run event sourcing in production — what bit you that you wish you’d known? Tell me in the comments.

CQRS and Event Sourcing with Spring Boot: A Practical Guide was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.

That “magic” is auto-configuration: a disciplined, conditional, ordered mechanism that looks at your classpath and existing beans and decides what infrastructure to wire up. Once you understand it, it stops being magic and becomes a tool you can read, debug, and extend. This article takes the lid off: how @EnableAutoConfiguration discovers configurations, how the @Conditional family decides what to apply, how ordering is resolved, how to read the conditions report, and how to write your own auto-configuration and package it as a starter.

Where it all begins: @SpringBootApplication

Your entry point is annotated with @SpringBootApplication, which is a meta-annotation composing three things:

@SpringBootConfiguration   // @Configuration + marks the primary config
@EnableAutoConfiguration   // the star of this article
@ComponentScan             // scans the current package and below
public @interface SpringBootApplication { }

@ComponentScan finds your beans. @EnableAutoConfiguration brings in Spring Boot's opinionated infrastructure beans. The two are independent: you can use component scanning without auto-configuration and vice versa.

@EnableAutoConfiguration and the import mechanism

@EnableAutoConfiguration is itself simple:

@AutoConfigurationPackage
@Import(AutoConfigurationImportSelector.class)
public @interface EnableAutoConfiguration { }

The workhorse is AutoConfigurationImportSelector, a DeferredImportSelector. "Deferred" matters: it runs after all your regular @Configuration classes and component-scanned beans are processed. That ordering is what allows auto-configuration to "back off" when you've already defined a bean yourself — it gets to look last.

The selector’s job is to produce a list of fully-qualified class names of configuration classes to import. Where does that list come from?

spring.factories vs AutoConfiguration.imports

Historically (Spring Boot 2.x and earlier), candidate auto-configurations were listed in META-INF/spring.factories under the key org.springframework.boot.autoconfigure.EnableAutoConfiguration:

# META-INF/spring.factories (legacy, deprecated since 2.7)
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.example.MyAutoConfiguration,\
com.example.AnotherAutoConfiguration

Since Spring Boot 2.7, and as the only supported mechanism in 3.x, the list lives in a dedicated file:

META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports

One fully-qualified class name per line:

com.example.MyAutoConfiguration
com.example.AnotherAutoConfiguration

This change is more than cosmetic. The new format is read with a faster, purpose-built parser and avoids the overloaded spring.factories file (which is still used for other extension points like ApplicationContextInitializer). If you maintain a library, migrate to AutoConfiguration.imports.

┌──────────────────────────────────────────────────────────────┐
│ Application start                                              │
│   @EnableAutoConfiguration                                     │
│                                                         │
│                                                           │
│        ▼                                                       │
│   AutoConfigurationImportSelector (DeferredImportSelector)     │
│        │  1. read every jar's AutoConfiguration.imports        │
│        │  2. remove exclusions & duplicates                    │
│        │  3. apply AutoConfigurationImportFilter (fast classpath│
│        │     checks via condition metadata)                    │
│        │  4. sort with @AutoConfiguration before/after + order  │
│        ▼                                                       │
│   Each surviving class evaluated against its @Conditional(s)   │
│        │                                                       │
│        ▼                                                       │
│   Beans registered only where every condition matches          │
└─────────────────────────────────────────────────────────────┘

@AutoConfiguration: the modern annotation

In Spring Boot 3.x, auto-configuration classes are annotated with @AutoConfiguration rather than plain @Configuration. It's a specialized @Configuration(proxyBeanMethods = false) that also carries ordering hints:

import org.springframework.boot.autoconfigure.AutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;

@AutoConfiguration(after = DataSourceAutoConfiguration.class)
@ConditionalOnClass(JdbcTemplate.class)
public class MyJdbcAutoConfiguration {
    // bean methods
}

proxyBeanMethods = false (the default for @AutoConfiguration) means bean methods are called in "lite" mode — faster and the recommended style, but it means calling one @Bean method from another returns a new instance, not the shared singleton. Inject dependencies as method parameters instead of calling sibling @Bean methods.

The @Conditional family

Conditions are the brain of auto-configuration. Each is a @Conditional that consults a Condition implementation returning true/false. Spring Boot ships a rich set:

Annotation Matches when… @ConditionalOnClass A class is present on the classpath @ConditionalOnMissingClass A class is absent @ConditionalOnBean A bean of a type/name already exists @ConditionalOnMissingBean No such bean exists yet (back-off) @ConditionalOnProperty A property has a given value (or just exists) @ConditionalOnResource A classpath resource exists @ConditionalOnWebApplication The app is a servlet/reactive web app @ConditionalOnNotWebApplication The app is not a web app @ConditionalOnExpression A SpEL expression evaluates true @ConditionalOnSingleCandidate Exactly one candidate bean (or a primary) exists

@ConditionalOnMissingBean: the back-off contract

This is the condition that makes Spring Boot feel polite. An auto-configuration provides a default only if you haven’t:

@AutoConfiguration
public class JacksonAutoConfigurationLike {

    @Bean
    @ConditionalOnMissingBean
    public ObjectMapper objectMapper() {
        return new ObjectMapper();
    }
}

If you define your own ObjectMapper bean anywhere, the auto-configured one backs off. Because AutoConfigurationImportSelector is deferred, the framework evaluates your beans first, so @ConditionalOnMissingBean sees them.

A subtle gotcha: @ConditionalOnMissingBean checks the bean type. If your custom bean is registered under a more specific type than the auto-config checks for, the back-off may not trigger. Be explicit:

@Bean
@ConditionalOnMissingBean(ObjectMapper.class)
public ObjectMapper objectMapper() { /* ... */ }

Ordering of conditions matters

@ConditionalOnBean and @ConditionalOnMissingBean operate on the current state of the bean factory at evaluation time. Because auto-configurations are evaluated in a specific order, a @ConditionalOnBean only sees beans defined earlier — either by user config or by an auto-configuration ordered before it. This is exactly why ordering exists.

Ordering auto-configurations

Three tools control order, and they all live on the class:

@AutoConfiguration(
    after = DataSourceAutoConfiguration.class,
    before = SomeOtherAutoConfiguration.class)

Or the standalone annotations for cross-cutting hints:

@AutoConfigureAfter(DataSourceAutoConfiguration.class)
@AutoConfigureBefore(JpaAutoConfiguration.class)
@AutoConfigureOrder(Ordered.HIGHEST_PRECEDENCE)

• @AutoConfigureAfter / before express relative ordering against named configurations. Use these when your config depends on another having run.
• @AutoConfigureOrder sets an absolute priority for tie-breaking.

Critically, ordering is not dependency injection ordering — it’s the order in which configuration classes are processed, which determines what beans exist when each class’s conditions are evaluated. The actual bean instantiation order is still driven by the dependency graph.

A complete worked example

Suppose we build a library that provides a GreetingService, but only when a greeting template property is present and the user hasn't supplied their own service.

package com.acme.greeting;

public class GreetingService {
    private final String template;
    public GreetingService(String template) { this.template = template; }
    public String greet(String name) { return template.formatted(name); }
}

The configuration properties record:

package com.acme.greeting;

import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "acme.greeting")
public record GreetingProperties(String template) {
    public GreetingProperties {
        if (template == null) template = "Hello, %s!";
    }
}

The auto-configuration:

package com.acme.greeting;

import org.springframework.boot.autoconfigure.AutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;

@AutoConfiguration
@ConditionalOnProperty(prefix = "acme.greeting", name = "enabled", havingValue = "true", matchIfMissing = true)
@EnableConfigurationProperties(GreetingProperties.class)
public class GreetingAutoConfiguration {

    @Bean
    @ConditionalOnMissingBean
    public GreetingService greetingService(GreetingProperties props) {
        return new GreetingService(props.template());
    }
}

Register it so the import selector finds it. Create:

src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports

with one line:

com.acme.greeting.GreetingAutoConfiguration

Now any Spring Boot app that adds this jar gets a GreetingService — unless they define their own, or set acme.greeting.enabled=false. That's the whole contract.

Generating metadata for fast filtering

Add the annotation processors so your config integrates with the conditions report and IDE auto-completion:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-configuration-processor</artifactId>
    <optional>true</optional>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-autoconfigure-processor</artifactId>
    <optional>true</optional>
</dependency>

The autoconfigure-processor generates META-INF/spring-autoconfigure-metadata.properties, which lets AutoConfigurationImportFilter reject configurations before loading their classes — a meaningful startup-time optimization in large apps. The configuration-processor generates spring-configuration-metadata.json so acme.greeting.template autocompletes in your IDE's application.yml.

Writing a custom @Conditional

When the built-in conditions aren’t enough, write your own Condition. Suppose you want a bean only when the app runs on a machine with at least 4 CPUs:

package com.acme.greeting;

import org.springframework.context.annotation.Condition;
import org.springframework.context.annotation.ConditionContext;
import org.springframework.core.type.AnnotatedTypeMetadata;

public class OnMultiCoreCondition implements Condition {
    @Override
    public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) {
        return Runtime.getRuntime().availableProcessors() >= 4;
    }
}

Apply it with @Conditional(OnMultiCoreCondition.class) on a @Bean method or a configuration class. For richer cases, extend SpringBootCondition, which provides a ConditionOutcome with a human-readable message that shows up in the conditions report — far more debuggable than a bare boolean:

import org.springframework.boot.autoconfigure.condition.ConditionOutcome;
import org.springframework.boot.autoconfigure.condition.SpringBootCondition;
import org.springframework.context.annotation.ConditionContext;
import org.springframework.core.type.AnnotatedTypeMetadata;

public class OnMultiCoreCondition extends SpringBootCondition {
    @Override
    public ConditionOutcome getMatchOutcome(ConditionContext ctx, AnnotatedTypeMetadata md) {
        int cores = Runtime.getRuntime().availableProcessors();
        return cores >= 4
            ? ConditionOutcome.match("found " + cores + " cores")
            : ConditionOutcome.noMatch("only " + cores + " cores");
    }
}

A crucial rule: conditions evaluated at the class level via @ConditionalOnClass must reference the gated class only in the annotation, never in a method signature you might touch before the condition runs. Spring Boot is careful to read @ConditionalOnClass values from the annotation metadata via ASM, without loading the class — so a missing class doesn't cause NoClassDefFoundError. If you reference the class directly in a field or return type that gets resolved during candidate filtering, you defeat that protection. Keep gated types behind the condition.

How auto-configuration interacts with your @Configuration

A common question: if both my @Configuration and an auto-configuration define a bean, who wins? Because the import selector is deferred, your beans are registered first. Auto-configurations using @ConditionalOnMissingBean then see your bean and back off — so you win. This is the entire design philosophy of Spring Boot: provide sensible defaults, but always let the application override them. The framework never fights you for a bean you explicitly declared.

This also means the order of your own configuration classes relative to auto-configuration is fixed and predictable: user configuration always precedes auto-configuration. You don’t need ordering annotations on your own @Configuration to "beat" an auto-config — the deferred import guarantees it.

The auto-configuration vs starter distinction

People say “starter” loosely, but there’s a clean separation:

• An auto-configuration is the code: @AutoConfiguration classes plus their .imports file.
• A starter is a (usually code-free) dependency aggregator: a pom.xml that pulls in the auto-configuration plus the libraries it configures.

By convention, name your starter acme-spring-boot-starter (third-party starters use the <name>-spring-boot-starter suffix; spring-boot-starter-<name> is reserved for the Spring team). The starter's only job is convenient dependency management:

<!-- acme-greeting-spring-boot-starter/pom.xml -->
<dependencies>
    <dependency>
        <groupId>com.acme</groupId>
        <artifactId>acme-greeting-autoconfigure</artifactId>
    </dependency>
    <!-- plus any libraries the autoconfigure module needs at runtime -->
</dependencies>

Keeping autoconfigure and starter as separate modules lets advanced users depend on the autoconfigure module alone if they want finer control.

Debugging with the conditions report

When auto-configuration does something you don’t expect — or doesn’t do something you expected — turn on the conditions evaluation report:

debug=true

or run with --debug. At startup you get a report with three sections:

• Positive matches — configurations that applied, with the condition that matched.
• Negative matches — configurations that did not apply, and exactly which condition failed.
• Unconditional classes — always applied.

A negative match reads like this:

DataSourceAutoConfiguration:
   Did not match:
      - @ConditionalOnClass did not find required class
        'javax.sql.DataSource' (OnClassCondition)

This is the single most useful debugging tool for auto-configuration. Instead of guessing why your DataSource didn't appear, the report tells you the exact condition that vetoed it.

You can also hit the Actuator conditions endpoint at runtime for the same data as JSON:

management.endpoints.web.exposure.include=conditions

Excluding auto-configurations

Sometimes you want to opt out. Three equivalent ways:

@SpringBootApplication(exclude = { DataSourceAutoConfiguration.class })

spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration

The property form is handy when the class isn’t on your compile classpath. If you exclude a class that isn’t a candidate auto-configuration, Spring Boot fails fast with a clear error — so typos surface immediately.

Common pitfalls

• Using plain @Configuration for auto-config. In 3.x, use @AutoConfiguration so ordering hints and the deferred-import contract work correctly. Plain @Configuration listed in .imports won't be ordered like an auto-configuration.
• Calling sibling @Bean methods. With proxyBeanMethods = false (the @AutoConfiguration default), calling one bean method from another returns a fresh instance. Inject the dependency as a parameter instead.
• Forgetting the .imports file. No file, no discovery. The class is invisible to the selector no matter how it's annotated.
• @ConditionalOnMissingBean type mismatch. If your custom bean's declared type differs from what the condition checks, back-off won't trigger. Name the type explicitly.
• Ordering assumptions. @ConditionalOnBean only sees beans from earlier-evaluated configs. If your condition depends on another auto-config's bean, declare @AutoConfigureAfter for it.
• spring.factories in new code. It still works for some extension points but is the wrong place for auto-config in 3.x. Use AutoConfiguration.imports.
• Heavy condition logic. Custom Condition classes run during context startup. Keep them cheap; avoid I/O.

Wrapping up

Auto-configuration is not magic — it’s a deferred import selector that reads a list of candidate @AutoConfiguration classes from AutoConfiguration.imports, filters them with fast metadata checks, orders them, and evaluates each against its @Conditional annotations. The back-off behavior you rely on every day comes from @ConditionalOnMissingBean plus the fact that your beans are processed first. Armed with the conditions report and the @Conditional family, you can read exactly why any bean did or didn't appear — and you can package your own infrastructure as a clean, reusable starter that behaves just like Spring Boot's own.

If this cleared up the magic for you, follow devdomain for more under-the-hood Spring deep dives, and tell me in the comments which auto-configuration surprised you most when you first read its source.

How Spring Boot Auto-Configuration Actually Works (Under the Hood) was originally published in devdomain on Medium, where people are continuing the conversation by highlighting and responding to this story.