Stage 10: Concurrency and the JVM, lesson 3 of 8

ExecutorService, ThreadPoolExecutor and Future

Advanced6 min read@since 8Code runs on your Java 25
Explain it forThe essentials plus production detail and pitfalls.

Creating a new thread for every task is slow (each platform thread is an OS thread with its own stack) and dangerous under load (thousands of threads can exhaust memory). An ExecutorService (Java 5) keeps a pool of reusable threads and a queue of waiting tasks.

  • submit(task) accepts a Runnable or a Callable and returns a Future; execute(runnable) just runs it.
  • Ready-made pools: Executors.newFixedThreadPool(n), newCachedThreadPool(), newSingleThreadExecutor(), newScheduledThreadPool(n), and since Java 21 newVirtualThreadPerTaskExecutor().
  • Most of them are a ThreadPoolExecutor underneath: a core size, a maximum size, a work queue and a rejection policy. In production, configure it yourself so the queue is bounded.
  • Future.get(timeout, unit) waits for a result; invokeAll runs a batch and waits for all of them.
  • Always shut the pool down: shutdown() stops new tasks and lets running ones finish.
Thread pool lab

The ready-made thread pools

  • newFixedThreadPool(n): exactly n threads and an unbounded queue. Predictable, but tasks can pile up.
  • newCachedThreadPool(): creates threads as needed and reuses idle ones (60-second timeout). Great for many short tasks, dangerous under heavy load (no upper limit).
  • newSingleThreadExecutor(): one thread, so tasks run one at a time, in order.
  • newScheduledThreadPool(n): runs tasks after a delay or repeatedly.
  • newWorkStealingPool(): a ForkJoinPool that keeps all cores busy with many small tasks.
  • newVirtualThreadPerTaskExecutor() (Java 21): a new virtual thread for every task; ideal for blocking I/O.

submit() vs execute()

execute(Runnable) returns nothing; if the task throws, the exception goes to the thread's uncaught-exception handler (usually printed). submit(...) returns a Future, and any exception is stored inside the Future: if you never call get(), the failure disappears silently. Use execute for fire-and-forget work, submit when you need the result or want to handle failures.

Java
pool.execute(() -> { throw new IllegalStateException("boom"); });   // printed by the thread
Future<?> f = pool.submit(() -> { throw new IllegalStateException("boom"); });   // silent...
f.get();                                                            // ...until get() throws ExecutionException

Future: a result that arrives later

A Future<V> is a handle to a result that isn't ready yet:

  • get() waits for it; get(timeout, unit) waits at most that long and then throws TimeoutException.
  • isDone() checks without waiting.
  • cancel(true) interrupts the task if it's running; isCancelled() tells you if it was.
  • If the task threw, get() throws ExecutionException; the original exception is getCause().

Future can't be chained or combined; that's what CompletableFuture (next lesson) adds.

invokeAll() and invokeAny()

invokeAll(tasks) runs a collection of Callables and returns their Futures once all have finished. invokeAny(tasks) returns the result of the first one to succeed and cancels the rest, handy for querying several mirrors and taking the fastest answer.

Java
List<Callable<Integer>> lookups = List.of(
        () -> stockIn("pune"), () -> stockIn("delhi"), () -> stockIn("goa"));

int total = 0;
for (Future<Integer> f : pool.invokeAll(lookups, 5, TimeUnit.SECONDS)) {
    if (!f.isCancelled()) total += f.get();     // tasks that missed the timeout are cancelled
}

String fastest = pool.invokeAny(List.of(() -> fetchFrom("mirror-1"), () -> fetchFrom("mirror-2")));

ThreadPoolExecutor: the seven settings

new ThreadPoolExecutor(core, max, keepAlive, unit, queue, threadFactory, handler):

  1. corePoolSize: threads kept alive even when idle.
  2. maximumPoolSize: the upper limit.
  3. keepAliveTime and unit: how long threads above core may stay idle.
  4. workQueue: where waiting tasks go. Use a bounded ArrayBlockingQueue in production.
  5. threadFactory: creates threads; use it to give them meaningful names.
  6. handler: the rejection policy when both queue and threads are full.

A new task: start a core thread if below core → otherwise queue it → queue full: add a thread up to max → still full: reject.

Rejection policies

When the queue is full and the pool is at its maximum:

  • AbortPolicy (default): throws RejectedExecutionException.
  • CallerRunsPolicy: the thread that submitted the task runs it itself, which naturally slows producers down (simple back-pressure).
  • DiscardPolicy: silently drops the task.
  • DiscardOldestPolicy: drops the oldest queued task and retries.

Choose deliberately: silently dropping payments or emails is rarely acceptable.

How many threads?

  • CPU-bound work (calculations, compression): about the number of cores. More threads only add switching.
  • I/O-bound work (HTTP calls, database queries): more, roughly cores × (1 + wait time ÷ compute time). A task that waits 90 ms for every 10 ms of CPU on 8 cores suggests about 80 threads.
  • Measure under realistic load, keep queues bounded, and use separate pools for different kinds of work, so a slow partner API can't starve everything else (the "bulkhead" idea). For lots of blocking I/O on Java 21+, virtual threads remove most of this tuning.

Shutting down properly

shutdown() stops accepting tasks and lets queued and running ones finish; awaitTermination() waits for that; shutdownNow() interrupts running tasks and returns the ones that never started. The pattern below is the one recommended in the ExecutorService documentation. Since Java 19, close() (and try-with-resources) does a graceful shutdown and waits.

Java
pool.shutdown();
try {
    if (!pool.awaitTermination(30, TimeUnit.SECONDS)) {
        pool.shutdownNow();                     // cancel whatever is still running
    }
} catch (InterruptedException e) {
    pool.shutdownNow();
    Thread.currentThread().interrupt();
}

Scheduled tasks

ScheduledExecutorService runs tasks later or repeatedly. scheduleAtFixedRate starts runs at a fixed rhythm (every 10 minutes from the first start); scheduleWithFixedDelay waits a fixed time after each run ends, so slow runs never overlap. Catch exceptions inside the task: an uncaught exception silently cancels all future runs. In Spring Boot, @Scheduled does the same with less code.

Java
ScheduledExecutorService timer = Executors.newSingleThreadScheduledExecutor();
timer.scheduleWithFixedDelay(() -> {
    try {
        refreshCache();
    } catch (Exception e) {
        log.error("Cache refresh failed", e);   // without this, one failure stops the schedule
    }
}, 0, 10, TimeUnit.MINUTES);

Virtual threads per task (Java 21)

Executors.newVirtualThreadPerTaskExecutor() starts a cheap virtual thread for every task, so you can run thousands of blocking calls without sizing a pool. It's the modern default for I/O-heavy work; keep a bounded platform-thread pool (or a Semaphore) when you must limit how many calls hit a downstream system. The "Virtual threads" lesson covers this in depth.

Java
try (ExecutorService perTask = Executors.newVirtualThreadPerTaskExecutor()) {
    for (String url : urls) perTask.submit(() -> download(url));
}   // waits for every task, then closes

Example

Java
// inside a method that declares: throws InterruptedException
ExecutorService pool = Executors.newFixedThreadPool(4);
try {
    Future<Integer> price = pool.submit(() -> fetchPrice("SPRING-101"));   // a Callable: returns a value
    Future<Integer> seats = pool.submit(() -> fetchSeats("SPRING-101"));   // both run at the same time

    System.out.println(price.get(2, TimeUnit.SECONDS) + " / " + seats.get(2, TimeUnit.SECONDS));
} catch (ExecutionException e) {
    log.error("A lookup failed", e.getCause());      // the task's own exception is the cause
} catch (TimeoutException e) {
    log.warn("A lookup took too long");
} finally {
    pool.shutdown();                                 // no new tasks; queued and running ones finish
}
A production-grade ThreadPoolExecutor
AtomicInteger n = new AtomicInteger();
ThreadPoolExecutor pool = new ThreadPoolExecutor(
        4,                                          // core threads (kept even when idle)
        8,                                          // maximum threads
        60, TimeUnit.SECONDS,                       // extra threads die after 60 s idle
        new ArrayBlockingQueue<>(200),              // BOUNDED queue: back-pressure, not OutOfMemoryError
        r -> new Thread(r, "orders-" + n.incrementAndGet()),   // named threads: readable logs and dumps
        new ThreadPoolExecutor.CallerRunsPolicy()); // when full, the caller runs the task (slows producers)

Common mistake

Forgetting shutdown(). The pool's worker threads aren't daemon threads, so the JVM never exits. Close behind: calling future.get() without a timeout, which can wait forever.

Under the hood

How a ThreadPoolExecutor handles a new task: (1) fewer than core threads running → start a new thread; (2) otherwise queue it; (3) queue full → start a thread up to maximum; (4) still full → reject it (the default AbortPolicy throws RejectedExecutionException). This is why newFixedThreadPool (an unbounded queue) never grows beyond its core size and can pile up tasks until memory runs out, and why newCachedThreadPool can create thousands of threads. Size pools by the work: about the number of cores for CPU-bound tasks, more for I/O-bound ones. Since Java 19 ExecutorService is AutoCloseable, so try-with-resources shuts it down; in Spring, configure a ThreadPoolTaskExecutor for @Async.

Check yourself

A newFixedThreadPool(4) receives 100 tasks at once. What happens?

How this connects

Part of Multithreading: beginner to advanced.

Was this lesson helpful?

Finished reading? Mark it complete to track your progress.