A short guide to Blocker

Update: The following blogpost was written for cats-effect 2. Cats-effect 3 has a completely new threading model and the Blocker was entirely removed. You can read more changes for blocking operations in CE3 here.

Developing a multi-threading application can be an uneasy and error-prone job. Fortunately using correct concurrency tools could make that task easier.

In this post, I will try to describe the best practices related to using thread pools with Blocker from cats-effect.

Why do we need to use separate thread pools?

We can categorize operations performed in a multi-threaded application into two main groups:

1. CPU-bound tasks. These operations do computations required by our application. They should run on pools with a fixed number of preallocated threads (usually equal to the number of available CPUs). Spawning more threads might harm an application performance since it would require a processor to handle context switching.
2. I/O-bound tasks. Such tasks do operations which involve I/O, like performing network request or accessing the hard drive. These kinds of operations can be non-blocking or blocking depending on the implementation.

When the operation is non-blocking it initializes I/O action and then releases the thread to do other stuff. Later, it should asynchronously notify other threads that I/O call is done (for example by calling a callback function or resolving a Future). Unfortunately, not all libraries support asynchronous non-blocking API.

If an operation is blocking it will force invoking thread to do nothing but wait idly for I/O call to finish. For that reason, it’s not the best idea to run blocking tasks on the thread pool with a fixed number of threads. It can very easily lead to a situation when all threads will be waiting for I/O instead of doing any computations.
To avoid that problem it is good practice to run blocking tasks on the separate thread pool. That pool is usually unbounded (if necessary additional threads are spawned to handle extra I/O calls) and cached (to allow reuse of threads that are no longer blocked).

You can learn more about uses of various kinds of thread pools on this gist by Daniel Spiewak.

Threads waiting in pool. Photo by sergio souza from Pexels

ContextShift

Before getting to examples, let’s define several tasks: both blocking and non-blocking:

We would also need ExecutionContext for running our blocking operations:

Prior to version 1.4.0 of cats-effect, there were two methods to shift threads.

Firstly, we can use IO.shift, which specifies that all effects coming after the shift should run on the execution context (or cats-effect’s ContextShift) provided as an argument.

After switching to blocking context we need to call IO.shift again to change back to the original thread pool. It gets even more complicated if blocking operation can raise an error. In this case, it’s essential to run the operation of shifting back to the original thread pool on finalizer.

Alternatively, we can use evalOn from ContextShift.

ContextShift is just a wrapper over ExecutionContext. It is required as an implicit argument by many concurrency-related methods from cats-effect API.

If you’re using IOApp to bootstrap your application, you probably noticed there’s already an instance of ContextShift in scope. It’s backed by execution context with a fixed number of threads. So it’s not the best fit for running IO‑bound tasks.

Method evalOntakes two arguments. The first is the execution context on which tasks will be executed. The second one is the effect to be run.
After executing operation context is switched back to the underlying thread pool of ContextShift.

Using that approach doesn’t require manual shifting back, but also has a drawback. Method evalOn is expecting a very general type of ExecutionContext. If you are using several execution contexts defined in your application it is possible to confuse them and execute blocking operation on a wrong thread pool.

You can read more on difficulties related to handling thread pool switching with cats‑effect in an article written by Adam Warski.

Using Blocker

To address the above-mentioned issues Blocker was introduced in cats-effect 1.4.0.

Blocker is just a data type that wraps ExecutionContext similarily as ContextShift. Its main purpose is to wrap a separate thread pool for running just blocking operations.

Blocker has method blockOn, which is similar to evalOn from ContextShift. Method blockOn executes effect on blocking thread pool, that is wrapped by Blocker . Then shifts backs to thread pool of implicit ContextShift being in scope.

As another option, you can use delay, which doesn’t take an effect, but a block of code producing a value. It works similarly to IO.delay but executes effect on blocking context.

You can also extract the underlying ExecutionContext of Blocker by just calling blockingContext.

Creating Blocker instances

Unlike ContextShift there’s no default Blocker provided by IOApp, so we will always have to create instances of Blocker by hand.

It is also advised to pass instances of Blocker explicitly instead of using them as implicit arguments. Quote from docs:

Instances of this class should *NOT* be passed implicitly because they hold state and in some cases your application may need different instances of Blocker.

We could create several instances of Blocker to create a logical separation of thread pools handling different kinds of I/O operations (for example one could be used for network calls to external services and another for calling database). By avoiding passing Blocker instances implicitly we reduce the chance of mistaking and switching them.

There are several methods to produce Blocker. First of all, we can create it by calling the apply method, which will return Resource[F, Blocker]. New Blocker instance will be backed by a cached thread pool. All spawned threads will be daemon threads.

After the resource is used the underlying ExecutorService will be automatically shut down.

If we want to use customized executor we can construct Blocker with Blocker.fromExecutorService. It expects effect creating ExecutorService as an argument. That method also returns Resource.

We can also create Blocker by lifting existing ExecutionConcext or ExecutorService with methods Blocker.liftExecutorService and Blocker.liftExecutionContext.

These methods return a plain instance of Blocker, instead of Resource. So in this case, it’s our responsibility to shut down executors.

Cats Effect 3

Wrapping up

Handling blocking operation correctly might be a tricky task. Luckily there’s the right tool to do the job.

You can find all examples used in this article on GitHub.

If you want to learn more about Blocker you can also check this video by Jakub Kozłowski.

Happy blocking!