Managing dependencies using ZIO

ZIO, a toolkit for type-safe, composable asynchronous and concurrent programming for Scala, has recently received a overhaul of its “environment” component: a new approach to defining the dependencies required to run a computation.

“Environment”, by Zofia Warska

What is ZIO’s environment? A (potentially asynchronous/concurrent) process in ZIO is described using a value of type ZIO[R, E, A]. The E type specifies the possible errors, with which the computation might fail; A specifies the type of the result of the computation, should it succeed; and finally R describes the environment (or requirement) that is needed by the computation.

You can think of ZIO[R, E, A] as a description of a function R => E | A, that is a function which takes a value of type R, and outputs either an error E, or success A. Of course, ZIO’s process descriptions are much more powerful than “normal” Scala functions. We get a number of combinators which make it feasible and safe to define computations which run asynchronously, spawn concurrent processes, signal and recover from errors, as well as define various environmental requirements.

Dependency management is a subject of many debates, and a common problem that needs to be addressed when developing any kind of application. How does ZIO environment implement dependency management? Is it a form of Dependency Injection (DI)? How does it compare with other Dependency Injection approaches?

We’ll try to address at least some of these questions below.

ZIO environment has already been a subject of an article comparing it to other Dependency Injection approaches, however it covered the previous generation of ZIO env. The current incarnation is significantly different (and improved), hence this new summary.

But what is DI?

Everybody has their own definition of Dependency Injection, hence before we start diving into ZIO, I think it might be beneficial to define what I have in mind when using this term.

From “What is Dependency Injection?”:

Dependency Injection is the process of creating the static, stateless graph of service objects, where each service is parametrised by its dependencies.

While ZIO environment can also be used for other purposes, here we’ll focus on how it can be used to create the (mostly) static graph of objects, which are (again, mostly) mostly used as stateless services to provide the functionality of our application.

Basic building blocks

While ZIO[R, E, A] is the basic building block of any ZIO-based application, there’s another component that is crucial when it comes defining and managing the dependencies that a process or program needs: ZLayer.

ZLayer[RIn, E, ROut] describes how to construct ROut dependencies, using RIn dependencies; this construction process might fail with an error of type E.

A layer can describe how to construct a single dependency. This can be thought of as a more powerful version of a constructor. A normal constructor:

class ServiceA(b: ServiceB, c: ServiceC)

which can also be viewed as a function:

(ServiceB, ServiceC) => ServiceA

in ZIO env could be represented as a value of type:

ZLayer[Has[ServiceB] with Has[ServiceC], Nothing, Has[ServiceA]]

We can describe the creation of the dependency in a variety of ways, comparing to a normal constructor. For example, a dependency can be created using effectful logic for allocation. Or, we get the option to specify how to release any resources that might be acquired during construction. That way, we can not only safely create the service, but also close it once it’s no longer needed. ZLayer contains a number of combinators, covering multiple use-cases.

You might be wondering about the Has[_] wrapper around service types. These wrappers are needed so that ZIO can represent multiple dependencies using with, introduce new dependencies to the mix or remove some, as well as ensure that there are no namespace clashes. Most of the time Has[_] can be treated as a technical detail, without going into its implementation details.

A layer can also describe how to construct multiple dependencies; layers can be combined, where one layer‘s outputs are fed into another layer’s inputs, and a bigger layer is created; this is similar to function composition. Again, there’s a number of combinators meeting a variety of needs.

Hence, a ZLayer describes how to construct a fragment of the application’s service graph, fitting nicely into our definition of Dependency Injection.

Yes, you’ve read that right — ZIO environment is a way to implement Dependency Injection in an application! In a type-safe, resource-safe, potentially concurrent way, with principled error handling, without reflection or classpath scanning.

“Animate and inanimate nature”, by Zofia Warska

Recipe for a single ZIO dependency

If we want to use ZIO environment and ZLayers to manage the dependencies in our applications, we’ll have to start with single services, and gradually build our way up, creating more and more complete fragments of the complete service graph.

How to define a single service and its corresponding ZLayer? That’s where things get a bit more complex, comparing to what you might know from e.g. a straightforward definition using a constructor.

Typically, we’ll need to provide four components, mostly combined in a single object (but they can live separately as well):

1. the interface of our service. The return types of methods in this interface typically result in an effectful computation, but without any dependencies. That is, each business method might return ZIO[Any, E, A] (or some variant), without any requirements on the environment (here, Any means “no requirement”, or “works in any environment”). That’s because the requirements on the environment are a detail of the service’s implementation, and we want to hide them from the service’s users (that’s also known as encapsulation)
2. an implementation of our service, described as a ZLayer. Dependencies that are required to implement the functionality of our service become the inputs of the layer. The output of the layer is an instance of the service
3. (optional) method accessors, one for each method in our interface. These values express the requirement that in order to invoke a method of the service, an instance of it must be present in the environment
4. (optional, in a package object) a type alias, wrapping the service’s interface in a Has[_]

While 1. and 2. are more or less essential when using any kind of dependency management/DI approach (with some variations on the amount of code needed), 3. is typical “boilerplate” code which is required by the ZIO environment approach. Scala is flexible, but not flexible enough to generate these automatically. Similarly 4.: the type alias is a technical detail that makes the code more readable when working with this DI approach.

However, 3. and 4. are optional. The bare minimum is some way to reference the service’s interface (e.g. a trait, or a class), and the layer definition.

By example

This probably sounds quite abstract; let’s look at an example. Here, we’ll be describing a UserModel service, which has a single insert method, and depends on a DB service instance to implement its functionality:

To reiterate some of the points from above:

1. The insert method returns a Task[Unit], which expands toZIO[Any, Throwable, Unit]. We expect this process to describe how to insert a user to a database; however, this description doesn’t declare any dependencies, as that’s only the interface; the dependencies should be provided by the implementation.
2. The live layer is an implementation of the service. It can be defined separately from the interface, and there might be other implementations e.g. for testing, or for other configurations. The layer here is constructed using one of the available combinators, fromService. This accepts a function, which given the dependencies (here: an instance of the DB service), returns an instance of the service. The implementation uses the methods of the dependency to implement its functionality. Similarly, db.execute returns a Task[Unit] — its dependencies are hidden, and we achieve encapsulation.
3. The method accessor is useful if we’d want to invoke the method outside of a layer definition. It expresses the requirement that to invoke the given logic, an instance of the service needs to be provided. This is not required for all dependencies, though.
4. The type aliases for conveniently expressing dependencies on our services. Note that when expressing the dependency on the DB instance in the layer definition, we are using the type alias from the package object: type DB = Has[DB.Service].

Here’s how we could use the UserModel dependency that we’ve defined in a UserRegistration service which, unlike the previous example, uses a class with constructor, instead of a trait + implementation in an anonymous class:

You can browse through the whole example on GitHub. It contains a very simple application, with 6 services, in three variants: wired using ZIO environment, and using constructors, and a mixed approach (more on that later). Its dependency graph is as follows:

To complete the picture, here’s the definition of two lower-lever services: DB, which depends on ConnectionPool, which in turn depends on DBConfig. Note that the layer for ConnectionPool is created differently, as this is a procedural service (for example, coming from a Java library), which needs to be “lifted” to the ZIO world, and defines acquire/release logic:

Putting things together

So far we’ve created a number of layers, each defining their dependencies and providing implementations on top of these dependencies. However, we still need to wire our application, and combine all of these layers into one final layer, without any dependencies, so that we can run the application.

We’ll use three basic operations on layers. The first is creating a layer with no dependencies — we have to start the wiring process somehow! That can be done with ZLayer.succeed, which takes a value and creates a layer which outputs it.

The second is layer1 >>> layer2, which feeds the output of layer1 into layer2. This is similar to function composition, or nesting constructor calls.

The third is layer1 ++ layer2, which creates a layer with combined inputs and combined outputs.

Using these operators, we can create a complete service graph for our application:

As mentioned in the code, we’re using the UserRegistration.register accessor method to describe the program we want to run (here, a simple invocation of a single method). This program value needs a UserRegistration instance (note: this is the type alias we are using here!) to run. This service is constructed through layer composition, and provided to the program so that we can construct the final, self-contained description of the application to run.

There’s more to layers

If all of this looks complex to create a simple application graph — you are probably right, it is. However, our example is simplistic, created only to showcase the basic mechanics, so probably shouldn’t be used to judge the overall complexity of the presented approach.

Layers have many other possibilities; just to name a few of the features:

• automatic parallel construction of layers, if possible
• locally updating dependencies
• error management and retries when constructing layers
• shared or local instances
• multiple combinators to integrate with effectful / managed instances (resources)

To explore ZLayers in more depth, the official documentation is a good start. The code examples to the “Functional and Reactive Domain Modeling” book by Debasish Ghosh, implemented using various effect representations, might also be a good illustration of this approach. Finally, you can find two other introductions to to ZLayer, by Pavels Sisojevs and by appddeevv.

“Natural environment” by Maria Fałkowska

An alternative

What’s the alternative? The simplest possible approach might also work, that is, using constructor arguments to express dependencies and then invoking them when creating the object graph.

However, we’ll face some problems. We’ll have to manually deal with resources which need to be acquired & released, or which need effectful instantiation; that’s where ZLayers truly shine, and when DI-using-constructors could be more verbose and complex.

Another area (not demonstrated by the example, as it’s too small), where ZLayers offer an elegant solution, is composing partial service graphs into bigger ones — using layer composition. When using constructors, we have to somehow modularize this process of creating the object graph as well; for example, using traits-as-modules.

On the other hand, for “normal” services, which just require a couple of dependencies and implement their logic in terms of these dependencies, ZLayer is an overkill. Let’s look at our example service implemented using constructors:

As mentioned, we would have to implement resource acquisition “by hand”. Here we have only one resource, so that’s quite simple:

Generally speaking, instead of using ZLayer mechanisms, we can use ZManaged for resource management, ZIO for effectful instantiation, constructors for “plain” services, and monadic composition (using for-comprehensions) when creating the final object graph.

Combining the two

Constructor-based and ZLayer-based dependency injection both have their pros and cons. The natural question comes up: is it possible to combine the two, and get the best of both worlds?

In general: yes. In practice: we’ll see. Here’s how wiring our example service could look, where the ConnectionPool and DB services are managed using layers, while the “pure” business logic components (UserModel, UserNotifier and UserRegistration) are wired using constructors:

The mixed variant of creating the object graph

This distinction comes from the fact that not all services, or more generally dependencies are equal. Some are stateful, and have a prescribed lifecycle — all kinds of resources, such as thread or connection pools, fall in that category. Here, we might get real benefits from using layers, which have rich error handling and resource management capabilities.

Such dependencies are usually low-level and used to integrate with the outside world.

Other services contain the core functionality of the application, and implement their business logic using the lower-level components. Grouping these methods into “services” is done more because of code organization concerns, readability and modularization.

Summing up

Of course, it would be best to try this approach on a larger project. However for that, we should probably at least wait for ZIO’s 1.0 final release. Would it be feasible, to use ZIO environment at scale? I think yes.

Most of the time it could be sufficient to use the mixed approach, and use a single layer to construct a part of the object graph (with multiple services) using plain constructors. Accessor methods could probably be avoided in most cases, and we would only need to create them for “leaf” services, on-demand. The type aliases would be a pain, but probably a pain of a manageable degree.

What are the good sides of ZIO environment?

• a unified, consistent approach to managing dependencies in the ZIO ecosystem
• same way of dealing with effectful, or managed resources, as with “plain” services which don’t require special allocation/deallocation logic
• multiple combinators to create single-service layers, and to compose many layers
• simplified code when expressing logic to create multiple resources, which need to be allocated in a specific order (or in parallel), with error handling
• representing partial object graphs as a layer value, and hence modularizing object graph creation in a consistent manner

But of course it’s not all roses. What are the bad sides?

• boilerplate code: layer description is needed even for most basic services; method accessors are needed to conveniently call service methods from the top-level; type aliases are recommended as well to maintain code readability
• everything needs to be lifted to the ZIO world: most Scala/Java libraries expose their functionality using plain values and constructors. These need to be wrapped into ZIO and ZLayer values.

When to use layers, and when to use constructors? We’ll have to wait for more experience from the trenches to find out how practical the mixed approach is, but for now a possible rule of thumb would be:

• ZIO environment: low-level, effectful dependencies, which directly integrate with the outside world, such as a database connection pool, email service integration or a message broker interface.
• ZIO environment: partial object graphs, fragments of our applications with relatively few depedencies; in other words, a representation of a wired application-level module
• Constructors: business logic, encapsulated into manageable, readable, functional pieces. This business logic might use the lower-level dependencies from the ZIO environment to implement their functionality.

Modularity and managing dependencies is definitely hard — it’s a topic of many books, articles, libraries and frameworks. Despite the amount work that has gone into tackling the problem, it seems we still keep discovering new approaches.

Great to see ZIO innovating in this area, bringing fresh ideas to the table!

If you’d like o keep exploring the above example on your own, all of the code is on GitHub.