Websockets in Scala using sttp
One feature missing from sttp for a long time was support for websockets. With the development of sttp 2, it’s about time to change that!
As a reminder, sttp is a programmer-friendly API for working with HTTP requests. The actual work of sending/receiving the data and managing connections is delegated to a backend, such as akka http, async-http-client or OkHttp.
Backends can have additional capabilities. One such capability is streaming support: some backends don’t support streaming, and those that do, typically support a backend-native type of stream.
Same with websockets: it’s an optional, additional characteristic of an sttp backend. Hence, a backend is now parametrised with three types:
- the effect wrapper (
Future/Task/IO/Identity/…) - supported streams (such as
fs2.Stream,monix.reactive.Observableorakka.stream.scaladsl.Source) - supported websocket handlers
sttp exposes two websocket interfaces:
- backend-native
- a higher-level, more “Scala-like” and “functional”
WebSocket[F]interface
Let’s look at both of these approaches in detail!
1. Backend-native websockets
Most backends offer natively some kind of websocket support. However, the interfaces they expose for communicating with websockets are different in each case. That’s why the SttpBackend is parametrised with a WS_HANDLER[_] type constructor, which is arbitrary and unconstrainted.
Such a handler accepts a single type as a parameter: it’s the type of the value that is returned when a websocket is established. This type is usually roughly determined by the backend and the handler, but can vary from request to request.
For backends which don’t support websockets, the
WS_HANLDERparameter will have theNothingTvalue, which is a simple type alias:type NothingT[X] = Nothing.
Let’s explore two backends in more detail.
akka-http websocket support
akka-http supports websockets using (reactive) streams. Such a stream processes incoming & produces outgoing websocket messages. The type of the handler in this case is Flow[Message, Message, T], where the T parameter is the value to which the stream is materalised.
Hence, to open a websocket using an sttp-defined request, we need to pass in a Flow instance. As an example, a simple handler which sends a message each second and prints each received response to the console would be:
Most of the code above is setting up akka, and creating the Flow: both of which have nothing to do with sttp, but rather with akka-streams. Only in the end, you can see that an sttp request is used to open a websocket.
The request is defined as any other, “regular” sttp request. In fact, both normal and websocket requests are defined in exactly the same way; the only difference is that in the end, you need to call the openWebsocket method, instead of send.
async-http-client websocket support
If you are using async-http-client-based backends (which come in different flavors, with the effect wrapper being Monix’s Task, ZIO’s Task, cats-effect F[_]: Effect or Scala’s Future), you can use async-http-client native websockets as well.
In this case, the handler is an implementation of the org.asynchttpclient.ws.WebSocketListener interface. It’s a Java interface, and it also has a “Java-like” feeling: interacting with the websocket is done in an imperative, procedural way.
Let’s see how this looks in practice. We need to create a listener, which will be called once websocket messages have arrived. We also get at our disposal a org.asynchttpclient.ws.WebSocket instance, which can be used to send messages:
Note that the listener callbacks are executed on Netty’s threadpool: you should be cautious not to run any blocking operations there, and essentially “escape” from that thread as soon as possible, not to block networking operations.
OkHttp websocket support
OkHttp websockets are supported in a nearly identical way as in the case of async-http-client. The major difference is that instead of a org.asynchttpclient.ws.WebSocketListener instance, we need to provide an instance of okhttp3.WebSocketListener.
This interface is slightly different, but the basic idea is the same.
OkHttp backends come in three flavors: synchronous (without an effect wrapper), Future-based and monix.eval.Task-based.
2. High-level websocket interface
When using an async-http-client backend in the Monix-Task or ZIO-Task flavors, you can also use a more Scala-like, functional interface, based on lazily evaluated tasks.
Using this approach, you need to use a MonixWebSocketHandler or ZioWebSocketHandler, which, after the websocket is established, will result in a sttp.client.ws.WebSocket[Task] instance. This gives access to two methods:
def receive: Task[Either[Close, WebSocketFrame.Incoming]]def send(f: WebSocketFrame): Task[Unit]
The first describes a process which will result in receiving a single message or information that the websocket has been closed. Note that if no messages are available, the calling fiber will be suspended (in a non-thread-blocking-way).
The second describes a process, which will complete once the given frame has been sent on the websocket.
When creating the web socket handler, we can optionally set a limit on the incoming message buffer size: in case messages arrive, but are never received using
Websocket.receive, we might want to either exhaust all available memory, or fail the process at some point.
How does our example look using this approach? Let’s find out! This time, error handling is built-in; if anything goes wrong, the resulting Task will fail:
Summing up
sttp2 brings a new feature: web socket support. You can either use a backend-native websocket inferface, such as Flow from akka-streams, or WebSocketListener from async-http-backend and OkHttp. Or, you can use sttp’s WebSocket[F] interface, to describe asynchronous processes which send and receive messages using websockets.
All of the above examples are self-contained, and need the appropriate sttp backend module. You can try this out using sttp 2.0.0-M6 (documentation is available).
If you have any comments on the design of the feature, or other ideas, please let us know!
