The Platform

---

In the previous chapter we learned how Elixir works, how to use its syntax, and how to write functions and small programs to do some basic stuff. But the real power of Elixir is the platform itself, based on the Erlang ecosystem.

In the introduction we said that one of the most used architectures in Erlang is the actor model, and that everything is a process. Let’s start with the process.

Spawning a process in Elixir is very easy and very cheap. They are not real operating system processes, but processes of the virtual machine in which the Elixir application runs. This is what gives them a light footprint, and it’s quite normal for a real-world application to spawn thousands of processes.

Let’s begin by learning how to spawn a process to communicate with it. Consider this module:

This is a basic “Hello World” example that we can execute just by calling HelloProcess.say("adam"), and it will print Hello adam. In this case, it runs in the same process of the caller:

Here we are using the module as usual, but we can spawn it in a different process and call its functions:

The spawn/3 function runs the say function of the module HelloProcess in a different process. It prints Hello adam and returns a PID (process ID), in this case 0.124.0. PIDs are a central part of the Erlang/Elixir platform because they are the identifiers for the processes.

A PID is composed of three parts: A.B.C.

A is the node number. We have not talked about nodes yet; consider them the machine in which the process runs. 0 stands for the local machine, so all the PIDs that start with 0 are running on the local machine.

B is the first part of the process number, and C is the second part of the process number (usually 0).

Everything in Elixir has a PID, even the REPL:

The self returns the PID of the current process, in this case the REPL (iex).

We can use the PID to inspect a process status using the Process module.

We can try with our HelloProcess module:

As we can see, the process is dead after the execution. This happens because there is nothing that keeps the process alive—it simply puts the string on the console, and then terminates.

The HelloProcess module is not very useful; we need something that do some calculation that we can spawn to another process to keep the main process free.

Let’s write it:

This module is very simple, but we need it to start thinking about process communication. So we can start using this module:

As we can see, it simply returns the PID of the spawned process. In addition, the process dies after the execution, so that we cannot obtain the result of the operation.

To make the two processes communicate, we need to introduce two new instructions: receive and send. Receive is a blocking operation that suspends the process waiting for new messages. Messages are the way process communicates: we can send a message to a process, and it can respond by sending a message.

We can refactor our module like this:

We have defined a start function that is the entry point for this module; we will use this function to spawn the process.

Inside the start function, we wait for a message using the receive do structure. Inside receive, we expect to receive a message (a tuple) with this format:

The format consists of an atom (:sum), an array with an argument for sum, and the pid of the sender. We pattern match on this and respond to the caller using a send instruction.

send/2 needs the pid of the process to send a message: a tuple with :result, and the result (sum of x + y).

If everything is set up correctly, we can load the new module and try it in the REPL:

What have we done here? We loaded the async_math module and spawned the process using the spawn function with the start function of the module.

Now the module is alive because it is waiting for a message (receive…do). We send a message requesting the sum of 1 and 3. The send function returns the sent message, but not the result. In addition to this, the process after the send is dead.

How can we get our result?

One thing I have not yet mentioned is that every process in Elixir has an inbox, a sort of queue in which all of its messages arrive. From that queue, the process dequeues one message at a time, processes it, and then goes to the next one. That’s why I said that inside a process, the code is single thread/single process, because it works on a single message at a time.

This mechanism is also at the base of the actor model, in which each actor has a dedicated queue that stores the messages to process, and an actor works on a single time.

Going back to our example, the queue that stores the messages is the queue of the REPLS, since it is that process that asks for the sum of 1 and 3. We can see what’s inside the process queue by calling the function flush, which flushes the inbox and prints the messages to the console:

Here is our expected result: flush prints the messages in the queue (in this case, just one). The message has the exact shape that we used to send the result.

Now that we have asked for a result, we can try to ask for another operation:

This time the inbox is empty: it seems like our request or the response to our request has gotten lost. The problem is that the AsyncMath.start function wait for the first message, but as soon as the first message is processed, it goes out of scope. The receive do macro does not loop to itself after a message is received.

To obtain the desired result, we must do a recursive call at the end of the start function:

At the end of the receive block, we do a recursive call to start so that the process will go in a “waiting for message” mode.

With this change, we can call the sum operation anytime we want:

When we call the flush function, it prints out the two messages in the inbox with the two results. This happens because the recursive call to start keeps the process ready to receive new messages.

We have seen a lot of new concepts about Elixir and processes.

To recap:

• We created basic module that, when started, waits for a message and responds with another message
• We spawned that function to another process
• We sent a message using the send function
• We flushed the inbox of the REPL to view the result

Is there a better way to capture the result? Yes, by using the same pattern of the AsyncMath module.

We can put our program in waiting even after the execution of the sum operation—remember that the messages remain in the inbox queue, so we can process them after they arrive (unlike with events).

Now we have seen the basics of processes. We also have seen that even if it is cheap to spawn a process, in a real-world application it’s not very feasible create processes and communicate with them using the low-level function that we have seen. We need something more structured and ready to use.

With Elixir and Erlang comes the OTP (Open Telecom Platform), a set of facilities and building blocks for real-world applications. Even though Telecom is in the name, it is not specific to telecommunications— it’s more of a development environment for concurrent applications. OTP was built with Erlang (and in Erlang), but thanks to the complete interoperability between Erlang and Elixir, we can use all of the facilities of OTP in our Elixir program with no cost at all.

Elixir applications

Until now, we’ve worked with simple Elixir script files (.exs). These are useful in simple contexts, but not applicable in real-world applications.

When we installed Elixir, we also got mix, a command-line tool used to create and manipulate Elixir projects. We can consider mix as a sort of npm (from Node.js). From now on, we will use mix to create projects and manage projects.

Let’s create a new project now:

This CLI command creates a new folder named sample_app and puts some files and folders inside.

We’ll now have a quick look at some of these files.

Mix.exs

The mix file acts as a project file (a sort of package.json). It contains the app manifest, the application to launch, and the list of dependencies. Don’t worry if everything is not clear right now—we will learn more as we go.

There are two important things to note here. First is the deps function that returns a list of external dependencies from which the application depends. Dependencies are specified as a tuple with the name of the package (in the form of an atom) and a string that represents the version.

The other important thing is the application function that we will use to specify with module should start at the beginning. In this template, there is only a logger.

Remember that comments start with a number sign (#) character.

Sample_app.ex

Sample_app.ex is the main file of this project, and by default consists only of a function that returns the atom :world. It’s not useful; it’s just a placeholder.

Sample_app_test.exs

This is a template for a simple test of the function SampleApp.hello. It uses ExUnit as a test framework. To run the tests from the terminal, we must write mix test:

The other files are not important right now; we will look more closely at some of them in the upcoming chapters.

To start the application, we must use mix from the terminal:

Actually, the application does nothing.

GenServer

One of the most frequently used modules of OTP is the GenServer that represents a basic generic server, a process that lives by its own and is able to process messages and response to action.

GenServer is a behavior that we can decide to implement to adhere to the protocol. If we do it, we obtain a server process that can receive, process, and respond to messages.

GenServer has the following capabilities:

• Creating a server process
• Managing the server state
• Creating a server process
• Manage the server state
• Handling requests and sending responses
• Stopping the server
• Handling failures

It is a behavior, so the implementation details are up to us. GenServer lets us implement some functions (called callbacks) for customizing its details:

• init/1 acts as a constructor, and is called when the server is started. The expected result is a tuple {:ok, state} that contains the initial state of the server.
• handle_call/3 is the callback that is called when a message arrives to the server. This is a synchronous function, and the result is a tuple {:reply, response, new_state} that contain the response to send to the caller, and the new state to apply to the server.
• handle_cast/2 is the asynchronous callback, which means that it doesn’t have to respond to the caller directly. The expected result is a tuple {:noreply, new_state} that contains the new state.
• handle_info/2 receives all the messages that are note captured (by pattern matching) by the others’ callbacks.
• terminate/2 is called when server is about to exit, and is useful for cleanup.

Our task is to implement the needed callbacks to define the right behavior of our application.

Let’s look at an example to see how GenServer works. We’ll implement a sort of key/value in the memory store (consider it a minimal Redis database implementation).

The server will process two kind of messages:

• {:set, key, value} will store the value in memory assigning the key
• {:get, key}will return the value of the given key

Here is a first basic implementation. The init function defines the initial state, which is basically an empty hash map.

The two hande_call functions handle the two types of messages that are important for the server: set and get a value. We are using the handle_call because we need a synchronous interface to interact with. The implementation of the two function is straightforward. Note that they reply with a tuple: an atom (:reply), a response value :ok for the set operation, the value for the get operation, and the new state.

The new state (in the case of get) is equal to the current state, since get is idempotent.

To try this server, we can open the REPL and do something like this:

Let’s see what we’ve got here. Using the c function, we load the module MiniRedis. Then we start it using the start_link function of GenServer module. The start_link function receives the module to start as a server, and a list of parameters (empty in this case).

The result of start is the :ok atom and the pid of the process. We use the pid to use the server thought the call function that expect the message in the correct format. In this example, we call set and then get to check the result.

Even if everything works perfectly, it’s not that easy to call a GenServer using a pid and know the exact format of the messages to send. That’s why a GenServer usually has a public interface that the client can interact with.

We now modify the first implementation to add the public interface:

This version of our MiniRedis server has three more functions. The start_link, set, and get functions are part of the public interface. This definition comes from the fact that these three functions run in the client process, and not in the server process. Their implementation in fact call the server function using the call function of the GenServer module. The call function sends a message to the real server, and handle_call receives those messages.

It’s also better to use the public interface to interact with the server because the client doesn’t need to know the pid of the server process; it just calls MiniRedis.put(…), which is the implementation of the client interface that manages the identification of the server to call.

In our example, we are using a simple technique to define a GenServer name: we are using the __MODULE__ macro that expands to module name (MiniRedis) as the name of our server. Using the module name means that we can have just one instance of the server running, and in some cases, this is just fine.

Another option is to use the options of the start_link function to specify the server name:

Using an atom to specify a name is another option. The atom can also be passed as a parameter so that we can create different process of the same GenServer.

A more common option is the Registry module that stores the active servers’ names and PIDs. We will see how to use Registry in the next chapter.

Supervisors

Another important component for writing Elixir (and Erlang) applications is the supervisor. Supervisors are special servers whose only task is to monitor another set of processes and decide what to do when they crash or die.

Suppose that our application is composed of a set of GenServers that do some job. Since these GenServers could crash, a supervisor is needed to monitor failures and (for example) restart the servers. That’s the role of a supervisor.

To start using them, we can create a new project using mix and a special flag:

This command creates a new project that starts with a supervisor. The differences are in mix.exs, in which we have a specific application that should start, and the presence of application.ex as the entry point of the application:

Application.ex uses the Application module to create the context of the application. Applications in Elixir (and Erlang) are like libraries or packages in other languages, but since Elixir is a multi-process platform, every library that you use is actually an application (a process by itself).

The interesting part here is the Supervisor.start_link function. The Application module actually acts as a supervisor, and the processes it supervises are in the list of children specified in the start_link function. In the options we specify the restart strategy, which can be one of the following:

• :one_for_one: If a child process terminates, only that process is restarted
• :one_for_all: If a child process terminates, all other child processes are terminated, and then all child processes (including the terminated one) are restarted
• :rest_for_one: If a child process terminates, the terminated child process and the rest of the children started after it, are terminated and restarted

To see the supervisor in action, let’s add a new module to this project. In the /lib/sup_sample folder, we create a new file called my_server.ex that contains this code:

Here we have a simple GenServer that responds to a ping function with a :pong.

After this, we have to change the application.ex to specify which server to start:

The children list contains the name of the module to start and the parameters.

To check that everything is wired correctly, we can use the REPL, specifying the application to start:

The -S mix arguments tell the REPL to start the application using the specifications in the mix.exs file. In our case, it started the application supervisor and the server MyServer.

We can call it using the REPL:

Observer

Inside the REPL, we can use an Erlang application to inspect in the virtual machine and understand which processes are alive, and what they are doing. This application is called Observer. It is started with this instruction:

A new window opens:

Figure 2 - Observer

It is a real desktop application with different tabs to inspect various sections of the current virtual machine. For now, we will investigate the Applications tab to see which applications are active, and to see how they are composed:

Figure 3 – Observer application tab

What we can see here is our application inside the REPL. The two important things here are the SupSample.Supervisor and MyServer that are linked together.

If, we select MyServer, we can see its PID in the status bar, which we can double-click to view more information about the process:

Figure 4 – Observer process details

Observer is a great tool for understanding how our application in configured, and how the processes are linked together. We will see more about Observer in the next chapter.

To see how the restart policy works, we will add a new function to the server to make it crash:

We simply add a new function and its callback to simulate a crash in the server. The crash function calls the handle_call callback that throws an error: Bang!.

As in other programming languages, if the error is not managed, it crashes the entire program. Let’s see what happens here. Open the REPL and execute the crash function:

We started the REPL and called the ping function to check that MyServer is up and running. Then we called the crash function, and the error (in red) is shown with all the stack. But if we call the ping after the error, :pong is returned as if nothing has happened.

This is the magic of the supervisor and the restart policy. We configured the supervisor to restart the process if something bad happens, and it did exactly that.

In fact, the process after the crash is not the same; it’s another process with another pid (we can inspect it using Observer), but from the point of view of the user, nothing has changed. This is one of the powerful features of Elixir (and Erlang). As a developer, you don’t need to evaluate all the possible errors, catch all possible exceptions, and figure out what to do in every possible error case. The philosophy is “let it crash” and let the virtual machine do the work for you: restart the process as if nothing has happened.

There are a couple of things to consider:

• The state of the process is going to be reset at the initial state when the process restarts
• The messages in the inbox are lost: when the process crashes, the inbox crashes with the process

These two facts complicate things a bit…but not by much.

If your process is stateful and needs to restore the state in case of a crash, we can dump the state just before exiting the process. Elixir gives us a special callback that is called just before the process is about to exit. In this callback we have access to the state, and we can serialize it to a database or a file, or something that can be read as soon as the process restarts.

We can add this function to MyServer in our example:

For the messages that are going to be lost in the inbox, we should write servers that are fast enough to keep the inbox empty as much as possible. Also consider the fact the supervisor can be hierarchical, and one supervisor can have other supervisors as children, and multiple servers, too. With this pattern we can scale to infinity, having big batteries of supervisors and servers to manage all the messages that arrives.