Understanding Process Linking and Exit Signals

In the previous part of this series, we explored process links. A linked worker that crashes sends an exit signal to the process linked to it. By default, that failure propagates and can terminate both processes. This default behavior, while simple, is often too brittle for robust distributed systems. We need a more sophisticated way to manage these failures.

Elixir processes can be configured to 'trap exits'. When a process traps exits, an incoming exit signal is not immediately acted upon to terminate the process. Instead, it's delivered to the process's mailbox as a message. This message typically takes the form of {:EXIT, pid, reason}, where pid is the identifier of the process that terminated, and reason explains why it terminated (e.g., :normal, :killed, or an exception term).

To enable exit trapping, a process can use the Process.flag(:trap_exit, true) function. This function returns the previous state of the exit trapping flag.

Process.flag(:trap_exit, true)

Once exit trapping is enabled, the process receiving the exit signal can inspect the {:EXIT, pid, reason} message and decide how to respond. It could restart the failed process, log the error, or take other corrective actions. This manual handling of exit signals is the fundamental building block for creating more resilient systems, paving the way for supervisors.

Implementing a Basic Supervisor

A supervisor's primary role is to monitor its child processes and restart them when they crash. This is a critical pattern for building fault-tolerant distributed systems. We can build a simplified supervisor from scratch to understand its core mechanics.

Let's start by defining a supervisor process. This process will accept a list of child specifications. Each specification will define how to start and monitor a child process. For simplicity, our initial supervisor will just start a single worker process.

Consider a supervisor that monitors a simple worker. The worker might be a process that performs a task and then exits. The supervisor, upon detecting the worker's exit, will restart it. This is a basic form of the 'restart strategy'.

We can define the supervisor's behavior using a module, say MySupervisor. The start_link function would initiate the supervisor process. The supervisor's init function would be responsible for setting up the monitoring. Inside the supervisor's handle_info function, we'll specifically look for the {:EXIT, pid, reason} messages.

defmodule MySupervisor do
  use GenServer

  def start_link(child_spec) do
    GenServer.start_link(__MODULE__, child_spec, [name: __MODULE__])
  end

  def init(child_spec) do
    # Enable exit trapping for the supervisor itself
    Process.flag(:trap_exit, true)

    # Start the child process
    case child_spec do
      {worker_module, args} do
        {:ok, pid} = worker_module.start_link(args)
        Process.link(pid)
        {:ok, pid}
      _ do
        {:error, :invalid_child_spec}
      end
    end
  end

  def handle_info({:EXIT, _pid, reason}, child_spec) do
    IO.puts("Child process exited with reason: #{inspect(reason)}")
    # Restart the child process
    {:ok, new_pid} = case child_spec do
      {worker_module, args} do
        worker_module.start_link(args)
      end
    end
    Process.link(new_pid)
    {:noreply, child_spec}
  end

  def handle_info(msg, state) do
    IO.puts("Received unexpected message: #{inspect(msg)}")
    {:noreply, state}
  end
end

The 'One For One' Restart Strategy

The example above implements a rudimentary 'one for one' restart strategy. This means that if one child process crashes, only that specific child is restarted. Other children are unaffected. This is the most common and straightforward restart strategy. The supervisor simply waits for an exit signal from a linked child, and upon receiving it, it executes the logic to start a new instance of that child.

To make this concrete, let's define a simple worker module:

defmodule MyWorker do
  use GenServer

  def start_link(initial_state) do
    GenServer.start_link(__MODULE__, initial_state, [name: __MODULE__])
  end

  def init(initial_state) do
    Process.flag(:trap_exit, true) # Workers typically don't trap exits unless they are also supervisors
    {:ok, initial_state}
  end

  def handle_call(:get_state, _from, state) do
    {:reply, state, state}
  end

  def handle_cast(message, state) do
    IO.puts("Worker received message: #{inspect(message)}")
    # Simulate a crash for demonstration
    if message == :crash do
      raise "Simulated crash!"
    end
    {:noreply, state ++ [message]}
  end
end

Now, we can start our custom supervisor and worker:

# In iex:
# Start the worker with initial state []
MySupervisor.start_link({MyWorker, []})

# Send a message to the worker
GenServer.cast(MyWorker, "hello")

# Simulate a crash
GenServer.cast(MyWorker, :crash)

When :crash is sent, MyWorker will raise an exception. Because the supervisor is linked to the worker and the worker is not trapping exits, the supervisor will receive the {:EXIT, pid, reason} message. Our MySupervisor's handle_info will catch this, print a message, and then restart MyWorker. You'll see the output indicating the child process exited and then the worker process starting up again.

Beyond Basic Restart: Supervisor Strategies

The 'one for one' strategy is a good starting point. However, Elixir's built-in Supervisor module offers more sophisticated strategies like:

  • One for all: If one child crashes, all other children are terminated, and then all are restarted. This is useful when child processes are tightly coupled and cannot function independently.
  • Rest for one: If a child crashes, only that child and any children started after it are restarted. This is useful for hierarchical structures where children depend on their siblings started earlier.

Implementing these strategies from scratch would involve more complex logic within the supervisor's handle_info function to determine which children to terminate and restart based on the crash event and the defined strategy. It would also require the supervisor to keep track of all its children's PIDs and their startup order.

For instance, a 'rest for one' strategy would require the supervisor to maintain a list of its children in the order they were started. When a child crashes, the supervisor would iterate through the list from the crashed child onwards, sending exit signals to those processes before restarting the crashed child and subsequent ones.

The core principle remains the same: the supervisor must trap exits, receive the {:EXIT, pid, reason} message, and then execute a predefined set of actions based on the crash and the configured strategy. This manual implementation highlights the elegance and necessity of Elixir's built-in supervision trees for managing complex distributed systems.

The Importance of Supervision Trees

Building supervisors from scratch, even simplified ones, reveals the power of Elixir's concurrency model. Supervisors form the backbone of fault tolerance in Elixir applications. They ensure that your system can recover from unexpected failures, which are inevitable in distributed environments. By defining clear restart strategies and child specifications, developers can build applications that are resilient and self-healing. The ability to trap exits and manually handle process termination is the foundation upon which these robust fault-tolerance mechanisms are built, making Elixir a preferred choice for building highly available systems.