In this post, we'll be demystifying GenServers. We'll discuss its anatomy and then later roll out our own version of GenServer from the ground up.
To fully grasp this article, you must at least know the basics of Elixir/Erlang processes.
GenServer in Action
If you already know how to use GenServers, feel free to skip this section and jump straight to Anatomy of a GenServer.
GenServer is a behavior that facilitates client-server interaction. It abstracts away all the nitty-gritty details when dealing with interprocess communications.
In this example, we'll build an item accumulator. It would be an Elixir process that accepts an item and stores it in a list. This is how it would look like if we were to use the process primitive:
iex(1)> pid = spawn(fn ->
...(1)> receive do
...(1)> {sender, acc, item} -> send(sender, [item | acc])
...(1)> end
...(1)> end)
iex(2)> send pid, {self, [], "apple"}
{
iex(3)> flush
["apple"]
As you may have noticed, we would need to pass a couple of things around. We would also need to make sure that it's a long-living process (we'll discuss more about this later). It's easy to get lost within this code. This is why GenServer behavior exists.
Let's rewrite this using GenServer. To start, we need to define a module that implements the GenServer behaviour.
defmodule ItemAccumulator do
use GenServer
end
First, it should be able to set an initial state, in our case, an initial accumulator. To do this, we need to implement the init/1 callback.
@impl true
def init(acc) do
{:ok, acc}
end
Our ItemAccumulator will have two functionalities. We need to have a function that would store the item and another function to return the items. When storing an item, we don't need a response; hence, we can use GenServer.cast/2. On our GenServer module, we then need to handle the cast request using the handle_cast/2 callback.
@impl true
def handle_cast({:add, item}, acc) do
{:noreply, [item | acc]}
end
On the other hand, when we ask for the list of items in the accumulator, we need to wait for a response from the server. Because of this, we need to use GenServer.call/3. On our GenServer module, we would need to implement the handle_call/3 callback.
Lastly, let's also show the accumulated items before the process exits.
@impl true
def terminate(_reason, acc) do
IO.puts "Last state: #{inspect(acc)}"
end
This is how our GenServer module should look like:
defmodule ItemAccumulator do
use GenServer
@impl true
def init(acc) do
{:ok, acc}
end
@impl true
def handle_call(:list_items, from, acc) do
{:reply, acc, acc}
end
@impl true
def handle_cast({:add, item}, acc) do
{:noreply, [item | acc]}
end
@impl true
def terminate(_reason, acc) do
IO.puts "Last state: #{inspect(acc)}"
end
end
Let us test this out.
iex(1)> GenServer.start_link(ItemAccumulator, [], name: ItemAccumulator)
{:ok,
The start_link/3 function starts the GenServer process. Giving it a name would register the process and would make it easier for us to locate and use it.
iex(2)> GenServer.cast(ItemAccumulator, {:add, "Apples"})
:ok
iex(3)> GenServer.cast(ItemAccumulator, {:add, "Oranges"})
:ok
iex(4)> GenServer.call(ItemAccumulator, :list_items)
["Oranges", "Apples"]
iex(5)> GenServer.stop(ItemAccumulator)
Last state: ["Oranges", "Apples"]
:ok
Since we have a working GenServer, we can now stop here or make the interface more developer-friendly. We can do that by abstracting the implementation details away from the client. They don't always need to know the implementation details, in our case, that we're using GenServers.
defmodule ItemAccumulator do
use GenServer
def start_link(state) do
GenServer.start_link(__MODULE__, state, name: __MODULE__)
end
def add(item) do
GenServer.cast(__MODULE__, {:add, item})
end
def list_items(), do: GenServer.call(__MODULE__, :list_items)
...
end
Let's try it out in the shell.
iex(1)> ItemAccumulator.start_link([])
{:ok,
iex(2)> ItemAccumulator.add("Apples")
:ok
iex(3)> ItemAccumulator.add("Oranges")
:ok
iex(4)> ItemAccumulator.list()
["Oranges", "Apples"]
The interface now looks cleaner and just focuses on the functionalities instead of implementation details.
Anatomy of a GenServer
A GenServer is just a regular Elixir/Erlang process that is stuck in a loop. Don't believe me? Here's what the observer shows when you're running a GenServer process.
It's an Erlang process that waits for a message, acts on the message, then loops and waits for a message again.
Take a look at the snippet below.
defmodule HealthCheck do
def loop() do
receive do
{pid, :ping} ->
send(pid,:pong)
loop()
:exit ->
Process.exit(self(), :normal)
end
end
end
We define a module called HealthCheck that has a function that just responds to a ping message and then loops. Let's use this module to build our first dumb version of our generic server.
iex(1)> pid = spawn(&HealthCheck.loop/0)
We now have a running generic server. To confirm that it's indeed alive, verify it using Process.alive?/1.
iex(2)> Process.alive?(pid)
true
Let's do a ping check and see if our server responds.
iex(3)> send(pid, {self(), :ping})
{
iex(4)> Process.info(self(), :messages)
{:messages, [:pong]}
Now confirm if our generic server is still alive.
iex(5)> Process.alive?(pid)
true
We just built a long-running process. This is GenServer under the hood, with some quirks sprinkled on top of it.
Now let's build our own version of a GenServer!
BasicServer
Requirements
In this mini-project, we aim to replicate the following functionalities of a GenServer:
init - We should be able to set the initial state.
call - We should be able to make synchronous calls to the server and wait for a reply.
cast - We should be able to send fire-and-forget calls to the server.
stop - We should be able to stop the server.
GenServers have other functionalities like multicast, multicall, etc but we'll only focus on the functionalities enumerated above.
__using__ Macro
The __using__ macro allows us to inject code into another module. We need this so that we can set the behavior for the server module and implement default callbacks.
defmodule BasicServer do
@callback init(state :: term()) :: {:ok, term()} | {:stop, term()}
defmacro __using__(_opts) do
quote do
@behaviour BasicServer
end
end
end
Setting Initial State
Setting the initial state of a GenServer is done via the init/1 callback. A module that is using our BasicServer must have an init/1 callback defined; otherwise, we raise an exception.
...
defmacro __using__(_opts) do
quote do
@behaviour BasicServer
def init(_state) do
raise "you forgot to implement the init callback!"
end
defoverridable init: 1
end
end
...
Starting the Server
Now that we're enforcing the presence of an init/1 callback, we can proceed with building our first function which is starting the BasicServer.
defmodule BasicServer do
@callback init(state :: term()) :: {:ok, term()} | {:stop, term()}
...
def start_link(callback_mod, init_state, opts \\ []) do
pid =
spawn_link(fn ->
{:ok, state} = apply(callback_mod, :init, [init_state])
loop(callback_mod, state)
end)
if opts[:name], do: Process.register(pid, opts[:name])
{:ok, pid}
end
defp loop(callback, state) do
receive do
_ -> loop(callback, state)
end
end
...
end
Synchronous Calls
In this section, we will be building the call part of GenServers. call is an asynchronous operation. It will not proceed with other actions until it receives a reply or it times out. How are we going to build this? call will send a message to our server process which is in a suspended state and just waiting for a message to arrive. Once our server process receives the message, it will call the callback module's handle_call/3 function, passing in the message and current state. The calling process would then enter a suspended state waiting for a reply.
Let's create the call/2 function.
...
def call(pid, message) do
send(pid, {:call, self(), message})
receive do
{:reply, reply} ->
reply
end
end
...
Now, we modify the receive-block inside the looping function to handle the call message and call the appropriate callback function.
...
defp loop(callback, state) do
receive do
{:call, caller_pid, message} ->
{:reply, reply, new_state} = apply(callback, :handle_call, [message, caller_pid, state])
send(caller_pid, {:reply, reply})
loop(callback, new_state)
end
end
...
Asynchronous Casts
Let's deal with the fire-and-forget requests. This is very similar to the call function, except that we don't need to send a reply back to the calling process.
...
def cast(pid, message) do
send(pid, {:cast, message})
end
...
Handling the cast message is also simpler. We just need to call the callback function.
...
defp loop(callback, state) do
receive do
{:call, caller_pid, message} ->
{:reply, reply, new_state} = apply(callback, :handle_call, [message, caller_pid, state])
send(caller_pid, {:reply, reply})
loop(callback, new_state)
{:cast, message} ->
{:noreply, new_state} = apply(callback, :handle_cast, [message, state])
loop(callback, new_state)
end
end
...
Hey Server, stop!
Finally, the last remaining function, stop. This function would just tell our server to exit. Well, it needs to do some cleanup first before exiting. Let's start by implementing the stop function.
...
def stop(server, reason) do
send(server, {:stop, reason})
:ok
end
...
Hmm... why do we need to send a message to the server? Remember, the server is in a suspended state waiting for a message. Sure, we can just kill the server process, but that's not a graceful exit. Once the server receives the exit message, it executes the terminate/2 callback and then escapes the loop; thus, terminating the server process.
...
defp loop(callback, state) do
receive do
{:call, caller_pid, message} ->
{:reply, reply, new_state} = apply(callback, :handle_call, [message, caller_pid, state])
send(caller_pid, {:reply, reply})
loop(callback, new_state)
{:cast, message} ->
{:noreply, new_state} = apply(callback, :handle_cast, [message, state])
loop(callback, new_state)
{:stop, reason} ->
apply(callback, :terminate, [reason, state])
end
end
...
Testing Our Basic Server
Here's the full code of our GenServer clone, BasicServer.
defmodule BasicServer do
@callback init(state :: term()) :: {:ok, term()} | {:stop, term()}
@callback handle_call(msg :: term(), from :: pid(), state :: term()) ::
{:reply, reply :: term(), state :: term()}
@callback handle_cast(msg :: term(), state :: term()) :: {:noreply, state :: term()}
@callback terminate(reason :: atom(), state :: term()) :: :ok
defmacro __using__(_opts) do
quote do
@behaviour BasicServer
def init(_state) do
raise "you forgot to implement the init callback!"
end
defoverridable init: 1
end
end
def start_link(callback_mod, init_state, opts \\ []) do
pid =
spawn_link(fn ->
{:ok, state} = apply(callback_mod, :init, [init_state])
loop(callback_mod, state)
end)
if opts[:name], do: Process.register(pid, opts[:name])
{:ok, pid}
end
def call(pid, message) do
send(pid, {:call, self(), message})
receive do
{:reply, reply} ->
reply
end
end
def cast(server, message) do
send(server, {:cast, message})
:ok
end
def stop(server, reason) do
send(server, {:stop, reason})
:ok
end
defp loop(callback, state) do
receive do
{:call, caller_pid, message} ->
{:reply, reply, new_state} = apply(callback, :handle_call, [message, caller_pid, state])
send(caller_pid, {:reply, reply})
loop(callback, new_state)
{:cast, message} ->
{:noreply, new_state} = apply(callback, :handle_cast, [message, state])
loop(callback, new_state)
{:stop, reason} ->
apply(callback, :terminate, [reason, state])
end
end
end
We'll reuse the ItemAcc we wrote before but instead of using GenServer, we use BasicServer.
defmodule ItemAcc do
use BasicServer
def start_link(state) do
BasicServer.start_link(__MODULE__, state, name: __MODULE__)
end
def add(item) do
BasicServer.cast(__MODULE__, {:add, item})
end
def list_items(), do: BasicServer.call(__MODULE__, :list_items)
def stop(), do: BasicServer.stop(__MODULE__, :normal)
@impl true
def init(state), do: {:ok, state}
@impl true
def handle_call(:list_items, _from, acc) do
{:reply, acc, acc}
end
@impl true
def handle_cast({:add, item}, acc) do
{:noreply, [item | acc]}
end
@impl true
def terminate(_reason, state) do
IO.puts "Terminating... Last state: #{inspect(state)}"
end
end
iex(1)> ItemAcc.start_link([])
{:ok,
iex(2)> ItemAcc.add("Apples")
:ok
iex(3)> ItemAcc.add("Oranges")
:ok
iex(4)> ItemAcc.list_items()
["Oranges", "Apples"]
iex(5)> ItemAcc.stop()
Terminating... Last state: ["Oranges", "Apples"]
:ok
We are now done implementing the core functionalities of a GenServer. I think it's important to highlight that you should not build your own GenServer. Elixir's GenServers are battle-tested already. You most likely don't need to reinvent the wheel. We only did it for educational purposes. If you need client-server communication, then just use the already-built GenServer.
If you enjoyed this insightful post and would like to receive more content like this, I warmly invite you to subscribe to my newsletter. Your support is truly appreciated! See you on the next one!
