Every journey into learning a programming language starts with a “Hello World.” I remember my first time compiling and executing my first program. It was exhilarating. It gave me a sense of accomplishment… even though it’s just a terminal that shows “Hello World.” Now, let’s revisit the same problem, equipped with the knowledge and experience we've gained along the way.

The Problem

Hello World in programming is just printing the phrase “Hello World”. Here’s how to do it in some languages:

// Elixir
IO.puts("Hello World")

// Python
print("Hello World")

// Ruby
puts "Hello World"

// Java
System.out.println("Hello World");

But, where’s the fun in that? In this article, we’ll be building a distributed Hello World system:

• multiple distributed nodes

• auto-discovery of nodes

• whenever a node joins, all the other nodes will send a “hello world” message to it

Project Setup

To kick off, let’s create an Elixir project with a supervision tree and application callback:

➜ mix new distributed_hello_world --sup

We want to generate it like that since we will be creating GenServers that executes instructions on application boot up.

Node Auto-discovery

One of our requirements is for nodes to automatically discover and connect to each other. There should be no manual Node.connect/1 calls whenever there’s a new node. There are couple of libraries we could use for this like libcluster which supports a bunch of strategies. If you’re looking for a robust and well-tested clustering mechanism, you should take a look at those. But since we are overengineering here, we would roll our own using the idea from libcluster’s Gossip strategy.

How it works

For our clustering mechanism, we’ll use UDP broadcast for node discovery. We’ll use UDP (User Datagram Protocol) to send packets over the network without needing a connection. It’s perfect for fire-and-forget type of messages like “Hey, I’m here”.

When we use UDP broadcast, we send a packet to everyone on the local network by targeting the limited broadcast address (255.255.255.255). All nodes that listen to the same UDP port will receive the message.

NodeManager

Since we have our strategy laid out, let’s start building it. We’ll be dealing with UDP so we need to use Erlang’s gen_udp module. To send and receive UDP packets, we need to call :gen_udp.open/2 which opens a UDP socket bound to a port. We need to make sure that all the nodes send and listen to the same UDP port. For this example, we’ll use the port 45826.

Once we have the socket, we would be able to broadcast messages via :gen_udp.send/4.

Let’s play with it a little bit. Fire up your IEx shell.

iex(1)> {:ok, socket} = :gen_udp.open(45826, [:binary, active: true, broadcast: true, reuseport: true])
{:ok, #Port<0.3>}
iex(2)> :gen_udp.send(socket, {255, 255, 255, 255}, 45826, "HELLO WORLD")
:ok
iex(3)> flush
{:udp, #Port<0.3>, {192, 168, 68, 61}, 45826, "HELLO WORLD"}
:ok

Once you open a UDP socket, it associates the port number to the calling process so all the packets will arrive in the calling process’ mailbox.

We want the node auto-discovery and connection to happen on application startup. To achieve this, we’ll create a GenServer that opens a UDP socket upon initialization.

defmodule DistributedHelloWorld.NodeManager do
  use GenServer

  @port 45826
  @heartbeat_interval 2_000

  def start_link(_) do
    GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
  end

  def init(_) do
    {:ok, udp_socket} =
      :gen_udp.open(@port, [:binary, active: true, broadcast: true, reuseport: true])

    {:ok, %{socket: udp_socket}, {:continue, :heartbeat}}
  end
end

Now that the UDP socket is open, we can start sending and receiving packets. Since UDP messages arrive in the calling process’ mailbox, we can use handle_info/2 to receive it. Let’s also create a catch-all handle_info/2 and inspect its messages.

defmodule DistributedHelloWorld.NodeManager do
  ...

  def handle_continue(:heartbeat, state) do
    send(self(), :heartbeat)
    {:noreply, state}
  end

  def handle_info(:heartbeat, state) do
    :gen_udp.send(state.socket, {255, 255, 255, 255}, @port, :erlang.term_to_binary(node()))

    Process.send_after(self(), :heartbeat, @heartbeat_interval)

    {:noreply, state}
  end

  def handle_info(msg, state) do
    IO.inspect(msg)
    {:noreply, state}
  end

  ...
end

Run the app while providing a name:

➜ iex --sname alice -S mix

Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:14:14] [ds:14:14:10] [async-threads:1] [jit]

Compiling 1 file (.ex)
Interactive Elixir (1.17.2) - press Ctrl+C to exit (type h() ENTER for help)
{:udp, #Port<0.5>, {192, 168, 68, 61}, 45826,
 <<131, 119, 10, 97, 108, 105, 99, 101, 64, 90, 101, 117, 115>>}

As you may have noticed, the calling node also received the UDP message. This is because we are broadcasting it to everyone on the local network… including us. Let’s handle the message. Since we now have the node name, we should be able to connect to it.

defmodule DistributedHelloWorld.NodeManager do
  use GenServer

  ...

  def handle_info({:udp, _, _, _, node_bin}, state) do
    node = :erlang.binary_to_term(node_bin)

    if node() != node && node not in Node.list() do
      Node.connect(node)
    end

    {:noreply, state}
  end

  ...
end

Make sure to add this GenServer to the supervision tree.

defmodule DistributedHelloWorld.Application do
  use Application

  @impl true
  def start(_type, _args) do
    children = [
      DistributedHelloWorld.NodeManager
    ]

    opts = [strategy: :one_for_one, name: DistributedHelloWorld.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Let’s test this out by spinning up two nodes, alice and bob. They should be connected automatically.

# Node: alice
➜ iex --sname alice -S mix
iex(alice@Zeus)1> Node.list()
[:bob@Zeus]

# Node: bob
➜ iex --sname bob -S mix
iex(bob@Zeus)1> Node.list()
[:alice@Zeus]

A bit of a problem…

Run alice node again then open up another terminal. On this new terminal, issue this command:

➜ echo "Test" | nc -u 127.0.0.1 45826

You’ll see that there’s an error that happened in alice node.

➜ iex --sname alice -S mix
Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:14:14] [ds:14:14:10] [async-threads:1] [jit]

Interactive Elixir (1.17.2) - press Ctrl+C to exit (type h() ENTER for help)

13:20:04.779 [error] GenServer DistributedHelloWorld.NodeManager terminating
** (ArgumentError) errors were found at the given arguments:

  * 1st argument: invalid external representation of a term

    :erlang.binary_to_term("Test\n")
    (distributed_hello_world 0.1.0) lib/distributed_hello_world/node_manager.ex:32: DistributedHelloWorld.NodeManager.handle_info/2
    (stdlib 6.0) gen_server.erl:2173: :gen_server.try_handle_info/3
    (stdlib 6.0) gen_server.erl:2261: :gen_server.handle_msg/6
    (stdlib 6.0) proc_lib.erl:329: :proc_lib.init_p_do_apply/3
Last message: {:udp, #Port<0.5>, {127, 0, 0, 1}, 62202, "Test\n"}
State: %{socket: #Port<0.5>}

This happens because our app listens to a specific UDP port. Our app will receive all the packets sent to that port. We can solve this by scoping. To scope the message, we’ll just prefix it with node::.

...
  def handle_info(:heartbeat, state) do
    :gen_udp.send(
      state.socket,
      {255, 255, 255, 255},
      @port,
      "node::" <> :erlang.term_to_binary(node())
    )

    Process.send_after(self(), :heartbeat, @heartbeat_interval)

    {:noreply, state}
  end

  def handle_info({:udp, _, _, _, <<"node::", node_bin::binary>>}, state) do
    node = :erlang.binary_to_term(node_bin)

    if node() != node && node not in Node.list() do
      Node.connect(node)
    end

    {:noreply, state}
  end

  def handle_info({:udp, _socket, _ip, _port, _}, state) do
    {:noreply, state}
  end
...

Now, our app only listens and reacts to packets that are prefixed with node:: and ignores the others.

Here’s the full NodeManager code:

defmodule DistributedHelloWorld.NodeManager do
  use GenServer

  @port 45826
  @heartbeat_interval 2_000

  def start_link(_) do
    GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
  end

  def init(_) do
    {:ok, udp_socket} =
      :gen_udp.open(@port, [:binary, active: true, broadcast: true, reuseport: true])

    {:ok, %{socket: udp_socket}, {:continue, :heartbeat}}
  end

  def handle_continue(:heartbeat, state) do
    send(self(), :heartbeat)
    {:noreply, state}
  end

  def handle_info(:heartbeat, state) do
    :gen_udp.send(
      state.socket,
      {255, 255, 255, 255},
      @port,
      "node::" <> :erlang.term_to_binary(node())
    )

    Process.send_after(self(), :heartbeat, @heartbeat_interval)

    {:noreply, state}
  end

  def handle_info({:udp, _, _, _, <<"node::", node_bin::binary>>}, state) do
    node = :erlang.binary_to_term(node_bin)

    if node() != node && node not in Node.list() do
      Node.connect(node)
    end

    {:noreply, state}
  end

  def handle_info({:udp, _socket, _ip, _port, _}, state) do
    {:noreply, state}
  end
end

Nice! All nodes automatically discover and connect to each other. Now let’s move on to the next part, the Greeter.

Hello World, folks

Once a new node joins a cluster, it will greet all the nodes and get some greetings too. Just like the previous part, we’ll also be building a GenServer.

The Greeter

For us to know when a node joins the cluster, we need to monitor the nodes. We can do this by using :net_kernel.monitor_nodes(true).

defmodule DistributedHelloWorld.Greeter do
  use GenServer

  require Logger

  def start_link(_) do
    GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
  end

  def init(_arg) do
    :net_kernel.monitor_nodes(true)

    {:ok, %{}}
  end

  def handle_info(msg, state) do
    IO.inspect(msg)

    {:noreply, state}
  end
end

Let’s see what messages we receive when we monitor the nodes.

# Node: alice
iex --sname alice -S mix
{:nodeup, :bob@Zeus}

# Node: bob
➜ iex --sname bob -S mix
{:nodeup, :alice@Zeus}

# alice disconnects...
{:nodedown, :alice@Zeus}

To fulfill the requirements, we just need to send a greeting message to the node when we receive the :nodeup message. And whenever we receive the :greet message, we print it via Logger along with the name of the node that sent it.

defmodule DistributedHelloWorld.Greeter do
  use GenServer

  ...

  def handle_info({:nodeup, node}, state) do
    GenServer.cast({__MODULE__, node}, {:greet, node()})

    {:noreply, state}
  end

  def handle_cast({:greet, node}, state) do
    Logger.info("Hello world from " <> inspect(node))

    {:noreply, state}
  end

  ...
end

Here’s the full code of the Greeter module.

defmodule DistributedHelloWorld.Greeter do
  use GenServer

  require Logger

  def start_link(_) do
    GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
  end

  def init(_arg) do
    :net_kernel.monitor_nodes(true)

    {:ok, %{}}
  end

  def handle_info({:nodeup, node}, state) do
    GenServer.cast({__MODULE__, node}, {:greet, node()})

    {:noreply, state}
  end

  def handle_info({:nodedown, _}, state) do
    {:noreply, state}
  end

  def handle_cast({:greet, node}, state) do
    Logger.info("Hello world from " <> inspect(node))

    {:noreply, state}
  end
end

Just like the NodeManager, make sure to add it to the supervision tree.

defmodule DistributedHelloWorld.Application do
  ...

  def start(_type, _args) do
    children = [
      DistributedHelloWorld.Greeter,
      DistributedHelloWorld.NodeManager
    ]

    opts = [strategy: :one_for_one, name: DistributedHelloWorld.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Let’s test our Distributed Hello World app by spinning up nodes alice and bob.

➜ iex --sname alice -S mix
Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:14:14] [ds:14:14:10] [async-threads:1] [jit]

Compiling 1 file (.ex)
Interactive Elixir (1.17.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(alice@Zeus)1>

Nothing happened yet. Now let’s observe what happens when we run bob.

➜ iex --sname bob -S mix
Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:14:14] [ds:14:14:10] [async-threads:1] [jit]

Interactive Elixir (1.17.2) - press Ctrl+C to exit (type h() ENTER for help)

00:59:02.539 [info] Hello world from :alice@Zeus
iex(bob@Zeus)1>

Go back to alice and you’ll notice that it also received a greeting from bob.

# Node: alice
00:59:02.538 [info] Hello world from :bob@Zeus

Now let’s see what happens when a third node joins the cluster. Enter charlie…

➜ iex --sname charlie -S mix
Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:14:14] [ds:14:14:10] [async-threads:1] [jit]

Interactive Elixir (1.17.2) - press Ctrl+C to exit (type h() ENTER for help)

01:02:19.497 [info] Hello world from :alice@Zeus

01:02:19.498 [info] Hello world from :bob@Zeus
iex(charlie@Zeus)1>

It got greetings from both alice and bob. Go back to both alice and bob nodes and see how it reacted.

# Node: alice
01:02:19.494 [info] Hello world from :charlie@Zeus

# Node: bob
01:02:19.498 [info] Hello world from :charlie@Zeus

Wrapping Up

So there you have it — a multi-node Hello World.

What did we actually do here?

• built our own node-discovery mechanism via UDP broadcast

• Node monitoring

• an app that says hello to everyone in the cluster

Was it overkill? Absolutely — and that’s the point of this series!

Again, this is just for fun and learning. If you need a robust node-discovery mechanism, check out libcluster.

Link to full source code: https://github.com/vinceurag/distributed_hello_world

Awesome! I hope you had fun reading and following along this first Overengineered post. I’ll be making more of this so if you want this kind of content, feel free to subscribe to my newsletter. For suggestions/feedback, please use this form. If you want to join the fun and take a jab at the problem yourself, please tag me — I'd love to see what you come up with!

Happy overengineering and see you on the next one!

What is PubSub?

PubSub (Publish/Subscribe) is a design pattern used for asynchronous communication between services. Instead of a process sending a message directly to another process, the publisher broadcasts a message with a topic and let susbcribers of that topic consume the message. This allows us to build loosely coupled services. To visualize this, let’s take a look at a real-world example, radio stations.

In this example, a radio station is considered a publisher. It broadcasts music, podcasts, or whatever through its assigned frequency. When we want to listen to a banger music, we tune in to our favorite radio station’s frequency. The radio station is not directly playing the music to us. It’s just publishing (broadcasting) to its assigned frequency, and us, being avid listeners, are subscribed (tuned in) to that frequency.

Local PubSub

Now, let us build our local PubSub. In this article, we’ll be naming our PubSub system Hermes, for obvious reasons.

mix new hermes --sup

In a PubSub pattern, we need a mechanism where we could determine which processes are subscribed to which topics. In other words, we need to have a key-value storage for our processes. Fortunately, Elixir already provides this under the Registry module.

Registry

Registry is a local key-value process storage. It allows you to register the current process under a given key. To understand it better, let’s play with it.

First, let’s start the Registry process.

iex(1)> Registry.start_link(keys: :duplicate, name: MyPubSub)
{:ok, #PID<0.105.0>}

keys: :duplicate just means that we allow multiple processes to be registered under a single key. This is exactly what we need for a PubSub, one key/topic for multiple processes.

Now, let’s create a bunch of subscribers and register them to our MyPubSub process registry. These processes would be registered under the key/topic, :event_topic. You can use any Elixir term as a topic. Depending on your use case, it’s usually an action that happened. For example, :"user.updated".

iex(2)> for _i <- 1..3 do
...(2)>     spawn(fn ->
...(2)>       Registry.register(MyPubSub, :event_topic, nil)
...(2)>
...(2)>       receive do
...(2)>         message ->
...(2)>           IO.puts("#{inspect(self())} received: #{inspect(message)}")
...(2)>       end
...(2)>     end)
...(2)>   end
[#PID<0.107.0>, #PID<0.108.0>, #PID<0.109.0>]

Now that we have subscribers, we can try and broadcast a message to them. The Registry module has a function called dispatch/4 which we could definitely use in this scenario. dispatch/4 will iterate over the entries (in our case, subscribers) and do what we asked it to do.

iex(3)> Registry.dispatch(MyPubSub, :event_topic, fn entries ->
...(3)>   for {pid, _} <- entries, do: send(pid, {:hello, "world"})
...(3)> end)
:ok
#PID<0.109.0> received: {:hello, "world"}
#PID<0.108.0> received: {:hello, "world"}
#PID<0.107.0> received: {:hello, "world"}

Voila! We were able to broadcast a message to all registered process under the key/topic. We just demoed PubSub in our iex shell. Now let’s make it pretty and put it in our app, Hermes.

We want the Registry to start as soon as we start our app. So let’s put it in our app’s supervisor.

# lib/hermes/application.ex
...
def start(_type, _args) do
  children = [
    {Registry, keys: :duplicate, name: Hermes.Registry}
  ]

  opts = [strategy: :one_for_one, name: Hermes.Supervisor]
  Supervisor.start_link(children, opts)
end
...

Our Hermes module should have two functions, subscribe/1 and publish/2. subscribe/1 will subscribe the process to the topic while publish/2 will broadcast a message to all subscribers of that topic.

# lib/hermes.ex
defmodule Hermes do
  @registry Hermes.Registry

  def subscribe(topic) do
    Registry.register(@registry, topic, nil)
  end

  def publish(topic, message) do
    Registry.dispatch(@registry, topic, fn entries ->
      for {pid, _} <- entries, do: send(pid, message)
    end)
  end
end

Let’s see if our local PubSub still works as expected.

iex -S mix

iex(1)> spawn(fn ->
...(1)>   Hermes.subscribe(:"user.updated")
...(1)>
...(1)>   receive do
...(1)>     message ->
...(1)>       IO.puts("#{inspect(self())} received: #{inspect(message)}")
...(1)>   end
...(1)> end)
#PID<0.167.0>
iex(2)> Hermes.publish(:"user.updated", {:user, %{id: 1, name: "John"}})
:ok
#PID<0.167.0> received: {:user, %{id: 1, name: "John"}}

Distributed PubSub

Our current version of Hermes is already good enough… but only for single-node apps. This won’t suffice for a clustered app.

To fact-check this, spin up two instances of the app. For easy identification, we’ll just name our nodes Alice and Bob.

# Node: Alice
iex --sname alice -S mix
iex(alice@Zeus)1>

# Node: Bob
➜ iex --sname bob -S mix
iex(bob@Zeus)1>

Connect the two nodes.

# Node: Alice
iex(alice@Zeus)1> Node.connect(:"bob@Zeus")
:ok

Verify that they are connected and then subscribe to the :"user.updated" topic.

# Node: Bob
iex(bob@Zeus)1> Node.list()
[:alice@Zeus]
iex(bob@Zeus)2> Hermes.subscribe(:"user.updated")
{:ok, #PID<0.136.0>}

From Alice’s shell, broadcast a PubSub message.

# Node: Alice
iex(alice@Zeus)2> Hermes.publish(:"user.updated", {:user, %{id: 1, name: "John"}})
:ok

Observe Bob’s shell. You’ll notice that nothing happened. Even though they are connected, they don’t share the registry. Yes, they are connected, but it just means they can send and receive messages from other nodes in the cluster.

Process Groups

This is where Erlang’s pg module would come in. In its simplest definition, it enables processes in the cluster to be a member of a named group, thus the name pg (process group).

Just like how we started using Registry, let’s play with pg on our iex shell. Spin up two nodes and connect them.

# Node: Alice
iex --sname alice
iex(alice@Zeus)1>

# Node: Bob
➜ iex --sname bob
iex(bob@Zeus)1> Node.connect :"alice@Zeus"
true
iex(bob@Zeus)2> Node.list()
[:alice@Zeus]

Just like the Registry, we need to to start a pg process on both nodes before we can group processes.

# Node: Alice
iex(alice@Zeus)1> :pg.start_link()
{:ok, #PID<0.114.0>}

# Node: Bob
iex(bob@Zeus)3> :pg.start_link()
{:ok, #PID<0.116.0>}

Then on Bob’s shell, join a process group called :my_group.

# Node: Bob
iex(bob@Zeus)4> :pg.join(:my_group, self())
:ok

Bob’s iex shell is now part of the :my_group process group. Go back to Alice’s shell and see all the members of the group.

# Node: Alice
iex(alice@Zeus)2> [pid] = :pg.get_members(:my_group)
[#PID<13033.110.0>]
iex(alice@Zeus)3> send(pid, {:hello, "world"})
{:hello, "world"}

# Node: Bob
iex(bob@Zeus)6> flush
{:hello, "world"}
:ok

You will see the group’s member process from the remote node. Using the send/1 function, you’ll be able to send a message to a process on a remote node.

We can use this for our distributed PubSub, right? Every subscriber process on each node can just join a pg group under the same key. Well, yes… but maybe not so efficient? If there are thousands of subscribers on a remote node, then we will be sending thousands of messages to a remote node. Phoenix.PubSub tackled this by having a designated local server process per node that is responsible for broadcasting to its local subscribers, kind of like a message coordinator. Only the local server processes are added in the pg group. So, in our case, if there are 3 nodes running, there will just be 3 members of the process group.

To ease up our life, let’s just use libcluster for clustering.

# mix.exs
...
{:libcluster, "~> 3.4"}
...

Since we are just running our nodes on localhost, we can use the LocalEpmd strategy.

# lib/hermes/application.ex
...
def start(_type, _args) do
  ...
  topologies = [
    hermes_local: [
      strategy: Cluster.Strategy.LocalEpmd
    ]
  ]

  children = [
    {Cluster.Supervisor, [topologies, [name: Hermes.ClusterSupervisor]]},
    {Registry, keys: :duplicate, name: Hermes.Registry}
  ]
  ...
end
...

Once you start the app for Alice and Bob, you should see that they are connected automatically.

# Node: Bob

➜ iex --sname bob -S mix

18:30:17.705 [info] [libcluster:hermes_local] connected to :alice@Zeus
Interactive Elixir (1.17.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(bob@Zeus)1> Node.list()
[:alice@Zeus]

Now that they are connected automatically, we can focus on building our distributed PubSub.

Let’s implement our local server process as a GenServer and add it to our app’s supervision tree.

# lib/hermes/pg_server.ex
defmodule Hermes.PGServer do
  use GenServer

  def start_link(_) do
    GenServer.start_link(Hermes.PGServer, [], name: Hermes.PGServer)
  end

  @impl GenServer
  def init(_args) do
    :pg.start_link()
    :pg.join({:hermes, Hermes.PubSub}, self())

    {:ok, []}
  end
end

# lib/hermes/application.ex
...
children = [
  {Cluster.Supervisor, [topologies, [name: Hermes.ClusterSupervisor]]},
  {Registry, keys: :duplicate, name: Hermes.Registry},
  Hermes.PGServer
]
...

Upon the GenServer’s initialization, it will also start the pg process on its default scope. Then, it will join the process group called {:hermes, Hermes.PubSub}. A process group’s name can be any Erlang term. Recompile both and verify that both Alice’s and Bob’s pg server are members of the {:hermes, Hermes.PubSub} process group.

# Node: Alice
iex(alice@Zeus)1> :pg.get_members({:hermes, Hermes.PubSub})
[#PID<22764.190.0>, #PID<0.208.0>]

Currently, Hermes.publish/2 broadcasts only to local subscribers. Instead of broadcasting to registry entries, we need it to send messages to all the other nodes’ Hermes.PGServer process, which are members of the {:hermes, Hermes.PubSub} process group. Here’s the updated Hermes.PGServer module:

# lib/hermes.ex
defmodule Hermes do
  @registry Hermes.Registry
  @pg_group {:hermes, Hermes.PubSub}

  def subscribe(topic) do
    Registry.register(@registry, topic, nil)
  end

  def publish(topic, message) do
    for pid <- :pg.get_members(@pg_group), node(pid) != node() do
      send(pid, {:broadcast_to_local, topic, message})
    end

    broadcast_local(topic, message)
  end

  def broadcast_local(topic, message) do
    Registry.dispatch(@registry, topic, fn entries ->
      for {pid, _} <- entries, do: send(pid, message)
    end)
  end
end

Hermes.publish/2 will now send a message to all the clustered nodes’ Hermes.PGServer. That module should be broadcasting it to its local subscribers.

# lib/hermes/pg_server.ex

...
@impl GenServer
def handle_info({:broadcast_to_local, topic, message}, state) do
  Hermes.broadcast_local(topic, message)
  {:noreply, state}
end
...

Once the node’s Hermes.PGServer process receives a message, it should broadcast it locally.

That should be it! You now have a PubSub that could broadcast to remote nodes… a distributed PubSub! 🎉

Before taking it out for a spin, let’s create a very basic client module so we can easily test that our PubSub works. This Client module just starts a GenServer process that subscribes to a specified topic. It will also just print out any messages it receive.

# lib/hermes/support/client.ex
defmodule Client do
  use GenServer

  def start_link(topic) do
    GenServer.start_link(__MODULE__, topic)
  end

  def init(topic) do
    Hermes.subscribe(topic)
    {:ok, topic}
  end

  def handle_info(msg, state) do
    IO.puts("Received: #{inspect(msg)}")
    {:noreply, state}
  end
end

Now that we have it in place, we can start testing it. I’m feeling a bit adventurous today so we’ll be connecting 3 nodes namely, Alice, Bob, and Carol.

# Node: Alice
iex --sname alice -S mix

# Node: Bob
iex --sname bob -S mix

# Node: Carol
iex --sname carol -S mix

Since we have configured libcluster, they should automatically discover and connect to each other.

For this test, we’ll have the following:

• Alice:

  • 1 client subscribed to :"user.created"

  • 1 client subscribed to :"user.updated"

• Bob:

  • 5 clients subscribed to :"user.created"

  • 0 clients subscribed to :"user.updated"

• Carol:

  • 0 clients subscribed to :"user.created"

  • 2 clients subscribed to :"user.updated"

# Node: Alice
iex(alice@Zeus)1> Client.start_link(:"user.created")
{:ok, #PID<0.214.0>}
iex(alice@Zeus)2> Client.start_link(:"user.updated")
{:ok, #PID<0.215.0>}

# Node: Bob
iex(bob@Zeus)1> for _i <- 1..5, do: Client.start_link(:"user.created")
[
  ok: #PID<0.201.0>,
  ok: #PID<0.202.0>,
  ok: #PID<0.203.0>,
  ok: #PID<0.204.0>,
  ok: #PID<0.205.0>
]

# Node: Carol
iex(carol@Zeus)1> for _i <- 1..2, do: Client.start_link(:"user.updated")
[ok: #PID<0.197.0>, ok: #PID<0.198.0>]

Let’s try publishing a :"user.created" message from Alice.

# Node: Alice
iex(alice@Zeus)3> Hermes.publish(:"user.created", %{name: "Jade"})
:ok
Received: %{name: "Jade"}

# Node: Bob
Received: %{name: "Jade"}
Received: %{name: "Jade"}
Received: %{name: "Jade"}
Received: %{name: "Jade"}
Received: %{name: "Jade"}

# Node: Carol
# ** nothing new **

Since no clients in Carol are interested when users are created, nothing was broadcasted to its local subscribers.

Cool. Now, let’s try publishing a :"user.updated" message from Bob.

# Node: Alice
Received: %{id: 1, name: "Jadyline"}

# Node: Bob
iex(bob@Zeus)2> Hermes.publish(:"user.updated", %{id: 1, name: "Jadyline"})
:ok

# Node: Carol
Received: %{id: 1, name: "Jadyline"}
Received: %{id: 1, name: "Jadyline"}

Awesome! Our distributed PubSub seems to be working fine!

I would like to reiterate that what we built here is a simple toy example solely for learning purposes and not meant to be used in production. If you need a distributed PubSub for production, just use the already battle-tested Phoenix.PubSub.

Link to full source code: https://github.com/vinceurag/hermes

See you on the next one!

To fully grasp this article, you must at least know the basics of Elixir/Erlang processes.

GenServer in Action

If you already know how to use GenServers, feel free to skip this section and jump straight to Anatomy of a GenServer.

GenServer is a behavior that facilitates client-server interaction. It abstracts away all the nitty-gritty details when dealing with interprocess communications.

In this example, we'll build an item accumulator. It would be an Elixir process that accepts an item and stores it in a list. This is how it would look like if we were to use the process primitive:

iex(1)> pid = spawn(fn ->
...(1)>     receive do
...(1)>       {sender, acc, item} -> send(sender, [item | acc])
...(1)>     end
...(1)>   end)
#PID<0.229.0>
iex(2)> send pid, {self, [], "apple"}
{#PID<0.169.0>, [], "apple"}
iex(3)> flush
["apple"]

As you may have noticed, we would need to pass a couple of things around. We would also need to make sure that it's a long-living process (we'll discuss more about this later). It's easy to get lost within this code. This is why GenServer behavior exists.

Let's rewrite this using GenServer. To start, we need to define a module that implements the GenServer behaviour.

defmodule ItemAccumulator do
  use GenServer
end

First, it should be able to set an initial state, in our case, an initial accumulator. To do this, we need to implement the init/1 callback.

@impl true
def init(acc) do
  {:ok, acc}
end

Our ItemAccumulator will have two functionalities. We need to have a function that would store the item and another function to return the items. When storing an item, we don't need a response; hence, we can use GenServer.cast/2. On our GenServer module, we then need to handle the cast request using the handle_cast/2 callback.

@impl true
def handle_cast({:add, item}, acc) do
  {:noreply, [item | acc]}
end

On the other hand, when we ask for the list of items in the accumulator, we need to wait for a response from the server. Because of this, we need to use GenServer.call/3. On our GenServer module, we would need to implement the handle_call/3 callback.

Lastly, let's also show the accumulated items before the process exits.

@impl true
def terminate(_reason, acc) do
  IO.puts "Last state: #{inspect(acc)}"
end

This is how our GenServer module should look like:

defmodule ItemAccumulator do
  use GenServer

  @impl true
  def init(acc) do
    {:ok, acc}
  end

  @impl true
  def handle_call(:list_items, from, acc) do
    {:reply, acc, acc}
  end

  @impl true
  def handle_cast({:add, item}, acc) do
    {:noreply, [item | acc]}
  end

  @impl true
  def terminate(_reason, acc) do
    IO.puts "Last state: #{inspect(acc)}"
  end
end

Let us test this out.

iex(1)> GenServer.start_link(ItemAccumulator, [], name: ItemAccumulator)
{:ok, #PID<0.165.0>}

The start_link/3 function starts the GenServer process. Giving it a name would register the process and would make it easier for us to locate and use it.

iex(2)> GenServer.cast(ItemAccumulator, {:add, "Apples"})
:ok
iex(3)> GenServer.cast(ItemAccumulator, {:add, "Oranges"})
:ok
iex(4)> GenServer.call(ItemAccumulator, :list_items)
["Oranges", "Apples"]
iex(5)> GenServer.stop(ItemAccumulator)
Last state: ["Oranges", "Apples"]
:ok

Since we have a working GenServer, we can now stop here or make the interface more developer-friendly. We can do that by abstracting the implementation details away from the client. They don't always need to know the implementation details, in our case, that we're using GenServers.

defmodule ItemAccumulator do
  use GenServer

  def start_link(state) do
    GenServer.start_link(__MODULE__, state, name: __MODULE__)
  end

  def add(item) do
    GenServer.cast(__MODULE__, {:add, item})
  end

  def list_items(), do: GenServer.call(__MODULE__, :list_items)

  ...
  # callbacks
end

Let's try it out in the shell.

iex(1)> ItemAccumulator.start_link([])
{:ok, #PID<0.155.0>}
iex(2)> ItemAccumulator.add("Apples")
:ok
iex(3)> ItemAccumulator.add("Oranges")
:ok
iex(4)> ItemAccumulator.list()
["Oranges", "Apples"]

The interface now looks cleaner and just focuses on the functionalities instead of implementation details.

Anatomy of a GenServer

A GenServer is just a regular Elixir/Erlang process that is stuck in a loop. Don't believe me? Here's what the observer shows when you're running a GenServer process.

It's an Erlang process that waits for a message, acts on the message, then loops and waits for a message again.

Take a look at the snippet below.

defmodule HealthCheck do
  def loop() do
    receive do
      {pid, :ping} ->
        send(pid,:pong)
        loop()
      :exit -> 
        Process.exit(self(), :normal)
    end
  end
end

We define a module called HealthCheck that has a function that just responds to a ping message and then loops. Let's use this module to build our first dumb version of our generic server.

iex(1)> pid = spawn(&HealthCheck.loop/0)
#PID<0.155.0>

We now have a running generic server. To confirm that it's indeed alive, verify it using Process.alive?/1.

iex(2)> Process.alive?(pid)
true

Let's do a ping check and see if our server responds.

iex(3)> send(pid, {self(), :ping})
{#PID<0.151.0>, :ping}
iex(4)> Process.info(self(), :messages)
{:messages, [:pong]}

Now confirm if our generic server is still alive.

iex(5)> Process.alive?(pid)
true

We just built a long-running process. This is GenServer under the hood, with some quirks sprinkled on top of it.

Now let's build our own version of a GenServer!

BasicServer

Requirements

In this mini-project, we aim to replicate the following functionalities of a GenServer:

• init - We should be able to set the initial state.

• call - We should be able to make synchronous calls to the server and wait for a reply.

• cast - We should be able to send fire-and-forget calls to the server.

• stop - We should be able to stop the server.

GenServers have other functionalities like multicast, multicall, etc but we'll only focus on the functionalities enumerated above.

__using__ Macro

The __using__ macro allows us to inject code into another module. We need this so that we can set the behavior for the server module and implement default callbacks.

defmodule BasicServer do
  @callback init(state :: term()) :: {:ok, term()} | {:stop, term()}

  defmacro __using__(_opts) do
    quote do
      @behaviour BasicServer
    end
  end
end

Setting Initial State

Setting the initial state of a GenServer is done via the init/1 callback. A module that is using our BasicServer must have an init/1 callback defined; otherwise, we raise an exception.

...
defmacro __using__(_opts) do
  quote do
    @behaviour BasicServer

    def init(_state) do
      raise "you forgot to implement the init callback!"
    end

    defoverridable init: 1
  end
end
...

Starting the Server

Now that we're enforcing the presence of an init/1 callback, we can proceed with building our first function which is starting the BasicServer.

defmodule BasicServer do
  @callback init(state :: term()) :: {:ok, term()} | {:stop, term()}

  ...

  def start_link(callback_mod, init_state, opts \\ []) do
    pid =
      # links to the parent process
      spawn_link(fn ->
        {:ok, state} = apply(callback_mod, :init, [init_state])

        loop(callback_mod, state)
      end)

    # registers the process
    if opts[:name], do: Process.register(pid, opts[:name])

    {:ok, pid}
  end

  defp loop(callback, state) do
    # this receive-block will do the heavy-lifting later
    receive do
      _ -> loop(callback, state)
    end
  end

  ...
end

Synchronous Calls

In this section, we will be building the call part of GenServers. call is an asynchronous operation. It will not proceed with other actions until it receives a reply or it times out. How are we going to build this? call will send a message to our server process which is in a suspended state and just waiting for a message to arrive. Once our server process receives the message, it will call the callback module's handle_call/3 function, passing in the message and current state. The calling process would then enter a suspended state waiting for a reply.

Let's create the call/2 function.

...
def call(pid, message) do
  # we need to mark the message as a `:call` to pattern-match later
  # in the receive-loop
  send(pid, {:call, self(), message})

  # since `call` is a synchronous operation, we need to wait for
  # a response from the server
  receive do
    {:reply, reply} ->
      reply
  end
end
...

Now, we modify the receive-block inside the looping function to handle the call message and call the appropriate callback function.

...
defp loop(callback, state) do
  receive do
    {:call, caller_pid, message} ->
      # for this example, we'll only handle the
      # `{:reply, reply, state}` response.
      {:reply, reply, new_state} = apply(callback, :handle_call, [message, caller_pid, state])
      send(caller_pid, {:reply, reply})
      loop(callback, new_state)
  end
end
...

Asynchronous Casts

Let's deal with the fire-and-forget requests. This is very similar to the call function, except that we don't need to send a reply back to the calling process.

...
def cast(pid, message) do
  # we don't need the caller_pid anymore
  # since we don't need to send a message back
  send(pid, {:cast, message})
end
...

Handling the cast message is also simpler. We just need to call the callback function.

...
defp loop(callback, state) do
  receive do
    {:call, caller_pid, message} ->
      {:reply, reply, new_state} = apply(callback, :handle_call, [message, caller_pid, state])
      send(caller_pid, {:reply, reply})
      loop(callback, new_state)

    {:cast, message} ->
      {:noreply, new_state} = apply(callback, :handle_cast, [message, state])
      loop(callback, new_state)
  end
end
...

Hey Server, stop!

Finally, the last remaining function, stop. This function would just tell our server to exit. Well, it needs to do some cleanup first before exiting. Let's start by implementing the stop function.

...
def stop(server, reason) do
  send(server, {:stop, reason})

  :ok
end
...

Hmm... why do we need to send a message to the server? Remember, the server is in a suspended state waiting for a message. Sure, we can just kill the server process, but that's not a graceful exit. Once the server receives the exit message, it executes the terminate/2 callback and then escapes the loop; thus, terminating the server process.

...
defp loop(callback, state) do
  receive do
    {:call, caller_pid, message} ->
      {:reply, reply, new_state} = apply(callback, :handle_call, [message, caller_pid, state])
      send(caller_pid, {:reply, reply})
      loop(callback, new_state)

    {:cast, message} ->
      {:noreply, new_state} = apply(callback, :handle_cast, [message, state])
      loop(callback, new_state)

    {:stop, reason} ->
      apply(callback, :terminate, [reason, state])
  end
end
...

Testing Our Basic Server

Here's the full code of our GenServer clone, BasicServer.

defmodule BasicServer do
  @callback init(state :: term()) :: {:ok, term()} | {:stop, term()}
  @callback handle_call(msg :: term(), from :: pid(), state :: term()) ::
              {:reply, reply :: term(), state :: term()}
  @callback handle_cast(msg :: term(), state :: term()) :: {:noreply, state :: term()}
  @callback terminate(reason :: atom(), state :: term()) :: :ok

  defmacro __using__(_opts) do
    quote do
      @behaviour BasicServer

      def init(_state) do
        raise "you forgot to implement the init callback!"
      end

      defoverridable init: 1
    end
  end

  def start_link(callback_mod, init_state, opts \\ []) do
    pid =
      spawn_link(fn ->
        {:ok, state} = apply(callback_mod, :init, [init_state])

        loop(callback_mod, state)
      end)

    if opts[:name], do: Process.register(pid, opts[:name])

    {:ok, pid}
  end

  def call(pid, message) do
    send(pid, {:call, self(), message})

    receive do
      {:reply, reply} ->
        reply
    end
  end

  def cast(server, message) do
    send(server, {:cast, message})

    :ok
  end

  def stop(server, reason) do
    send(server, {:stop, reason})

    :ok
  end

  defp loop(callback, state) do
    receive do
      {:call, caller_pid, message} ->
        {:reply, reply, new_state} = apply(callback, :handle_call, [message, caller_pid, state])
        send(caller_pid, {:reply, reply})
        loop(callback, new_state)

      {:cast, message} ->
        {:noreply, new_state} = apply(callback, :handle_cast, [message, state])
        loop(callback, new_state)

      {:stop, reason} ->
        apply(callback, :terminate, [reason, state])
    end
  end
end

We'll reuse the ItemAcc we wrote before but instead of using GenServer, we use BasicServer.

defmodule ItemAcc do
  use BasicServer

  def start_link(state) do
    BasicServer.start_link(__MODULE__, state, name: __MODULE__)
  end

  def add(item) do
    BasicServer.cast(__MODULE__, {:add, item})
  end

  def list_items(), do: BasicServer.call(__MODULE__, :list_items)

  def stop(), do: BasicServer.stop(__MODULE__, :normal)

  @impl true
  def init(state), do: {:ok, state}

  @impl true
  def handle_call(:list_items, _from, acc) do
    {:reply, acc, acc}
  end

  @impl true
  def handle_cast({:add, item}, acc) do
    {:noreply, [item | acc]}
  end

  @impl true
  def terminate(_reason, state) do
    IO.puts "Terminating... Last state: #{inspect(state)}"
  end
end

iex(1)> ItemAcc.start_link([])
{:ok, #PID<0.155.0>}
iex(2)> ItemAcc.add("Apples")
:ok
iex(3)> ItemAcc.add("Oranges")
:ok
iex(4)> ItemAcc.list_items()
["Oranges", "Apples"]
iex(5)> ItemAcc.stop()
Terminating... Last state: ["Oranges", "Apples"]
:ok

We are now done implementing the core functionalities of a GenServer. I think it's important to highlight that you should not build your own GenServer. Elixir's GenServers are battle-tested already. You most likely don't need to reinvent the wheel. We only did it for educational purposes. If you need client-server communication, then just use the already-built GenServer.

If you enjoyed this insightful post and would like to receive more content like this, I warmly invite you to subscribe to my newsletter. Your support is truly appreciated! See you on the next one!

In this post, we'll talk about tracing a function to know its input arguments and return value. For demonstrations of the principles, we'll be using Erlang's dbg module (but of course, using Elixir syntax) then later on we will use a library called SmartTracer.

There are a lot of opinions on whether you should or should not trace on a live production node. This article is not about it. I think it's fine as long as you know what you're doing and you added ample safeguards to your traces.

Anyway, let's jump in.

Using Erlang's dbg

The dbg module is included in the runtime_tools app which is part of OTP. It contains tools that could help you in debugging/tracing and is suitable to be included in production systems.

For demonstration purposes, we'll be using the Foo module defined below as an example.

defmodule Foo do
  @moduledoc false

  @doc false
  def bar(name) do
    "Hello, my name is #{get_name(name)}"
  end

  defp get_name(name) do
    "Fizz" <> name  
  end
end

Fire up your IEx and load the Foo module.

To start a tracer process we just need to invoke :dbg.tracer/0. This process will be the recipient of all trace messages.

iex(2)> :dbg.tracer()
{:ok, #PID<0.121.0>}

Now we need to configure which processes or ports to trace and what. For this example, let's trace all the function calls in existing and future processes.

iex(3)> :dbg.p(:all, :c)
{:ok, [{:matched, :nonode@nohost, 56}]}

:all means we want to trace existing processes before this was set and all processes that would be created. The second argument specifies what we want to trace, in this case :c, which means, function calls.

Erlang documentation has a long list of flags to configure your tracer. You can take a look at those here.

You might be wondering about the somehow cryptic return value of :dbg.p/2. It's just saying that it's matching 56 processes based on your configuration. To verify this, You can run this:

iex(4)> length(Process.list())
56

Now we need to set up a trace pattern. We'll tell the tracer which function to trace.

iex(5)> :dbg.tp(Foo, :bar, 1, [{:_, [], [{:return_trace}]}])
{:ok, [{:matched, :nonode@nohost, 1}, {:saved, 1}]}

The first to the third argument is the MFA. The last argument is called MatchSpec. We won't dive into it but Erlang has a comprehensive document that you can read here. Basically, we told the tracer that we want to trace calls to Foo.bar/1 and we also want the return values.

We're quite done with setting up the tracer. Let's try it out.

iex(6)> Foo.bar("Buzz")
"Hello, my name is FizzBuzz"

iex(7)> (<0.104.0>) call 'Elixir.Foo':bar(<<"Buzz">>)
(<0.104.0>) returned from 'Elixir.Foo':bar/1 -> <<"Hello, my name is FizzBuzz">>

Woohoo! 🎉

Remember to stop the tracer since it also consumes memory.

iex(8)> :dbg.stop_clear()
:ok

Let's try tracing a local function call. In this case, a call to get_name/1 which is a private function.

iex(9)> :dbg.tracer()
{:ok, #PID<0.130.0>}

iex(10)> :dbg.p(:all, :c)
{:ok, [{:matched, :nonode@nohost, 56}]}

iex(11)> :dbg.tp(Foo, :get_name, 1, [{:_,  [], [{:return_trace}]}])
{:ok, [{:matched, :nonode@nohost, 0}, {:saved, 1}]}

iex(12)> Foo.bar("Buzz")
"Hello, my name is FizzBuzz"

Nothing happened. It's because :dbg.tp/4 does not trace local function calls. To trace local function calls, we need to use :dbg.tpl/4. I assume the l there stands for local. 🤷‍♂️

iex(13)> :dbg.tpl(Foo, :get_name, 1, [{:_,  [], [{:return_trace}]}])
{:ok, [{:matched, :nonode@nohost, 1}, {:saved, 1}]}

iex(14)> Foo.bar("Buzz")
(<0.104.0>) call 'Elixir.Foo':get_name(<<"Buzz">>)
(<0.104.0>) returned from 'Elixir.Foo':get_name/1 -> <<"FizzBuzz">>
"Hello, my name is FizzBuzz"

That's about everything you need to know to start tracing functions in Elixir. In the following section we'll talk about using the library called SmartTracer by yours truly.

Using SmartTracer

SmartTracer is an Elixir library I built to aid developers with tracing function calls. It has some added safeguards and rich features like trace recording and defining your own custom trace handlers.

Design

As show in the diagram above, every process spawned by the SmartTracer has the IEx shell as its parent process. We don't want to have dangling/rogue processes in our system. This way, if the shell exits, all SmartTracer-related processes will be brought down as well. The shell also trap exits so it won't die when the tracer and trace handler processes exit.

The recorder process is not in the tracer process chain since we want it to be independent from the tracer processes. If ever the tracer process terminates, we'd still be able to playback the recorded function calls.

Safeguards

Rate Limiting

As I have mentioned before, tracing function calls consumes resources so it's not ideal to trace unlimited function calls as it may degrade your app's performance. Most of the time, you just need a small sample data. When using the SmartTracer library, a limit is necessary. When the rate limit is tripped, the tracer processes are killed. You would still be able to playback the recorded calls.

Shell as the Parent Process

Having the IEx shell as the parent process of all SmartTracer-related processes, we could be sure that once the IEx shell exits, there will be no rogue processes left behind.

Features

Easy-to-use interface

Instead of manually starting the tracers and setting the pattern, starting a trace can be done via a one-liner:

iex(2)> SmartTracer.trace(&Foo.bar/1, 5)
:ok

iex(3)> Foo.bar("Buzz")
"Hello, my name is FizzBuzz" # <- return value from shell, ignore

03:31:34.429 [info] Elixir.Foo.bar/1 is being called with: ["Buzz"]

You just need to provide the function reference and the number of calls you want to trace.

If you need the return value, it's just an option away.

iex(7)> SmartTracer.trace(&Foo.bar/1, 5, return: true)
:ok

iex(8)> Foo.bar("Buzz")
03:33:00.636 [info] Elixir.Foo.bar/1 is being called with: ["Buzz"]
"Hello, my name is FizzBuzz" # <- return value from shell, ignore
03:33:00.636 [info] Elixir.Foo.bar/1 returns: "Hello, my name is FizzBuzz"

To trace local functions, you just need to set the scope to :local.

iex(13)> SmartTracer.trace(&Foo.get_name/1, 5, scope: :local)
:ok

iex(14)> Foo.bar("Buzz")
"Hello, my name is FizzBuzz" # <- return value from shell, ignore

03:39:42.136 [info] Elixir.Foo.get_name/1 is being called with: ["Buzz"]

Call Recording

Let's say you would be tracing a function that is being called very frequently, there's a chance that the tracer's output will be buried deep in the shell. To avoid this, you can record the calls and play them back when you need to (as long as the IEx and recorder agent are alive).

iex(15)> SmartTracer.trace(&Foo.bar/1, 5, record: true)
:ok

iex(16)> Foo.bar("Buzz")                                     
03:47:35.238 [info]  Elixir.Foo.bar/1 is being called with: ["Buzz"]
"Hello, my name is FizzBuzz" # <- return value from shell, ignore

iex(17)> SmartTracer.playback()
[
  %SmartTracer.Utils.Recorder.Call{
    args: ["Buzz"],
    arity: 1,
    datetime: ~U[2021-02-05 19:47:35Z],
    function: :bar,
    module: Foo,
    type: :call
  }
]

Custom Tracer

Another big feature of SmartTracer is the ability to define your own tracer. By default, SmartTracer would just use Logger to output the function calls. When you define your own tracer, you would have full control on what to do with the traces. You can write it to a file, send it to an API, etc. Just be mindful of any performance impact.

Here's a very simple definition of a CustomTracer.

defmodule MyApp.CustomTracer do
  @moduledoc false

  use SmartTracer.Custom

  require Logger

  def handle(:call, {_module, _fun, args}) do
    # maybe write to a file?
    Logger.info("The function was called with #{inspect(args)}")
  end

  def handle(:return, {_module, _fun, _arity, return_value}) do
    Logger.info("The function returned: #{inspect(return_value)}")
  end
end

Now, instead of calling SmartTracer directly, you would now invoke the trace/3 function from your CustomTracer.

iex(4)> MyApp.CustomTracer.trace(&Foo.bar/1, 5)
:ok

iex(5)> Foo.bar("Buzz")
"Hello, my name is FizzBuzz" # <- return value from shell, ignore
03:59:26.435 [info]  The function was called with ["Buzz"]

That's all! To learn more, refer to SmartTracer's documentation. Also, feel free to contribute to this project. [Github]

Docker is a containerization platform that has been around since 2013. It tries to eliminate the it-works-on-my-machine problem. In this mini-article, we'll discuss what are containers and images in a very simple way. We'll also try to run a PHP file via Docker. Let's get started!

Basic Terminologies

Images

A Docker image is basically a file that's a snapshot of a container. Images are used to run containers. Think of it as a blueprint and the house is the container.

Containers

Docker Container is an instance of a Docker image. Think of it as a 'running image'.

Demo

In this demo, we'll be running a simple PHP script that prints a text on the browser without installing any other shits (👀Apache).

Create a simple PHP script

Create a new directory called scripts, then inside it, create a new file luffy.php.

<?php
  // scripts/luffy.php

  echo "Gomu Gomu no bazooooka!";

Dockerfile

So we've already talked about images and containers but we haven't actually tried making one. A Dockerfile is a text file that holds commands and keywords that Docker uses to build the image.

Create a new file directly above the scripts directory and name it Dockerfile. Copy and paste the code below.

FROM php:7.2-apache

COPY scripts/ /var/www/html

EXPOSE 80

Dockerfile Dissection

• FROM - This instruction pulls an image from a public Docker repository like Docker Hub. In our case, we need to have php and apache installed. With luck on our side, there's already an image that has PHP and Apache. The format for setting the base image is FROM image:tag.

• COPY - This instruction copies the files in <source> to <destination>.

• EXPOSE - This instructions tells Docker which ports the container is listening during runtime.

Building the Docker Image

Since we have already created the Dockerfile, it's time for Docker to build the image. While in the directory where Dockerfile is located, issue this command to build the image:

docker build -t bazooka .

During the first time, it would download all the layers needed. So you will see something like this:

Sending build context to Docker daemon  3.584kB
Step 1/3 : FROM php:7.2-apache
7.2-apache: Pulling from library/php
be8881be8156: Pull complete
69a25f7e4930: Pull complete
65632e89c5f4: Pull complete
cd75fa32da8f: Pull complete
15bc7736db11: Pull complete
b2c40cef4807: Pull complete
f3507e55e5eb: Pull complete
e6006cdfa16b: Pull complete
a3ed406e3c88: Pull complete
ae70e16ef035: Pull complete
b9501b07bdf1: Pull complete
f9ecb48a3c12: Pull complete
23d9db872f7e: Pull complete
c41139f0d4f5: Pull complete
f180ba12d718: Pull complete
Digest: sha256:34d20a90946b2a2e05ca6d2fe71f1394168af6dc1d18ab8211cc43f1679410cb
Status: Downloaded newer image for php:7.2-apache
 ---> 6a99292078a8
Step 2/3 : COPY scripts/ /var/www/html
 ---> c00fd3a92d60
Step 3/3 : EXPOSE 80
 ---> Running in bf9e253efab6
Removing intermediate container bf9e253efab6
 ---> 81b417c196b3
Successfully built 81b417c196b3
Successfully tagged bazooka:latest

Running the image

To run the created image:

docker run -p 80:80 bazooka

Formula: docker run <flags|options> <image-name>

• -p - This flag is used to forward a port. <host|your-machine>:<container>

Appreciate the magic!

Open your browser then access localhost/luffy.php. You should now see the text "Gomu Gomu no bazooooka!" printed.