Debug Phoenix on Gigalixir with Remote Console and Observer

Once you have your Phoenix app deployed to Gigalixir using Elixir releases, we’re going to learn how to use a remote console and remote observer to debug your app in production. We can use these debugging tools to run code in production and monitor the Erlang Virtual Machine, processes, and applications remotely.

Overview

Here are the topics we’ll cover while deploying the project.

  1. Remote Console with IEx
  2. Remote Observer

Prerequisites

To follow this guide, make sure you have at least the following software installed on your machine. In the brackets are the versions with which this guide was written.

  • Elixir (1.12.3-otp-24)
  • Erlang (OTP 24.1)
  • Phoenix (1.5.13)
  • Node.js with npm (15.14.0)

Make sure to install at least Elixir 1.9 since we are using releases. We recommend using asdf to manage Erlang and Elixir versions.

The guide assumes you have deployed a Phoenix app to Gigalixir using our Gigalixir deployment guide. If you run into problems, the finished project is available on GitHub as StakNine HelloGigalixir.

Remote Console with IEx

Elixir provides an interactive shell called IEx which stand for Interactive Elixir. You can use IEx for debugging and accessing data in your application. Gigalixir allows you to use a remote console to access IEx on your production app.

If you haven’t already, install the gigalixir CLI and sign up for an Gigalixir account. We are using macOS. If your are using another OS, check out the install options for Windows and Linux.

brew tap gigalixir/brew && brew install gigalixir

Next you need to copy your SSH pubkey to the production server. Gigalixir does not restrict SSH access. If you don’t have an SSH key, you can follow GitHub’s guide: Generating a new SSH key and adding it to the ssh-agent.

You may need to update the filename depending on how you set up you SSH keys. If you need to view or delete SSH keys, check out the Gigalixir docs for managing SSH keys.

There are lot of useful gigalixir commands available in the gigalixir CLI. Run the following from the command line to copy your SSH pubkey to the production machine.

gigalixir account:ssh_keys:add "$(cat ~/.ssh/id_ed25519.pub)"

It could take up to a minute for the SSH keys to update in your containers. Then run the following command to enter a console.

gigalixir ps:remote_console

You can use the console to execute code in your production system. For example, you can run a query for the user we created previously to test the app.

Erlang/OTP 24 [erts-12.1] [source] [64-bit] [smp:8:1] [ds:8:1:10] [async-threads:1] [jit]
Interactive Elixir (1.12.3) - press Ctrl+C to exit (type h() ENTER for help)

iex(app-name@ip)1> HelloGigalixir.Repo.all(HelloGigalixir.Accounts.User)
[
  %HelloGigalixir.Accounts.User{
    __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
    id: 1,
    inserted_at: ~N[2021-09-23 14:49:48],
    name: "User",
    updated_at: ~N[2021-09-23 14:49:48],
    username: "user"
  }
]

Remote Observer on Gigalixir

Sometimes debugging by jumping into the code with IEx is not enough. Erlang provides observer to understand the whole virtual machine, processes, applications, as well as set up tracing mechanisms.

Since we deployed our Phoenix project with Elixir releases, Gigalixir enables us to connect a observer.

Because Observer runs on your local machine and connects to a production node by joining the production cluster, you first have to have clustering set up. We don’t need multiple nodes, but we need to follow the instructions below for Clustering Nodes.

Clustering will also allow you to distribute workload evenly and set up distributed Phoenix channels. The cool thing with Gigalixir is once you have successfully clustered your nodes, then distributed Phoenix channels just work out of the box.

Gigalixir uses libcluster to manage node clustering. For more information, see libcluster’s documentation.

To install libcluster, add this to the deps list in mix.exs

{:libcluster, "~> 3.3"}

Next, we need to edit you application.ex file. Add the following to the existing start function.

# lib/hello_gigalixir/application.ex
def start(_type, _args) do
  topologies = Application.get_env(:libcluster, :topologies) || []

  children = [
    {Cluster.Supervisor, [topologies, [name: HelloGigalixir.ClusterSupervisor]]},
    ... # other children
  ]
  ...
end

Then add libcluster config to your config/prod.exs file

...
config :libcluster,
  topologies: [
    k8s_example: [
      strategy: Cluster.Strategy.Kubernetes,
      config: [
        kubernetes_selector: System.get_env("LIBCLUSTER_KUBERNETES_SELECTOR"),
        kubernetes_node_basename: System.get_env("LIBCLUSTER_KUBERNETES_NODE_BASENAME")]]]
...

Gigalixir handles permissions so that you have access to Kubernetes endpoints and we automatically set your node name and erlang cookie so that your nodes can reach each other.

Gigalixir also automatically set the environment variables LIBCLUSTER_KUBERNETES_SELECTOR, LIBCLUSTER_KUBERNETES_NODE_BASENAME, APP_NAME, and MY_POD_IP for you. There is no need to set up additional Gigalixir config.

Your local machine also needs to have lsof installed. Our mac already had it installed.

Now you’re ready to commit and push your changes.

```bash
git add .
git commit -m "add clustering for remote observer"
git push gigalixir main

Once your code is deployed to production and your app restarts, you’re ready to open observer.

gigalixir ps:observer

Follow the instructions. It will prompt you for your local sudo password so it can modify iptables rules. It will open another Graphical User Interface that provides many panes to fully understand and navigate the runtime and your project.

Remote Observer - System tab
System Tab
Remote Observer - Load Charts
Load Charts Tab
Remote Observer - Applications
Applications Tab

Conclusion

Congratulations! You’ve used the remote console for IEx and observer to debug your Phoenix application in production on Gigalixir. Next, check out our article covering the best error monitoring services for Elixir, so you can effectively manage errors in your system using these debugging tools.

Want To Know How To Deploy Phoenix Apps Using A single Command?

This brand-new FREE training reveals the most powerful new way to reduce your deployment time and skyrocket your productivity… and truly see your programming career explode off the charts!