Building and deploying a Phoenix application using Nix flakes
building-and-deploying-a-phoenix-application-using-nix-flakesPreface
A few weeks ago, I released a new version of Charsh to the world. Still very much a work in progress, but a major upgrade from what was previously available. Aside from now using WebGL (and ThreeJS) to display character sprites, the biggest overhaul was making the switch from an ExpressJS back-end to an Elixir Phoenix one.
After all the MVP features I wanted were built, I needed to do one last thing:
Figure out my CI/CD pipeline.
When I started working with Phoenix a couple of years ago, I could not figure out how to get a release out. I don't remember what the issue was, I just know that I tried but nothing came out of it. So in my wisdom, to get CI for Rriver (the API powering these posts), I opted to just git clone my code onto my server with gitlab runners and simply run the mix phx.server command in a docker container.
And it worked, it certainly did. But I wasn't happy with it knowing that releases were a thing and that I wanted people to be able to run their own Charsh instances.
Fast forward to June and I am finally ready to launch that app. Except I am pretty new to flakes and I don't understand much more than copy-paste basics. So today, I'd like you to introduce to how I now build and deploy my projects, to my server or to Fly.io.
Acknowledgments
As I said, I am very much a beginner when it comes to flakes. I was brought up to speed reading the fantastic fasterthanli.me article 1. Then delved into Elixir specific logic written up in these two blog posts 2
The flake
This is not a tutorial on installing Nix or Nixos so I won't go into too much details there. Just make sure you have flakes enabled, as they are experimental.
There are two things we want to achieve with this flake:
A dev environment that will never fail us no matter the machine
A build script that will produce a release, a docker image and that can be executed everywhere
Before we start, let's create a folder we will work in: mkdir hello & cd hello
Dev environment
My machine is equipped with
nix-direnv[^3]: a library that will automatically launch the nix develop shell and pass the it environment variables. If you're following along with nix installed, you will need to enter the dev shell withnix develop.
Let's assemble our dev environment. This is the simplest form of flake but can save you many headaches by fetching the exact version of your dev dependencies on all your machines.
Create the hello/flake.nix and copy the following content:
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable"; # Here pick the nix channel we want our packages to be grabbe from.
};
outputs = { self, nixpkgs }:
let
# We are going to re-use this so it's a good idea to store it in a variable. Make sure you use your own system's architecture.
system = "x86_64-linux";
# And here we grab the packages for our target system.
pkgs = nixpkgs.legacyPackages.${system};
name = "hello";
version = "0.1.0";
in {
# here we finally define our dev shell, with all the packages we need. And that's it!
devShells.${system}.default = pkgs.mkShell {
packages = with pkgs; [ elixir_1_17 mix2nix esbuild dive] # We are getting elixir as well as packages we will need for our build step
++
pkgs.lib.optionals pkgs.stdenv.isLinux (with pkgs; [
inotify-tools ## We also need additional, OS specific (Linux in my case), packages to make sure everything runs smoothly.
libnotify
]);
};
};
}Seems easy enough and yet the first time I ever gave this a shot, I kept on bumping into errors and could not get my flake working.
Let's try it out! Open a terminal in this same folder and launch a nix dev shell with nix develop. After a brief loading period, you should be loaded into a new shell, the dev shell.
Setting up the project
You're going to need a Phoenix project to build! Luckily we have a shell that comes pre-packed with all the tools we need. Let's spin up the default Phoenix project in our folder. First, let's install Phoenix itself: mix archive.install hex phx_new
Now enter the mix phx.new . --app hello --no-ecto --no-tailwind command, to generate a project name hello in the current folder, without any database and tailwind (as it deserves its own post), and follow the instructions that come up on your screen.
We've seen this in my previous blog post but Phoenix has a fantastic guide and documentation[^4].
Once the dependencies are done installing, let's make sure that our app actually does something. Run your Phoenix server with mix phx.server. You should get the usual Phoenix console.
Hurrah! You now have a development environment that lets you get to work in a single command! Provided you have nix installed.
Let's quickly finish the setup to avoid having to come back to this. Our Phoenix server will want a SECRET_KEY_BASE. Let's generate one with mix phx.gen.secret and paste it in a .env file under the SECRET_KEY_BASE key.
Building a release
Back to our hello/flake.nix file!
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
};
outputs = { self, nixpkgs }:
let
system = "x86_64-linux";
pkgs = nixpkgs.legacyPackages.${system};
# Adding our beam packages here
beamPackages = pkgs.beam.packagesWith pkgs.beam.interpreters.erlang;
name = "hello";
version = "0.1.0";
# This is the first important block
# The nix beam package mixRelease will handle creating the release for us.
release = beamPackages.mixRelease {
inherit version;
# Use mix2nix's output file to specify our mix dependencies in a file nix can make use of.
# You won't have this deps.nix file for now. We will get to it in the next step.
mixNixDeps = with pkgs; import ./deps.nix {
inherit lib beamPackages;
};
pname = name;
src = ./.;
removeCookie = false;
# Interesting part too: nix can't make use of external libraries that are not part of its packages. Your build would fail when processing tailwind or esbuild.
# For this reason, we symlink our nix package executables where the build expecs them to be.
postBuild = ''
ln -s ${pkgs.esbuild}/bin/esbuild _build/esbuild-linux-x64
mix do deps.loadpaths --no-deps-check, assets.deploy
'';
## This line below is how you prevent your docker image from being >1.5GB##
# This took me too long to find but I am giving it to you for free.
stripDebug = true;
};
in {
devShells.${system}.default = pkgs.mkShell {
packages = with pkgs; [ elixir_1_17 mix2nix esbuild dive]
++
pkgs.lib.optionals pkgs.stdenv.isLinux (with pkgs; [
inotify-tools
libnotify
]);
};
# And finally our second block. Here we tell our flake that our output for the `nix build` command will be the release.
packages.${system} = {
packages = with pkgs; [ elixir_1_17 erlang_otp_27];
inherit release;
default = release;
};
};
}Good! We have all the moving pieces. But we still need to generate some files so that we are able to do a proper release. If you tried to get your build going with nix build, your build will fail. You're missing deps.nix and a release scripts.
Release scripts
To make your life easier, when creating a release, Phoenix will generate release scripts. These scripts will help when it comes to starting up the server and migrating our database.
To generate these scripts, use the mix phx.gen.release. This will create the overlay file rel/overlays/bin/server.
If we had instanciated the project with a database, this command would also have created an overlay for migrations.
deps.nix
Next is time to take care of our deps.nix file. We will generate it using the mix2nix package we added to our nix dev shell. As the repository[^5] states, mix2nix is "a command line utility to create a set of nix package definitions based on the contents of a mix.lock file". Using their example given in the readme will serve our purpose just fine. In your nix develop shell, type:
mix2nix > deps.nix
This will use your mix.lock file to generate a nix file containing each Elixir package as well as their version and lets us easily use them in our flake.nix without referencing them all manually.
Building a release.
We're finally ready to test our release! Try it with mix build.
There won't be a big "Congratulations, you've done it!" message, but you should see a result directory appear at the root of the project. Remember our overlays from 20 lines ago? This is where we use them to start out server! Unless you've modified paths, you should able able to start your server with results/bin/server.
Has it spun up? Fantastic! This is already a big quality of life improvement, in my opinion. But let's go further. I either host my apps on my own VPS as docker containers or on Fly.io, a hosting service which also functions on docker images.
Creating a docker image
If you were wondering what the heck the rest of our packages were, they come useful here when debugging for example, oh I don't know, why your docker image is 1.5GB?
For now, I'll give you the version that works (at least at the time of writing this article). We'll break things later and inspect them with dive.
Here's our updated flake.nix:
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
};
outputs = { self, nixpkgs }:
let
system = "x86_64-linux";
pkgs = nixpkgs.legacyPackages.${system};
beamPackages = pkgs.beam.packagesWith pkgs.beam.interpreters.erlang;
name = "hello";
version = "0.1.0";
release = beamPackages.mixRelease {
inherit version;
mixNixDeps = with pkgs; import ./deps.nix {
inherit lib beamPackages;
};
pname = name;
src = ./.;
removeCookie = false;
postBuild = ''
ln -s ${pkgs.esbuild}/bin/esbuild _build/esbuild-linux-x64
mix do deps.loadpaths --no-deps-check, assets.deploy
'';
stripDebug = true;
};
# Defining a constant for a package to use later
weasyprint = pkgs.python312Packages.weasyprint;
# This entrypointSh defines a script that will be run once our docker image is run.
# Here we will simply start the server, but you could execute migrations too if you had a database.
entrypointSh = pkgs.writeShellScript "entrypoint" ''
${release}/bin/server
'';
dockerImage = pkgs.dockerTools.buildLayeredImage { #
name = "hello";
tag = "latest";
contents = [ weasyprint ]; # If your app depends on any other packages, this is where you can define them
config = {
Env = [
"RELEASE_DISTRIBUTION=none"
"LANG=C.UTF-8"
"LC_ALL=C.UTF-8"
];
Cmd = [ entrypointSh ];
};
};
in {
devShells.${system}.default = pkgs.mkShell {
packages = with pkgs; [ elixir_1_17 mix2nix esbuild dive]
++
pkgs.lib.optionals pkgs.stdenv.isLinux (with pkgs; [
inotify-tools
libnotify
]);
};
packages.${system} = {
packages = with pkgs; [ elixir_1_17 erlang_otp_27];
inherit release;
default = release;
# Here we instruct nix that the "dockerImage" attribute of our flake should export our, well, dockerImage.
dockerImage = dockerImage ;
};
};
}Let's see it in action: start the build with nix build .#dockerImage.
You'll see that our image is being built on multiple layers, as we're using the buildLayeredImage of the dockerTools.
You can look at the size of our image with the ls -lh command. As most things nix, they are symlink to the nix store. You do a simple ls first to find where the linked file is.
> ls
>> -rw-r--r-- 1 owen users 646 Jul 13 10:11 README.md
>> drwxr-xr-x 3 owen users 4.0K Jul 13 10:24 rel
>> lrwxrwxrwx 1 owen users 56 Jul 13 10:45 result -> /nix/store/8bvbli2nmxbknjh6959h13ps6xr7ymjx-hello.tar.gz
>> drwxr-xr-x 4 owen users 4.0K Jul 13 10:11 testThen with ls -lh:
> ls -lh /nix/store/8bvbli2nmxbknjh6959h13ps6xr7ymjx-hello.tar.gz
>> -r--r--r-- 1 root root 165M Jan 1 1970 /nix/store/8bvbli2nmxbknjh6959h13ps6xr7ymjx-hello.tar.gz165M, not bad!
Loading our file into docker:
Now, how do me make use of this? My usual workflow, and in my CI pipelines, is to stop and rename any previous images of the same project, then load the new image. This is what it looks like:
> docker tag hello:latest hello:previous
> docker stop hello
> docker load < result
> docker run -d --rm --name hello --publish 4000:4000 --env-file ./.env hello:latest
After that last command, you should be able to access your release from within the docker container at http://localhost:4000
Hope it helped!
[^3]: nix-direnv github repository
[^4]: Phoenix - Getting started guide
[^5]: https://github.com/ydlr/mix2nix
[^6]: test footote