By DevOps on Sun 16 June 2019
inWe run high traffic apps on dedicated servers. Elixir runs great on bare metal, as it easily takes advantage of all the cores on the machine. We can get a machine with 24 CPU cores, 32 GB of RAM and 10TB of traffic for under $100 month. Try that in the cloud. You can cut your hosting bills by 90%.
When deploying to bare metal, we use Ansible, an easy-to-use standard tool for managing servers. It has reliable and well documented primitives to handle logging into servers, uploading files and executing commands: just what we need to deploy an Elixir app. We also use it when deploying to AWS, setting up AMIs for use in auto-scaling groups. This guide shows how to set up and run a Phoenix app, talking to a Postgres database. It is based on this working template and the principles in "Best practices for deploying Elixir apps". It is recommended that you get the example up and running before you go on to prepare your own applications for deployment.
On your local dev machine, install Ansible with the Python pip command:
pip install ansible
See the Ansible docs for other options.
git clone https://github.com/cogini/mix-deploy-example.git
Ansible uses ssh to talk to the server. On your local dev machine, add an ssh
host alias in the ~/.ssh/config
file so you can reference the server using
its name.
# Set this to the actual IP of your server
Host web-server1
HostName 123.45.67.89
You can use any name you like, but it needs to match the name in the Ansible
"inventory", e.g. ansible/inventory/hosts
. This file puts hosts
into groups so we can manage them together, e.g.:
[web_servers]
web-server1
web-server2
[db_servers]
db-server1
db-server2
[web-servers]
is a group of web servers. web-server1
is a hostname from
the Host
line in your .ssh/config
file.
The configuration variables defined in inventory/group_vars/all
apply to all hosts in
your project. They are overridden by vars for more specific groups like
inventory/group_vars/web_servers
or for individual hosts, e.g.
inventory/host_vars/web-server
.
We use our elixir-release Ansible role to set up and deploy the Elixir app.
Under the ansible
dir, edit inventory/group_vars/all/elixir-release.yml
to
match the Elixir app you are deploying:
# External name of the app, used to name directories and the systemd unit
elixir_release_app_name: mix-deploy-example
See the docs for the elixir-release Ansible role for more options.
We use our users Ansible role to manage the accounts used to deploy and run the app, as well as control access for system administrators and developers.
For security, we use separate accounts to deploy the app and to run it. The
deploy account owns the code and config files, and has rights to restart the
app. We normally use a separate account called deploy
. The app runs under a
separate account with the minimum permissions it needs. We normally create a
name matching the app, e.g. foo
or use a generic name like app
.
Configure users and associated ssh keys in inventory/group_vars/all/users.yml
.
The following defines a user jake
, getting the ssh keys from their GitHub profile,
and a user ci
for a CI/CD server account, getting the ssh key from a file.
users_users:
- user: jake
name: "Jake Morrison"
github: reachfh
- user: ci
name: "CI server"
key: ci.pub
users_deploy_users
defines users that are allowed to log into the deploy
account on the server via ssh, e.g.:
users_deploy_users:
- jake
- jenkins
users_global_admin_users
defines admin users, e.g. for your ops team. The
following creates a separate account called bob
on the server with sudo:
users_global_admin_users:
- bob
See the docs for the users Ansible role for more options.
We split the deployment into two phases, setup and deploy. In the setup phase, we do the tasks that require elevated permissions, e.g. creating user accounts, creating app dirs, installing OS packages, and setting up the firewall.
In the deploy phase, we push the latest code to the server and restart it. The deploy doesn't require admin permissions, so it can run from a regular user, e.g. the build server.
Once things are configured, run Ansible to do initial server setup, creating users and configuring the firewall.
From your local dev machine, run this playbook:
ansible-playbook -u root -v -l web_servers playbooks/setup-web.yml -D
The -u
flag specifies the user for bootstrapping, after that you would
normally use your own admin user. The user needs to be root or have sudo
permissions. Depending on the hosting provider's provisioning process, that
might be root
or a default user like centos
or ubuntu
with your keypair
installed. See playbooks/manage-users.yml
for other connection options, e.g.
specifying a root password manually.
The -v
flag controls verbosity, you can add more v's to get more debug info.
The -D
flag shows diffs of the changes Ansible makes on the server.
If you add --check
to the Ansible command, it will show you the changes it is
planning to do, but doesn't actually run them.
This playbook uses the iptables
and iptables-http
roles to set up the base
firewall and port forwarding
to redirect port 80/443 to the non-privilged port the app listens on.
Next, set up the server to run the app, creating directories and configuring the systemd unit. The mix_systemd Elixir library creates a systemd unit file for the app, and the mix_deploy library generates utility scripts.
When deploying to a single server, you can build on the server and run the scripts to set it up. When deploying to more servers, we use the elixir-release Ansible role. It creates directories and copies the generated files from the local project to the server.
From your local dev machine, run this playbook:
ansible-playbook -u $USER -v -l web_servers playbooks/deploy-app.yml --skip-tags deploy -D
This runs all the tasks in the role except those tagged with deploy
, which
are instead run from the CI/CD server.
Instead of baking secrets like db passwords into the release file, we create a
config file and copy it to the app config dir under /etc
.
Here we use the Ansible vault to manage app secrets.
First, configure the db server settings in inventory/group_vars/all/db.yml
db_password
in inventory/group_vars/all/secrets.yml
and secret_key_base
in inventory/group_vars/web_servers/secrets.yml
.
From your local dev machine, run this playbook to generate a TOML config file with the secrets and push it to the web servers:
ansible-playbook -u $USER -v -l web_servers playbooks/config-web.yml -D
See templates/app.config.toml.j2
.
Finally, we are ready to deploy the app. We build the release on a build server, then push it to prod servers and restart.
On a CI/CD server, run the following playbook:
ansible-playbook -u deploy -v -l web_servers playbooks/deploy-app.yml --tags deploy --extra-vars ansible_become=false -D
-u deploy
specifies that the CI server should connect to the target server as
the deploy
user using the ssh key we set up before.
Ansible is useful for all sorts of admin tasks.
This playbook sets up the build server, installing ASDF version manager and checking out the app from git:
ansible-playbook -u root -v -l build_servers playbooks/setup-build.yml -D
See inventory/group_vars/build_servers/vars.yml
, particularly app_repo
for
the URL of the git repo.
You can install Ansible on the build machine with:
ansible-playbook -u $USER -v -l web_servers playbooks/setup-ansible.yml -D
The following playbook sets up a Postgres database:
ansible-playbook -u $USER -v -l db_servers playbooks/setup-db.yml -D
Configuration is in inventory/group_vars/db_servers/postgresql.yml
.