Deploying production, staging and review apps environments automatically with Kamal on CI/CD

One of the features I truly appreciate about PaaS platforms like Heroku, Netlify, or Vercel is review apps. The idea is simple but powerful: spin up a temporary environment to test new functionality from a Git branch before pushing it to production. It's such a smooth way to catch issues early and share progress with your team.

I wanted to bring that same experience to my deployments with Kamal.

If you’re new to Kamal, I’ve written quite a few blog posts on it that I suggest you check out. They’ll give you a solid foundation for understanding the setup.

ℹ️ This blog post is part of a series about Kamal. Many of the configurations and concepts discussed here remain consistent with those covered in the previous post, How to deploy Rails with Kamal, PostgreSQL, Sidekiq, and Backups on a single host.

This time around, we’ll be diving into Kamal 2, instead of Kamal 1, exploring its new features and improvements. Let's dive in!

I made a video highlighting the steps featured in this article!

1. Set up your server

Ensure the server has sufficient power, as it will host multiple Rails applications, including at least one production app, one staging app, and several review apps.

2. Configure DNS

Next, configure your DNS settings. Since the subdomains will correspond to branch names, we'll use wildcard subdomains.

To simplify server management, it's helpful to create a CNAME record that acts as an alias for your server. This approach allows you to avoid relying on raw IP addresses, which can be hard to remember, and instead use a friendly alias in tools like Kamal or other configurations.

scw-everest 86400 IN A 192.168.0.1 ; Alias for the server.

timecop 86400 IN CNAME scw-everest.example.com. ; CNAME for production.
*.timecop 86400 IN CNAME timecop.example.com. ; Wildcard for staging and all review apps.

3. Prepare your server

With Kamal, everything runs in Docker, so theoretically, you don't need to install anything additional on your server. Kamal will even install Docker for you automatically.

However, if you've just set up a new server, it's a good idea to configure basic security settings and optimize the default configuration.

To simplify this process, I've created an Ansible playbook to automate these actions. This is especially useful when deploying your application to multiple servers.

If you're interested, you can follow the instructions here:
👉 Kamal Ansible Manager

This playbook handles tasks such as upgrading existing packages and installing essential tools like the UFW firewall, Fail2ban, and NTP.

Once cloned, you can use the CNAME you just created to prepare your server by specifying it in your hosts.ini file.

[webservers]
scw-everest.example.com ansible_become_method=su ansible_user=root

$ ansible-playbook -i hosts.ini playbook.yml

Finally, reboot the server, and voilà! Your server is now secure and ready to host your applications.

4. Set up Kamal configuration files

This is the key step where the core setup takes place. We'll rely heavily on Kamal's destination configuration.

The production environment will serve as the default configuration. Below is a simplified example of my config/deploy.yml file. For this setup, I'm using the server's CNAME alias for the hosts and limiting memory usage to 256 MB.

Additionally, it's necessary to configure the app_port on the proxy, as the default is not set to 3000.

# Name of your application. Used to uniquely configure containers.
service: my_awesome_app

# Name of the container image.
image: your-name/my_awesome_app

# Deploy to these servers.
servers:
  web:
    hosts:
      - scw-everest.example.com
    options:
      memory: 256m

  job:
    hosts:
      - scw-everest.example.com
    cmd: bundle exec sidekiq
    options:
      memory: 256m

proxy:
  ssl: true
  app_port: 3000
  host: timecop.example.com

# Inject ENV variables into containers (secrets come from .kamal/secrets).
env:
  clear:
    RAILS_ENV: production
    POSTGRES_USER: 'timecop'
    POSTGRES_DB: 'timecop_production'
    POSTGRES_HOST: 'timecop-db' # With the pattern: <service_name>-<accessory_name>
  secret:
    - RAILS_MASTER_KEY
    - POSTGRES_PASSWORD

accessories:
  db:
    image: postgres:16
    host: scw-everest.example.com
    env:
      clear:
        POSTGRES_USER: 'timecop'
        POSTGRES_DB: 'timecop_production' # The database will be created automatically on first boot.
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data

Next, we'll create a destination for the staging environment. This environment plays a crucial role, as all the review apps will share the same database with it. Managing a single PostgreSQL instance for both staging and review apps is far more efficient than setting up separate instances for each review app, especially when those instances have no data.

To achieve this, create a file named config/deploy.staging.yml. The configurations in this file will automatically merge with those in config/deploy.yml, so you only need to include the settings you want to override.

# Name of your application. Used to uniquely configure containers.
service: timecop-staging

# Name of the container image.
image: timecop-staging

proxy:
  host: staging.timecop.example.com

env:
  clear:
    RAILS_ENV: staging
    POSTGRES_USER: 'timecop'
    POSTGRES_DB: 'timecop_staging'
    POSTGRES_HOST: 'timecop-staging-db' # With the pattern: <service_name>-<accessory_name>

accessories:
  db:
    image: postgres:16
    host: scw-everest.example.com
    env:
      clear:
        POSTGRES_USER: 'timecop'
        POSTGRES_DB: 'timecop_staging' # The database will be created automatically on first boot.
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data

By default, secrets in Kamal are scoped to specific destinations. However, to simplify management, we can share secrets across multiple destinations. To do this, place all shared secrets in the .kamal/secrets-common file.

You can still define destination-specific secrets in .kamal/secrets for production or .kamal/secrets-staging for staging if needed.

Before the first deployment, be sure to run the kamal setup command to initialize the configuration.

$ kamal setup # For the production
$ kamal setup -d staging # For the staging

Both applications should now be accessible at timecop.example.com and staging.timecop.example.com. 🚀

5. Configuring the review-app destination

This is where the real magic happens.

Since Kamal is written in Ruby, we can use ERB tags within YAML files to dynamically access environment variables. This technique allows us to deploy review apps dynamically by leveraging environment variables.

The concept is simple: create a new Kamal destination configuration file, such as config/deploy.review-app.yml.

Instead of hardcoding values in the configuration, you can use ERB tags using the following syntax:

key: <%= ENV['YOUR_VARIABLE'] %>

You can even define default values to use if the environment variable does not exist, like this:

key: <%= ENV.fetch('YOUR_VARIABLE', 'default_value') %>

With that in mind, here's an example of my config/deploy.review-app.yml destination configuration file:

service: <%= ENV['SERVICE_NAME'] %> # We are going to use the name of the git branch dynamically here, because the name must be unique
image: <%= ENV['SERVICE_NAME'] %>

proxy:
  host: <%= ENV['HOST'] %> # We are going to use the name of the git branch dynamically here

env:
  clear:
    RAILS_SERVE_STATIC_FILES: true
    RAILS_ENV: staging
    POSTGRES_USER: 'timecop'
    POSTGRES_DB: 'timecop_staging'
    POSTGRES_HOST: 'timecop-staging-db' # With the pattern: <service_name>-<accessory_name>
    HOST: <%= ENV['HOST'] %>

As shown in the example, I'm using the staging database for the review apps. You could take a similar approach for other resources like Redis, S3 buckets, and more. The idea here is to avoid the inconvenience of having an empty database on a newly deployed review app. I also prefer this approach because it avoids the need to create seed data repeatedly to populate the database, this choice is entirely up to you though.

That said, if you prefer, you can dynamically provision a separate database for each review app, just as we've done for staging and production. It's a matter of finding the balance between simplicity and the degree of isolation you require.

You can test this configuration directly in the command line before integrating it into our CI/CD pipeline:

SERVICE_NAME=new-feature HOST=new-feature.timecop.guillaumebriday.fr kamal deploy -d review-app

To tear down the review app, simply ensure that the variables remain consistent:

SERVICE_NAME=new-feature HOST=new-feature.timecop.guillaumebriday.fr kamal remove -y -d review-app

And it should work as expected! 🚀

6. Setting up GitLab CI

Let's take the next step: configuring our pipeline to automatically deploy a review app whenever a Merge Request is opened on GitLab.

For this example, I'm using GitLab since it's my go-to CI/CD platform. However, the core principles are easily adaptable to other tools like CircleCI, GitHub Actions, or similar platforms. Feel free to adapt the concept to suit your preferred tools and workflow.

For those who need a reference, see: https://docs.gitlab.com/ee/ci/.

Below is a sample .gitlab-ci.yml file designed for a Ruby on Rails application, pretty standard for this kind of project:

image: circleci/ruby:3.3.6

stages:
  - lint
  - test
  - review
  - deploy

include:
  - local: '.gitlab/ci/lint.yml'
  - local: '.gitlab/ci/test.yml'
  - local: '.gitlab/ci/review-apps.yml'
  - local: '.gitlab/ci/deploy.yml'

I've divided my configuration into multiple files for better organization, but that's not the focus here. The file we're interested in is the .gitlab/ci/review-apps.yml. This is where the logic happens for setting up review apps:

.setup_deploy_env: &setup_deploy_env
  only:
    - merge_requests
  image: docker:dind
  services:
    - docker:dind
  variables:
    SERVICE_NAME: timecop-${CI_COMMIT_REF_SLUG}
    REVIEW_APP_HOST: scw-everest.guillaumebriday.fr
    HOST: $CI_COMMIT_REF_SLUG.timecop.guillaumebriday.fr
  before_script:
    - 'command -v ssh-agent >/dev/null || (apk update && apk add openssh-client)'
    - eval $(ssh-agent -s)
    - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "${REVIEW_APP_HOST}" >> ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts
    - apk update && apk add bash curl git yaml-dev build-base openssl-dev readline-dev zlib-dev libffi-dev
    - git clone https://github.com/rbenv/rbenv.git ~/.rbenv
    - echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
    - echo 'eval "$(rbenv init -)"' >> ~/.bashrc
    - source ~/.bashrc
    - git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
    - rbenv install 3.3.6 && rbenv global 3.3.6
    - gem install kamal

deploy_review_app:
  <<: *setup_deploy_env
  stage: review
  script:
    - kamal deploy -d review-app
  environment:
    name: review/$CI_COMMIT_REF_NAME
    url: 'https://${HOST}'
    auto_stop_in: 1 day
    on_stop: stop_review_app

stop_review_app:
  <<: *setup_deploy_env
  stage: review
  script:
    - kamal remove -y -d review-app
  when: manual
  environment:
    name: review/$CI_COMMIT_REF_NAME
    action: stop

At first glance, the configuration might seem a bit complex, but once you break it down and consider the steps we've already covered, it becomes quite straightforward and easy to follow.

Here's the key section to focus on:

.setup_deploy_env: &setup_deploy_env
  only:
    - merge_requests # We want to run this job only when we open a merge request.
  image: docker:dind # We use this image to be able to use Docker in Docker (dind). This is required to run kamal deploy and build the image.
  services:
    - docker:dind
  variables: # These environment variables will be used in our `config/deploy.review-app.yml` destination configuration file.
    SERVICE_NAME: timecop-$CI_COMMIT_REF_SLUG
    REVIEW_APP_HOST: scw-everest.guillaumebriday.fr
    HOST: $CI_COMMIT_REF_SLUG.timecop.guillaumebriday.fr

The $CI_COMMIT_REF_SLUG variable is a predefined environment variable in GitLab CI, automatically provided for your pipelines. For details, see: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html#predefined-variables.

This variable is incredibly useful because it represents a parameterized version of the current branch name. For instance, if you're working on the branch feature/new-homepage, $CI_COMMIT_REF_SLUG will automatically be transformed into feature-new-homepage. This is particularly handy when deploying review apps or managing branch-specific tasks.

In the before_script section of the .gitlab-ci.yml file, you'll need to handle the SSH setup and install Ruby to prepare the environment.

Before you can deploy, it's crucial to configure SSH access between GitLab and your server. To do this, use the SSH_PRIVATE_KEY variable in GitLab CI. This requires adding an SSH key to your server and copying the private key into the GitLab CI/CD variables. For more information, check out the documentation: https://docs.gitlab.com/ee/ci/variables/index.html.

7. Automating deployment with GitLab CI

At this stage, the main job in the pipeline is designed to handle deployment automatically. It simply executes the kamal deploy command, ensuring that the correct flags and environment variables are passed along.

deploy_review_app:
  <<: *setup_deploy_env
  stage: review
  script:
    - kamal deploy -d review-app
  environment:
    name: review/$CI_COMMIT_REF_NAME
    url: "https://${HOST}"
    auto_stop_in: 1 day
    on_stop: stop_review_app

In this step, the environment key is utilized specifically for GitLab's environments feature. This allows us to map our review apps to GitLab's environment management tools seamlessly.

Now, whenever you open a Merge Request on a new branch or push new commits, your review app will be deployed automatically! 🚀

What's even better is that you can manage deployments directly from the GitLab UI. You can trigger a new deployment or stop an active one with just a few clicks! An incredibly convenient feature when working with non-tech teams, like QA.

8. Conclusion

And that's a wrap! While there's room for further optimization, such as creating a Docker image in the before_script to speed up deployments, it ultimately depends on your specific needs and preferences.

It's worth noting that deployments in CI/CD pipelines can sometimes be slow, and there are certainly ways to enhance performance if needed. That said, this approach has worked reliably for me, providing a balance between simplicity and functionality.

Feel free to adapt and optimize it to fit your workflow.

Happy deploying! 🚀