Deploy docker-compose using Drone CI

Here I will show the simplest and fastest way to automate the deployments for the web apps with any complexity level. We will use a secure SSH connection between the Drone CI host and the Application host. In short, the way is next:

  1. We will create a docker-compose description that delivers resources and apps on any machine.
  2. Order VPS (or dedicated) host with public IP, ssh access, and open application ports.
  3. Tweak SSH settings on the VPS to allow multisession connections for docker deploys.
  4. Create bash script which creates ssh private/public key pair inside of deployer-container, then exports DOCKER_HOST=ssh://[email protected]<host IP> and runs docker-compose up.

All instructions are explained below.

We used Ubuntu 18.04 OS for my Amazon EC2 t3a.small instance (you can use any hosting). Less than 2 GB might cause memory and subsequent hanging up issues.

You need to have the next TCP ports open:

If you use Amazon EC2, all ports are closed by default, but you can open ports in Security Group configuration. To do it on instance launch dialog:

EC2 Security Group step

Many VPS/Dedicated hosting providers have all ports open by default.

Login by keypair

Login as ubuntu user with .pem file downloaded from AWS Console:

ssh -i ~/xx.pem ubu[email protected]

Edit file:

sudo nano /etc/ssh/sshd_config

Add lines to the end (These settings are required to properly work with docker builds):

PermitRootLogin yes
MaxSessions 500
MaxStartups 500

Run:

sudo service sshd restart

To apply changes✅.

Add your local public ssh key 🔑 content (cat ~/.ssh/id_rsa.pub, use ssh-keygen on your localhost, if you have no such file) to the next file on the remote host (add to beginning with one line break after):

sudo nano /root/.ssh/authorized_keys

By doing this you allow direct connection using your ssh key with a command from your OS user, like this:

ssh [email protected]<APP_HOST_IP>

Prepare docker host

Now, on remote host, uninstall all old dockers:

sudo apt remove docker docker-engine docker.io

Installing docker

 sudo apt update && sudo apt install docker.io

Prepare repository:

As an example, we will show the config for the Django project.

/.drone.yml

kind: pipeline
name: default
type: docker
concurrency:
limit: 1
steps:
- name: slack-begin
image: plugins/slack
settings:
webhook:
from_secret: slack_webhook
username: Drone
icon_url: ${DRONE_COMMIT_AUTHOR_AVATAR}
template: >
{{repo.name}}/{{build.branch}} - Started #{{build.number}} "${DRONE_COMMIT_MESSAGE}" by ${DRONE_COMMIT_AUTHOR} (${DRONE_COMMIT_AUTHOR_EMAIL}) (<{{build.link}}|Open>)
- name: build
environment:
VAULT_MASTER_SSH_PRIV_KEY:
from_secret: VAULT_MASTER_SSH_PRIV_KEY
VAULT_MASTER_SSH_PUB_KEY:
from_secret: VAULT_MASTER_SSH_PUB_KEY
VAULT_MASTER_PORTAINER_PASSHASH:
from_secret: VAULT_MASTER_PORTAINER_PASSHASH
VAULT_MASTER_DB_PASS:
from_secret: VAULT_MASTER_DB_PASS
VAULT_MASTER_SECRET_KEY:
from_secret: VAULT_MASTER_SECRET_KEY
image: devforth/drone-builder
commands:
- cd deploy && /bin/bash build.sh
- name: slack-end
image: plugins/slack
settings:
webhook:
from_secret: slack_webhook
username: Drone
icon_url: ${DRONE_COMMIT_AUTHOR_AVATAR}
template: >
{{repo.name}}/{{build.branch}} - #{{build.number}} {{uppercasefirst build.status}} after {{since build.started}} (<{{build.link}}|Open>)

This drone file uses 5 Drone Secrets. VAULT_MASTER_SSH_PRIV_KEY and VAULT_MASTER_SSH_PUB_KEY should be copied from your PC files (use cat ~/.ssh/id_rsa for first and cat ~/.ssh/id_rsa.pub) to Drone Vault.

VAULT_MASTER_PORTAINER_PASSHASH - password hash for Portainer (docker admin panel), you can remove it from config if you don't need Portainer.

👆TO GENERATE for VAULT_MASTER_PORTAINER_PASSHASH use next command (works on any Linux/Mac or Docker on WSL2 on windows):

docker run --rm httpd:2.4-alpine htpasswd -nbB admin 'yourPortainerPpass' | cut -d ":" -f 2

Other secrets are defined to improve application security (better to store all sensitive values in drone vault)

/deploy/build.sh

#!/bin/bash
branch=$(git branch | grep \* | cut -d ' ' -f2)
branch=`echo $branch | sed 's/feature\///g'`
echo "Building branch $branch"
if [ -z "$VAULT_MASTER_SSH_PRIV_KEY" ]; then
echo "Running not on Drone (localy)"
else
mkdir -p ~/.ssh/
echo "$VAULT_MASTER_SSH_PRIV_KEY" > ~/.ssh/id_rsa
echo "$VAULT_MASTER_SSH_PUB_KEY" > ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub
eval `ssh-agent -s`
ssh-add ~/.ssh/id_rsa
echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
fi
if [ "$branch" = "master" ]; then
HOST_DOMAIN=<APP_HOST_IP>
# add more environemnts for branches here, e.g.:
# elif [ "$branch" = "develop" ]; then
else
echo "No configuration for branch $branch"
exit -1
fi
export DOCKER_HOST=ssh://[email protected]$HOST_DOMAIN
# cleanup cache from previous builds
docker builder prune -a -f
docker container prune -f
docker rmi $(docker images -f "dangling=true" -q) || true
# build and run db image (migration will connect to db)
docker-compose -p stack-$branch -f docker-compose.yml -f assets/variables_$branch.yml up -d --build db
# build django app contanier
docker-compose -p stack-migrate-$branch -f docker-compose.yml -f assets/variables_$branch.yml build --compress django
docker-compose -p stack-migrate-$branch -f docker-compose.yml -f assets/variables_$branch.yml run --no-deps --workdir="/code/" django bash -c 'pipenv run python manage.py collectstatic --noinput'
docker-compose -p stack-migrate-$branch -f docker-compose.yml -f assets/variables_$branch.yml run --no-deps --workdir="/code/" django pipenv run python manage.py migrate
docker-compose -p stack-$branch -f docker-compose.yml -f assets/variables_$branch.yml up -d --build --remove-orphans
docker-compose -p stack-$branch restart nginx

stack-migrate-$branch is used to reduce downtime during redeploys (migrating and collecting static actually performed before old container stopping).

/deploy/docker-compose.yml

version: '3.5'
services:
db:
network_mode: host
image: mysql:5.7
restart: always
command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_bin
volumes:
- v-db-data:/var/lib/mysql
django:
network_mode: host
build: ../src/
command: /bin/bash -c "cd /code/ && pipenv run gunicorn --worker-tmp-dir /dev/shm conf.wsgi:application --bind 0.0.0.0:8001 --workers 5 --worker-connections=1000"
# working_dir: /root/
restart: always
volumes:
- v-django:/code/serve_static
nginx:
network_mode: host
build: nginx
restart: always
volumes:
- type: volume
source: v-django
target: /static/django
read_only: true
volume:
nocopy: true
command: /bin/bash -c "envsubst '$$CERT_NAME $$DOMAIN $$BASIC_AUTH_ENABLE' < /etc/nginx/conf.d/conf.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
portainer:
image: portainer/portainer:1.24.1-alpine
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- v-portainer-data:/data
volumes:
v-portainer-data:
name: v-portainer-data
v-db-data:
name: v-db-data
v-django:
name: v-django

network_mode: host is recommended for better networking performance (no docker virtual DNS names resolution is available in this mode, but network performs x1.5 - x2 times better)

/deploy/assets/variables_master.yml

This file is used to configure variables per one environment.

version: '3.5'
services:
db:
environment:
- MYSQL_ROOT_PASSWORD=${VAULT_MASTER_DB_PASS}
- MYSQL_DATABASE=db_live
nginx:
environment:
# - BASIC_AUTH_ENABLE=1
- DOMAIN=yur_site.com
- CERT_NAME=yur_site_cert
django:
environment:
- ENV_NAME=live
- DEBUG=False
- SECRET_KEY=${VAULT_MASTER_SECRET_KEY}
- DOMAIN=hinty.io
- DATABASE_URL=mysql://root:${VAULT_MASTER_DB_PASS}@127.0.0.1:3306/db_live?charset=utf8mb4
portainer:
command: --admin-password=${VAULT_MASTER_PORTAINER_PASSHASH}

Drone config might look like this:

Drone config

Drone.ci poster logo

#droneci #dockercompose #docker #ci #ec2 #django
6
Ivan Borshchov profile picture
Apr 28, 2020
by Ivan Borshchov
Did it help you?
Yes !
No

Best related