Here I will show the simplest and fastest way to automate the deployments for web app with any complexity level. We will use a secure SSH connection between the Drone CI host and the Application host. In short, the way is next:
- We will create a docker-compose description that would deliver resources and apps on any machine.
- Order VPS (or dedicated) host with public IP, ssh access, and open application ports.
- Tweak SSH settings on the VPS to allow multisession connections for docker deploys.
- Create bash script which creates ssh private/public key pair inside of deployer-container, then exports
DOCKER_HOST=ssh://[email protected]<host IP>
and runsdocker-compose up
.
All instructions are explained below.
We used Ubuntu 18.04
OS for my Amazon EC2 t3a.small
instance (you can use any hosting). Less than 2 GB might cause memory and hanging up issues.
You need to have the next TCP ports open:
22
for SSH- all ports that your application are listening on (e.g.
80
,443
)
If you use Amazon EC2, all ports are closed by default, but you can open ports in Security Group configuration
. To do it on instance launch dialog:
Many VPS/Dedicated hosting providers have all ports open by default.
Login by keypair
Login as ubuntu
user with .pem
file downloaded from AWS Console:
ssh -i ~/xx.pem ubuntu@xxxxx
Edit file: sudo nano /etc/ssh/sshd_config
Add lines to the end (These settings are required to properly work with docker builds):
PermitRootLogin yes
MaxSessions 500
MaxStartups 500
Run service sshd restart
to apply changes✅.
Add your local public ssh key 🔑 content (cat ~/.ssh/id_rsa.pub
, use ssh-keygen
on your localhost, if you have no such file) to the next file on the remote host (add to beginning with one line break after):
sudo vim /root/.ssh/authorized_keys
By doing this you allow direct connection using your ssh key with a command from your OS user, like this:
ssh [email protected]<APP_HOST_IP>
Prepare docker host
Now on remote host Install docker:
sudo apt update && sudo apt remove docker docker-engine docker.io && sudo apt install docker.io
Prepare repository:
As an example, we will show the config for the Django project.
/.drone.yml
kind: pipeline
name: default
type: docker
concurrency:
limit: 1
steps:
- name: slack-begin
image: plugins/slack
settings:
webhook:
from_secret: slack_webhook
username: Drone
template: >
{{repo.name}}/{{build.branch}} - Started #{{build.number}} "${DRONE_COMMIT_MESSAGE}" by {{build.author}} (<{{build.link}}|Open>)
- name: build
environment:
VAULT_MASTER_SSH_PRIV_KEY:
from_secret: VAULT_MASTER_SSH_PRIV_KEY
VAULT_MASTER_SSH_PUB_KEY:
from_secret: VAULT_MASTER_SSH_PUB_KEY
VAULT_MASTER_PORTAINER_PASSHASH:
from_secret: VAULT_MASTER_PORTAINER_PASSHASH
VAULT_MASTER_DB_PASS:
from_secret: VAULT_MASTER_DB_PASS
VAULT_MASTER_SECRET_KEY:
from_secret: VAULT_MASTER_SECRET_KEY
image: devforth/drone-builder
commands:
- cd deploy && /bin/bash build.sh
- name: slack-end
image: plugins/slack
settings:
webhook:
from_secret: slack_webhook
username: Drone
template: >
{{repo.name}}/{{build.branch}} - #{{build.number}} {{uppercasefirst build.status}} after {{since build.started}} (<{{build.link}}|Open>)
This drone file uses 5 Drone Secrets. VAULT_MASTER_SSH_PRIV_KEY
and VAULT_MASTER_SSH_PUB_KEY
should be copied from your PC to Drone Vault.
VAULT_MASTER_PORTAINER_PASSHASH
- pass for portainer (docker admin panel), you can remove it from config if you don't need portainer.
👆TO GENERATE for VAULT_MASTER_PORTAINER_PASSHASH
use next command (on any Linux/Mac with docker or WSL2 on windows:
docker run --rm httpd:2.4-alpine htpasswd -nbB admin 'yourPortainerPpass' | cut -d ":" -f 2
Other secrets are defined to improve application security (better to store all values in drone vault)
/deploy/build.sh
#!/bin/bash
branch=$(git branch | grep \* | cut -d ' ' -f2)
branch=`echo $branch | sed 's/feature\///g'`
echo "Building branch $branch"
if [ -z "$VAULT_MASTER_SSH_PRIV_KEY" ]; then
echo "Running not on Drone (localy)"
else
mkdir -p ~/.ssh/
echo "$VAULT_MASTER_SSH_PRIV_KEY" > ~/.ssh/id_rsa
echo "$VAULT_MASTER_SSH_PUB_KEY" > ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub
eval `ssh-agent -s`
ssh-add ~/.ssh/id_rsa
echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
fi
if [ "$branch" = "master" ]; then
HOST_DOMAIN=<APP_HOST_IP>
# add more environemnts for branches here, e.g.:
# elif [ "$branch" = "develop" ]; then
else
echo "No configuration for branch $branch"
exit -1
fi
export DOCKER_HOST=ssh://[email protected]$HOST_DOMAIN
# cleanup cache from previous builds
docker builder prune -a -f
docker container prune -f
docker rmi $(docker images -f "dangling=true" -q) || true
# build and run db image (migration will connect to db)
docker-compose -p stack-$branch -f docker-compose.yml -f assets/variables_$branch.yml up -d --build db
# build django app contanier
docker-compose -p stack-migrate-$branch -f docker-compose.yml -f assets/variables_$branch.yml build --compress django
docker-compose -p stack-migrate-$branch -f docker-compose.yml -f assets/variables_$branch.yml run --no-deps --workdir="/code/" django bash -c 'pipenv run python manage.py collectstatic --noinput'
docker-compose -p stack-migrate-$branch -f docker-compose.yml -f assets/variables_$branch.yml run --no-deps --workdir="/code/" django pipenv run python manage.py migrate
docker-compose -p stack-$branch -f docker-compose.yml -f assets/variables_$branch.yml up -d --build --remove-orphans
docker-compose -p stack-$branch restart nginx
stack-migrate-$branch
is used to reduce downtime during redeploys (migrating and collecting static actually performed without old container stopping).
/deploy/docker-compose.yml
version: '3.5'
services:
db:
network_mode: host
image: mysql:5.7
restart: always
command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_bin
volumes:
- v-db-data:/var/lib/mysql
django:
network_mode: host
build: ../src/
command: /bin/bash -c "cd /code/ && pipenv run gunicorn --worker-tmp-dir /dev/shm conf.wsgi:application --bind 0.0.0.0:8001 --workers 5 --worker-connections=1000"
# working_dir: /root/
restart: always
volumes:
- v-django:/code/serve_static
nginx:
network_mode: host
build: nginx
restart: always
volumes:
- type: volume
source: v-django
target: /static/django
read_only: true
volume:
nocopy: true
command: /bin/bash -c "envsubst '$$CERT_NAME $$DOMAIN $$BASIC_AUTH_ENABLE' < /etc/nginx/conf.d/conf.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
portainer:
image: portainer/portainer:1.24.1-alpine
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- v-portainer-data:/data
volumes:
v-portainer-data:
name: v-portainer-data
v-db-data:
name: v-db-data
v-django:
name: v-django
network_mode: host
is recommended for better networking performance (no docker virtual DNS names resolution)
/deploy/assets/variables_master.yml
This file is used to configure variables per one environment.
version: '3.5'
services:
db:
environment:
- MYSQL_ROOT_PASSWORD=${VAULT_MASTER_DB_PASS}
- MYSQL_DATABASE=db_live
nginx:
environment:
# - BASIC_AUTH_ENABLE=1
- DOMAIN=yur_site.com
- CERT_NAME=yur_site_cert
django:
environment:
- ENV_NAME=live
- DEBUG=False
- SECRET_KEY=${VAULT_MASTER_SECRET_KEY}
- DOMAIN=hinty.io
- DATABASE_URL=mysql://root:${VAULT_MASTER_DB_PASS}@127.0.0.1:3306/db_live?charset=utf8mb4
portainer:
command: --admin-password=${VAULT_MASTER_PORTAINER_PASSHASH}
Drone config might look like this: