In this publish I are making an strive to contemporary a brand unique deployment capacity I came up with while working on drwn.io.
I needed it to meet about a requirements:
- Straight forward
- In step with git tags
- Zero-downtime
- Straight forward rollbacks
Increasing an empty faraway within the server
Imagine you already like your mission with some code that is being synchronized with a git provider fancy GitHub. To love a git push based mostly deployment, we like to love our possess faraway. We can push our code to that faraway the same arrangement we attain to GitHub.
It’s doubtless you’ll perhaps attain that with any faraway server you will like by increasing a folder (I’ll name it gitrem/) and executing git init --naked throughout the newly created folder. That declare will initialize an empty git faraway that it’s doubtless you’ll perhaps additionally use.
Now the folder will like some unique issues interior, it ought to appear something fancy:
branches config description HEAD hooks index data logs objects refs
(I could perhaps even bustle the total instructions as if I had root assemble admission to to the server).
In your local computer, streak to your mission’s folder and bustle:
You can like to replace root with your username and /root/gitrem with no topic folder you will like old.
That declare will add a brand unique faraway to our mission. It’s doubtless you’ll perhaps inspect the total remotes with the declare git faraway -v. This might perhaps well perhaps camouflage something fancy:
Executing something after we push unique code
To originate something after a git motion we can use git hooks. Git hooks reside within the .git/hooks folder. There are client-facet hooks and server-facet hooks. Client-facet hooks bustle on the computer doing the motion (your local computer). Server-facet hooks bustle on the computer “receiving” the motion (our faraway server). We like to standing up a publish-receive hook. That hook will bustle after unique code is pushed to the faraway.
Some notes concerning the publish-receive hook: It would’t discontinue the code from being pushed, and to boot you’ll care for linked to the faraway server while it’s executing.
Our Docker photography
Sooner than going via our git hook, let me contemporary how the architecture seems to be to be like. We’ve one load balancer/reverse proxy, I’ll be the use of Caddy here. Then we would be working 2 copies of our web backend. In preference to replicating the docker-originate provider, we can address them as 2 different products and services with different names, they factual bustle the same code. The 2 copies like different names so that we can care for one alive if our deployment is no longer a success. Right here’s the docker-originate.yaml (with some tiny print overlooked):
model: "3.8"
products and services:
caddy:
form:
context: .
dockerfile:
Dockerfile.caddy
# giving it a title to use with ufw-docker
container_name: caddy_cont_1
ports:
- "80: 80"
- "443: 443"
volumes:
- caddy_data:/recordsdata
- caddy_config:/config
web:
form:
context: .
dockerfile:
Dockerfile
ports:
# expose port to localhost too
- "8000: 8000"
web2:
form:
context: .
dockerfile:
Dockerfile
ports:
- "8000"
volumes:
caddy_data:
caddy_config:
There are 2 important issues to peep here.
web and web2 are the same container, but with different names. The utterly distinction is that web exposes the port each to the internal docker-originate community and to localhost (that will seemingly be important later). web2 totally exposes it to the internal docker-originate community.
I’m giving a custom title to the reverse proxy picture. Docker doesn’t play properly with iptables, so I use ufw-docker to standing up the firewall.
Increasing our hook
The hook is a bash script that will attain the following.
- Fabricate the
caddycontainer - Originate the firewall ports (if indispensable)
- Fabricate the
webcontainer - If there are no longer any errors, delivery the
webcontainer. - Wait till
webis ready (with a timeout) - Fabricate and begin
web2. Because it’s the same asweb, ifwebbecame built and bustle accurately, we can safely attain all without prolong withweb2.
We can use some git environment variables to bustle and transfer the code. We’re in 2 of them:
GIT_DIR: location of the farawayGIT_WORK_TREE: location to position the code when it’s bought
We’ve already got the folder for GIT_DIR (the one called gitrem/), but we need one other folder for our code. We can make it wherever we need, let’s name it appcode/.
Then we can like a loop checking for the inputs. The line if [[ $ref =~ .*/main$ ]]; is checking that we are pushing to the most principal division (replace it to master or no topic you utilize if indispensable). Then with git --work-tree=$GIT_WORK_TREE --git-dir=$GIT_DIR checkout -f most principal we are copying the contents from that division to our GIT_WORK_TREE folder (appcode/).
Lastly, we can use curl interior a loop to wait till the first replica is alive and recreate the 2nd one.
Right here’s the chunky code. I included so extra comments interior:
#!/usr/bin/env bash
standing -euo pipefail
fail () { echo $1 >&2; exit 1; }
[[ $(id -u) = 0 ]] || fail "Please bustle 'sudo $0'"
unset GIT_INDEX_FILE
unset GIT_DIR
unset GIT_WORK_TREE
export GIT_DIR=/root/gitrem
export DOCKER_OPTS=""
export GIT_WORK_TREE=/root/appcode
while learn oldrev newrev ref
attain
# replace for tags
if [[ $ref =~ .*/main$ ]];
then
echo "Foremost ref bought. Deploying most principal division to production..."
git --work-tree=$GIT_WORK_TREE --git-dir=$GIT_DIR checkout -f most principal
else
echo "Ref $ref efficiently bought. Doing nothing: totally the most principal division will seemingly be deployed on this server."
fi
performed
# NOTE
# here it's doubtless you'll perhaps additionally attain other operations indispensable to bustle your app fancy environment the acceptable
# permissions to assemble admission to folders, and loads others.
# ufw enable http && ufw enable https
# here is the motive to use a custom container title for the reverse proxy
# these 2 instructions will delivery ports 80 and 443 from outdoor to the specified docker originate provider (caddy_cont_1)
ufw-docker enable caddy_cont_1 443
ufw-docker enable caddy_cont_1 80
cd $GIT_WORK_TREE
# form the containers
docker-originate -f docker-originate.yaml form
# exit code of the final executed declare
# if or no longer it is just not 0, discontinue and exit
if [[ "$?" != "0" ]]; then
echo "error while constructing picture."
exit 1
fi
echo "Starting unique container..."
sleep 1
# delivery reverse proxy and replica 1
docker-originate -f docker-originate.yaml up -d --no-deps caddy
docker-originate -f docker-originate.yaml up -d --no-deps web
# if the first replica did no longer delivery accurately, exit
if [[ "$?" != "0" ]]; then
echo "error while deploying picture."
exit 1
fi
# (no longer obligatory) some sleep time to let the first replica delivery
sleep 5
# sit down up for it to be on hand
attempt_counter=0
# max quantity of curl retries
max_attempts=10
# since the "web" provider is exposing port 8000 to the localhost, we can ship requests to it from our
# script
# the following loop will place a matter to the /healthz enpoint till it receives
# an "okay" response.
# This might perhaps well perhaps retry every 5 seconds and each search data from has a timeout of 6 seconds.
# This might perhaps well perhaps attain a maximum of 10 makes an strive.
# In summary, if the app has no longer began in 10*6*5 = 300 seconds = 5 minutes, exit.
# This quantity will seemingly be to high for many use cases, so replace those variables for your wants.
till $(curl --output /dev/null --max-time 6 --restful --assemble --fail localhost: 8000/healthz); attain
if [ ${attempt_counter} -eq ${max_attempts} ];then
echo "Max makes an strive reached"
exit 1
fi
printf '.'
attempt_counter=$(($attempt_counter+1))
sleep 5
performed
# if the loop finishes accurately, it capacity the "web" provider is up and the /healthz endpoint
# is working, so we can recreate the 2nd replica ("web2")
echo "Replica is up, increasing 2nd replica"
docker-originate -f docker-originate.yaml up -d --no-deps web2
docker-originate -f docker-originate.yaml up -d
It’s doubtless you’ll perhaps customize properly being assessments on your reverse proxy to lower the likelihood of having a failed search data from while apps are getting recreated. In my case (with a Caddyfile) I became the use of:
drwn.io {
reverse_proxy web: 8000 web2: 8000 {
health_interval 300ms
lb_policy least_conn
health_path /healthz
}
}
Technically, there’s a 300ms window the build a search data from might perhaps perhaps fail for the rationale that reverse proxy hasn’t seen that this upstream server is down, and it ought to also forward a search data from to it. We might perhaps perhaps lower that to a lower cost if indispensable.
We like to position that code interior .git/hooks/publish-receive in our faraway server. In our instance it might perhaps perhaps probably perhaps well be /root/gitrem/.git/hooks/publish-receive. Don’t forget to give it execution permissions with chmod +x /root/gitrem/.git/hooks/publish-receive.
Abet to our local computer
We’ve now standing up every thing we need in our faraway. In our local computer we can bustle:
git add .
git commit --enable-empty -m "deploy" && git push production most principal
That will push our code to our custom git faraway, the publish-receive hook will bustle, and we can like our app built and working!
Rollbacks and more straightforward deployments
Now we like deployments in line with git push, but we can attain higher. We can attain it the use of git tags. In case you don’t know what git tags are, it’s doubtless you’ll perhaps additionally agree with it’s doubtless you’ll perhaps additionally be taking part in a on-line game the build it’s doubtless you’ll perhaps additionally build your game. You build rather in most cases (git push), but some saves are more important, and to boot you will give them a title or ID, that’s a git label. Git tags are in most cases old to name release variations. We can use them to name variations in our app, we need the following:
- Make a label for a particular release
- Pass our custom faraway to that label. The
publish-receivehook will assemble executed with the code we had after we created the label.
To make a label, we can bustle the following instructions. They’ll make a brand unique commit and an associated label called v2.
# add recordsdata
git add .
git commit --enable-empty -m "tagger"
git label -a v2 -m "model v2"
We can assemble the commit hash linked to that label the use of git rev-listing: git rev-listing -n 1 v2.
For this situation, we’ll agree with our hash is 425368b5 (a accurate hash is longer than that).
Okay, we like tags and the commit hash associated to that label. The very last thing we need is some system to transfer our custom faraway to that commit. Fortunately, there’s also a declare for that:
git push -f production +425368b5:most principal
In case it’s doubtless you’ll perhaps additionally be wondering concerning the +425368b5:most principal, here’s called refspec. The tl;dr is:
425368b5: commit referencemost principal: division title+: replace the reference even though it isn’t a lickety-split-forward
That will affect our custom faraway streak to that particular commit, attain a checkout and standing off the publish-receive hook. Now will seemingly be a legit time to wrap issues in bash functions:
characteristic label {
git add -u .
git commit --enable-empty -m "tagger"
git label -a "$1" -m "model $1"
}
characteristic totag {
tagname="$1"
git push -f production +"$(git rev-listing -n 1 $tagname)":most principal
}
characteristic deploy {
label "$1"
totag "$1"
}
Now we can bustle deploy v2 and bam! We’ve our app working. In case it’s doubtless you’ll perhaps additionally be making an strive to roll help to a old label, it’s doubtless you’ll perhaps additionally attain it by working totag v1 (or every other label title).
Additional: it’s doubtless you’ll perhaps additionally inspect the total tags in a git repository sorted by advent date with the declare:
git label --form=taggerdate
This might perhaps well also be a legit likelihood to give the Taskfile a try.
Wrapping up
We’ve created a custom git faraway with a publish-receive hook. This might perhaps well perhaps form our app as a docker container after we push unique code. We can use about a git instructions to transfer that faraway to a particular commit. Lastly, we like also old git tags to name important commits (releases).
This old to be a field selling a e-newsletter, but I judge we already like enough newsletters. All americans wants to assemble a standing on your mailbox.
In preference to that, subscribing to my RSS feed will seemingly be extra special higher, and no more intrusive on your life.
In case it’s doubtless you’ll perhaps additionally be making an strive for a e-newsletter to subscribe to, click here.
