Automated Backups with Ansible on Exoscale S3 (SOS)

Apart from the snapshots we create on Exoscale on a regular basis we also wanted to have an automated backup of some key files that would simplify data extraction and specifically, easy access to old database versions.

Exoscale is the Swiss Amazon AWS. Its S3 implementation is pretty advanced and the storage buckets are also accessible through a nice and fast web interface on exoscale.ch (each organisation has its own instances and storage).

Here is how we integrated Exoscale S3 into our deployment:

# This role should be executed as root
---

- name: create s3cmd config file
  template: src=s3cfg.j2 dest=/root/.s3cfg
  become: yes
  become_user: root

- name: Install S3 packages
  apt: pkg={{ item }} update-cache=yes cache_valid_time=3600
  become: yes
  become_user: root
  with_items:
    - s3cmd

- name: create the bucket
  command: chdir={{ project_root }} s3cmd mb s3://{{ project_name }}
  become: yes
  become_user: root


- name: Dump postgres db
  become_user: postgres
  shell: pg_dump {{ db_name }} > /tmp/db.sql
  when: database_system == "postgres"
  become: yes
  become_user: root


- name: Backup media directory
  command: chdir={{ project_root }} s3cmd put --recursive {{ project_media }} s3://{{ project_name }}/{{ backup_folder_name }}/
  become: yes
  become_user: root

- name: Backup locale directory
  command: chdir={{ project_root }} s3cmd put --recursive {{ project_root }}/locale s3://{{ project_name }}/{{ backup_folder_name }}/
  become: yes
  become_user: root

- name: Backup sqlite db
  command: chdir={{ project_root }} s3cmd put {{ project_root }}/db.sqlite3 s3://{{ project_name }}/{{ backup_folder_name }}/
  when: database_system == "sqlite"
  become: yes
  become_user: root

- name: Backup postgres dump
  command: chdir={{ project_root }} s3cmd put /tmp/db.sql s3://{{ project_name }}/{{ backup_folder_name }}/
  when: database_system == "postgres"
  become: yes
  become_user: root

The s3cmd config file looks like this:

[default]
host_base = sos.exo.io
host_bucket = %(bucket)s.sos.exo.io
access_key = {{ exoscale_s3_key }}
secret_key = {{ exoscale_s3_secret }}
use_https = True
signature_v2 = True

This is based on https://community.exoscale.ch/documentation/storage/quick-start/

You can get the key and secret for exoscale S3 as well as the S3 endpoint url from https://portal.exoscale.ch/account/profile/api

The ansible defaults vars are

---

# can be used to create unique directory and file names
datetime_stamp: "{{ lookup('pipe', 'date +%Y%m%d-%H%M') }}"

backup_folder_name: "{{ datetime_stamp }}-{{ deploy_type }}"

Debugging Web Views on Android

  1. connect your phone through USB (+enable USB debugging on the phone somewhere in the settings).
  2. run application on phone (leave it on login screen)
  3. run Chrome on the PC and go to the chrome://inspect/devices#devices URL
  4. Click “Inspect” at the bottom of your phone (should be named)
  5. On the phone login to the system using your credentials
  6. Check chrome what does it have in the console tab. It should print something useful.

by kamil@what.digital

CSS Nesting and Why Its a Good Thing

Please read this first: https://signalvnoise.com/posts/3003-css-taking-control-of-the-cascade

So, in CSS we can establish a hierarchy so that a class only applies if it’s within a parent class. This hierarchy is a good and very much needed thing. Because this way you can define your class names for specific areas of the site with freedom of mind – it will never interfere with or be overwritten by another class that has (accidentally) the same name. Think of class names such as .title. And no, .newsletter-title and .homepage-newsletter-title is not a good idea (CSS supports native hierarchy, so why would you want to fake it?).

Example of SASS Good Practise:

.newsletter
   margin: 20px

   .title-box
         .title
             margin-top: 20px

        .subtitle
            margin-top: 10px

     .email-input
         padding: 50px

Example of SASS Bad Practise:

.title
    margin-top: 20px

.subtitle
    margin-top: 10px

.email-input
    padding: 5px

.newsletter
    margin: 20px

You agree? Are you ready to overshoot the target? Read this: http://thesassway.com/beginner/the-inception-rule

Fachmessen

http://www.igeho.ch/ – November 2017 – Basel
http://www.baselworld.com/ – März 2017 – Basel
http://www.svit-immo-messe.ch/ – März 2017 – Zürich

Trigger jenkins deployment from github push with ansible

We are looking at a simple and robust way to automate deployment. The following show some of the non-intuitive things necessary to set this up.

Jenkins

  • I recommend using the github oauth plugin for authentication and of course the github plugin to easily clone github repos.
  • use https://wiki.jenkins-ci.org/display/JENKINS/Build+Token+Root+Plugin – because the default endpoint to trigger builds cannot be accessed by anonymous users
  • Do not allow job names with spaces: In the jenkins global settings look for Restrict project naming and enter \S*
  • Set up a jenkins job for the repository that contains your ansible deploy script
  • Set up jenkins like this with the following Dockerfile
FROM jenkins
# if we want to install via apt
USER root
# stuff required by jenkins jobs, inlcuding ansible
RUN apt-get update && apt-get install -y php5-cli php5-curl python-setuptools python-dev build-essential libssl-dev libffi-dev

RUN easy_install pip
RUN pip install virtualenv

USER jenkins

Ansible

This script bootstraps ansible

#!/bin/bash

# exit the bash script if one of the command returns an error code
set -e

# Setup a proper path, I call my virtualenv dir "venv" and
# I've got the virtualenv command installed in /usr/local/bin

echo "setting path variable"
PATH=$WORKSPACE/venv/bin:/usr/local/bin:$PATH

echo "create virtualenv"
if [ ! -d "venv" ]; then
    virtualenv venv
fi

echo "activate virtualenv"
source venv/bin/activate

echo "installing ansible"
pip install ansible boto httplib2

echo "tossing in some secrets"
# use the secret file function in jenkins to fill in these variables
mkdir credentials
echo "move $all_yml, $deploy_key_pem and $server_stage_pem to $(pwd)/credentials"
mv "$all_yml" credentials/
mv "$deploy_key_pem" credentials/
mv "$server_stage_pem" credentials/

chmod 600 -R credentials/*

# if ansible roles are pulled in from other git repos:
git submodule update --recursive

echo "deploying"
./deploy stage 3-deploy-site.yml

Github

  • Set up a webhook. Example: http://jenkins.what.digital/buildByToken/build?job=Job-Name&token=2093....234