9. DevOps

Let’s see how we may use Docker to be a part of a simple continuous integration/continuous delivery CICD pipeline. In this example, we have an demo Python API. The file and directory structure looks like the following. It’s not important what this Python API does since we are more interested in how to use Docker to build, test and publish the API to pypi.

python/
├── Dockerfile
├── Makefile
├── publish.sh
├── pydemo
│   ├── __init__.py
│   └── poco.py
├── README.md
├── README.rst
├── requirements.txt
├── setup.cfg
├── setup.py
└── tests
    ├── __init__.py
    └── test_poco.py

Using Docker containerization to perform CI/CD is just a matter of scripting. Here’s the script publish.sh that builds, tests and publishes the API. The important caveat here is the acquisition of the ${API_VERSION} environment variable value. We pass this value in to the Docker container and the script now has access to it.

#!/bin/bash

SOURCE_DIST=/pydemo/dist/pydemo-${API_VERSION}.tar.gz

buildCode() {
  echo "start the build"
  cd /pydemo \
    && make clean \
    && make \
    && python setup.py sdist bdist bdist_wheel \
    && twine check dist/*
}

updateVersion() {
  echo "replace version of software to ${API_VERSION}"
  sed -i "s/version='0.0.1'/version='${API_VERSION}'/g" /pydemo/setup.py
}

copyCredentials() {
  if [[ -f /pydemo/.pypirc ]]; then
    echo "copying over .pypirc"
    cp /pydemo/.pypirc /root/.pypirc
  fi
}

publish() {
  echo "python publish"

  if [[ -f /root/.pypirc ]]; then
    if [[ -f ${SOURCE_DIST} ]]; then
      echo "uploading source"
      cd /pydemo \
        && make clean \
        && python setup.py sdist \
        && twine upload --repository ${PYPI_REPO} ${SOURCE_DIST}
    else
      echo "no ${SOURCE_DIST} found!"
    fi
  else
    echo "no .pypirc found!"
  fi
}

cleanUp() {
  if [[ -f /root/.pypirc ]]; then
    echo "cleaning up"
    rm -f /root/.pypirc
  fi
}

build() {
  echo "python build"
  buildCode
  publish
}

conda init bash
. /root/.bashrc
updateVersion
copyCredentials
build
cleanUp

echo "done!"

Here’s the Dockerfile. Here, we use the ARG instruction to pass build arguments to the container while it’s being created.

FROM continuumio/anaconda3

ARG ARG_VERSION
ARG ARG_REPO

ENV API_VERSION=$ARG_VERSION
ENV API_REPO=$ARG_REPO
ENV PATH /opt/conda/bin:$PATH

RUN apt-get update \
    && apt-get upgrade -y \
    && apt-get install build-essential -y
    
WORKDIR /pydemo

COPY . .

RUN conda install --file requirements.txt -y

RUN /pydemo/publish.sh

As we iterate through our development, we perform CI/CD by executing the command as below. Here, we specify that the API is at version 0.0.1.

docker build --no-cache \
    --build-arg ARG_VERSION=0.0.1 \
    --build-arg ARG_REPO=pydemo \
    -t pydemo:0.0.1 .

If we are at version 0.0.2, then the command changes to something as simple as the following.

docker build --no-cache \
    --build-arg ARG_VERSION=0.0.2 \
    --build-arg ARG_REPO=pydemo \
    -t pydemo:0.0.1 .