Intro to docker compose

In this post we want to look at ways to orchestrate containers with docker compose instead of doing this manually all the time.

We will look at the following points:

• Why should you utilize docker-compose
• How does it work on a conceptual level
  • yaml file
  • example
  • command reference

Why docker compose?

Often when we are working with docker we want to have more than one containerized application working together in some form.
Then in most cases they depend on a SDN (software defined network), other containers to be available and different configurations like volumes and such.

One way to orchestrate this is to start every container by hand and then link them together in a network or with the deprecated –link flag.
Another way could also be to write a script that automates this for you.
But as you can probably tell this will be error prone and not the generic solution you would like to have for this kind of problem.

This is were docker-compose comes into play.
With docker-compose you get a standard to describe complex applications in a consistent and predictable manner which is done by a docker compose file.
Behind the scenes a docker-compose sets up all the needed infrastructure like SDNs and volumes for all specified containers.

The docker-compose.yml file contains all the images to be build and container run instructions you would normally do manually from the command line.

In essence docker-compose constructs the application containers and the relationship between them.

You can also use docker-stack for such orchestration but we will look at this with another post on docker swarms because docker stack works for swarms on a cluster as docker-compose works locally on a single machine. (Although for clusters you probably want something like kubernetes)
Although to replicate docker-compose you could create a swarm with a single machine, but we will look into docker swarm and docker stack in another introduction. Docker-compose is also easier to grasp at first.

How does docker compose work?

To work with docker-compose you need to set up a docker-compose.yml file in your working directory.
This is a yaml file that describes all the build and container run instructions you would normally employ by the docker command line tools.

If you are not familiar with yaml following are the most important rules in short:

• — new Section
• # comments one line
• strings are either in “double” or ‘single’ quotes
• Lists are defined with hyphens like so
  • – list_item_1
  • – list_item_2
  • – list_item_3
• it is also allowed to define a list like so:
  • [list_item1, list_item_2, list_item3]
• dictionaries are either defined with a colon or with curly brackets
  • key: value
  • { key: value, key2:value2 }
• | defines a block where carriage return is preserved
• > defines a line with no carriage return but with empty lines preserved
• You are not allowed to use tabs but only spaces
• To analyze yaml files you can use python shyaml to read yaml files and check them for validity.

Pitfalls of docker-compose:

Be aware that docker-compose does not wait for a container to be ready (whatever this means in terms of your application) to start a dependent container.
A common scenario here is when a database container is setup while another (web) application is trying to access this database. I have a docker-compose recipe where this can be the case and have some external soultions presented in the advanced scenarios there.

Nevertheless the best way to handle this will be from inside the application. This is because for distributed systems you should always be prepared for failures in the system anyways.

Hands on docker-compose

For a simple hands-on see my recipe on how to use docker-compose with a local wordpress installation depending on a maria db backend database.

Be aware that I am using macOS which can lead to some issues on Linux or Windows versions (e.g. with the volume because there is some issues with the docker daemon running in a virtual machine on macOS)

Next up we will look at the most important commands inside a docker-compose.yml file to be used.

Build:

Do you pull an image from a repository like the docker-hub or do you utilize dockerfile, both is possible with the following syntax.

..
build: <directory with Dockerfile>
# or
build:
context: .
dockerfile: Dockerfile

Network

• Always creates an own network for the composed containers
• If you want to use your own SDN you need to create them in the file:

  services:
  web:
  ...
  networks:
  - my_net_1
  # Top Level
  networks:
  my_net_1:
  external:
  name: host

  Network-ports

Make the opened ports in the container available to the outside world

ports:
- "8082:80"
- "8443:443"

always:<maps host_port:container_Port

if you only want to expose the port(s) only in the container network (SDN) then use expose

expose:
- "7500"
- "7501"

Volumes

The key word volumes follows a list of folders where you want the volumes of the image to be put.
Again the mapping looks like: volume_mapping_host:volume_mapping_container

volumes:
- /tmp/db:/var/lib/mysql
# path can be relative to docker-compose.yml file
# ./<path>
#  be aware that you cannot map a file to a directory and vice versa:
# not allowed!!
volumes:
- ./default.conf:/etc/nginx/conf.d/
# use like so
volumes:
- ./default.conf:/etc/nginx/conf.d/default.conf

If you want to use the volume for more than one service you need to specify it as a top level section

version: '3'
services:
nginx:
volumes:
- webdata:/var/www/html
wordpress:
volumes:
- webdata:/var/www/html
volumes:
webdata:

Environment variables

The environment key word follows a list or key value pairs to specify your envrionment variables for a service.
The following listings are equivalent

environment:
WORDPRESS_DB_HOST: mariadb
WORDPRESS_DB_NAME: docker_compose_db
environment:
- WORDPRESS_DB_HOST=mariadb
- WORDPRESS_DB_NAME=docker_compose_db

If you have a lot of environment variables to be set you can also utilize a environment config file like so:

env_file: <path_to_file>

Entrypoint/CMD

You can also override the Entrypoint of the container or the command that is executed in a container on the start.

entry point: ["some-script.sh"]
command: ["/bin/bash"]
command: ["php", "-a"]
# or
command:
- "php"
- "-a"

Secrets

You can read secretes like database passwords and such from a file by creating a toplevel secrets directive.

version: '3'
services:
secrettest:
image: alpine
secrects:
- password
secrets:
password:
file: ./top-secret.txt

Restart behavior

With this you can control how the container restarts if the running docker host is stopped and restarted, or if the container fails during its lifetime. The default for this is no.
The other options are:

• always
• on-failure
• unless-stopped

Docker-compose command reference

Last but not least here is some reference of the docker-compose commands that can be used.

• docker-compose config checks if syntax of docker-compose.yml is correct
  be aware that this sorts all the keys alphabetically
• docker-compose down stops all in docker-compose.yml described services, removes networks and volumes
• docker-compose events shows all events that happened for the container that were setup with compose
• docker-compose kill stops all containers without hesitation (do only use when docker-compose down does not work)
• docker-compose logs <servicename> shows container logs (only works for running containers), better use docker-compose up without -d flag
• docker-compose pause stops execution, unpause starts it again
• docker-compose ps lists all containers of a docker-compose.yml
• docker-compose rm removes all stopped containers, with -v all volumes are removed too
• docker-compose run <servicename> starts a single container and its dependencies if specified by depends_on
• docker-compose start/stop/restart says exactly what it does for all containers of yml file
• docker-compose top shows all the processes of the containers with its UID,PID,STIME,TIME CMD and other information
• docker-compose up launches all the containers that are listed in the top-level section of the docker-compose.yml. It creates a network for those containers so that those can communicate. All container tags will be prefixed with the current directory. For debugging you can leave the -d and have all loggings from the services displayed on the terminal