Quickly Improve your Docker Compose configuration

Docker has become the ubiquitous solution to standardize and deploy applications, and it's something I rely on for work and personal projects. There are many ways to configure and use Docker, ranging from basic "docker run" commands to utilizing Nomad clusters and beyond.

The way most people set up and run Docker containers is with Docker Compose. In this tutorial, we'll talk about how to make Docker Compose easier to use, safer, and more stable.

Basic Configuration

Let's start with the basic configuration of Docker Compose, which includes two component services (backend and frontend), a compose file located in the root folder, and a dedicated Dockerfile for each service.

project\
  backend\
    Dockerfile
    ...
  frontend\
    Dockerfile
    ...
  docker-compose.yaml

services:
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    ports:
      - "8000:80"
    networks:
      - app-network

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    ports:
      - "8080:80"
    depends_on:
      - backend
    networks:
      - app-network

networks:
  app-network:
    driver: bridge

Improvement #1: Always set the project & image name

By default, Docker determines the project name by matching it with the folder name. However, when I review Docker configurations at work, this step is often overlooked. As a result, new team members find it somewhat challenging to troubleshoot the containers.

For instance, if you name the folder "app-new," the container names that come up are "app-new-backend-1" and "app-new-backend-2." However, I found that for some configurations, the folder name is unrelated to what the project/application is about.

name: app

services:
  backend:
    ...
    ...

Adding the name value at the top of the configuration will make things consistent for your team and help reduce unintended confusion.

Next is the image name, while this isn't a concern if you use an external image from Docker Hub, but for some configurations, including what we currently use, it builds a new image from a Dockerfile within the project.

services:
  backend:
    image: gemawardian/app-backend:latest
    build:
      context: ./backend
      dockerfile: Dockerfile
  ...

Even if you are not planning to publish the image, by implementing the image name value in the service, it will help differentiate the image name on the Docker server, especially if there are a lot of Docker images within the server.

Improvements #2: Pull & Restart Policy

By learning how pull_policy works, you will simplify and make your Docker experience easier by telling Docker how the startup behaviour for your use case.

There are multiple possible values for pull_policy as per
Docker documentation:

• always: Compose always pulls the image from the registry.
• never: Compose doesn't pull the image from a registry and relies on the platform's cached image. If there is no cached image, a failure is reported.
• missing: Compose pulls the image only if it's not available in the platform cache. This is the default option if you are not also using the Compose Build Specification. if_not_present is considered an alias for this value for backward compatibility. The latest tag is always pulled, even when the missing pull policy is used.
• build: Compose builds the image. Compose rebuilds the image if it's already present.
• daily: Compose checks the registry for image updates if the last pull took place more than 24 hours ago.
• weekly: Compose checks the registry for image updates if the last pull took place more than 7 days ago.
• every_<duration>: Compose checks the registry for image updates if the last pull took place before <duration>. Duration can be expressed in weeks (w), days (d), hours (h), minutes (m), seconds (s) or a combination of these.

For example, let's say I want to always pull new images when starting the container as a means of automated version update, without running docker compose pull Every time I started the container.

backend:
    image: gemawardian/app-backend:latest
    build:
      context: ./backend
      dockerfile: Dockerfile
    pull_policy: always
    ...

Next is the restart policy, and this is an important one to configure, especially when deploying for a production environment, since application crashes and downtime could occur, and keeping the service uptime high.

backend:
  ...
  restart: unless-stopped
  ...

As in the example above, I specified that unless I stopped the container/service manually, it would keep restarting the service if it unexpectedly crashed or an unintended shutdown occurred.

There are a few restart options available:

• no: The default restart policy. It does not restart the container under any circumstances.
• always: The policy always restarts the container until its removal.
• on-failure[:max-retries]: The policy restarts the container if the exit code indicates an error. Optionally, limit the number of restart retries the Docker daemon attempts.
• unless-stopped: The policy restarts the container irrespective of the exit code, but stops restarting when the service is stopped or removed.

Sometimes you'll need to use a different restart behavior for a database container or service; in order to prevent data corruption, you must troubleshoot the container first and manually start the service in the event of an unexpected shutdown.

Improvements #3: Resource Management

When you start a Docker container, it will use all available resources by default, and this could result in unintended behaviour if the application is having issues such as a memory leak or a race condition, causing the host system to be exhausted of memory or compute resources.

To prevent this, set resource limits and reservations by adding the configuration below to the configuration.

services:
  backend:
    ...
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 128M
    ...

When setting up the resources, there are two types of configuration available:

• limits: The platform must prevent the container from allocating more resources.
• reservations: The platform must guarantee that the container can allocate at least the configured amount.

For instance, a Python application that uses about 60 MB of memory at startup needs to have more than 60 MB added in the reservations section in order to ensure that it is safe from OOM kills.

Things to look for when setting up limits and reservations:

• Set reservations to the absolute minimum required to avoid crashes or slowdown when memory or compute is low. Do not be stingy either.
• Allow limits to be comfortably higher than reservations so that services can use more RAM and CPU when available.
• Instead of guessing, monitor actual memory and CPU usage over time. Adjust the limits and reservations accordingly.
• Consider the implications of performance limits. Find the appropriate balance.
• Set reasonable limits relative to other services to avoid starving critical ones.
• Reservations that exceed the limits will not be allocated more than the limit. Limitations take precedence.

Improvements #4: Health Check

The last thing we'll cover in this article is configuring healthcheck to improve the maintainability of containers and improve overall stability with conditional checking on startup.

services:
  backend:
  ...
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:80"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
  ...

  frontend:
  ...
    depends_on:
      backend:
        condition: service_health
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:80"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 30s
  ...

As shown in the code section above, we implemented a health check for both the frontend and backend services, with one exception: we added depends_on in the frontend to ensure that the frontend is started only when the backend service is ready and healthy.

The health check part follows the convention from the Dockerfile configuration:

• test: Defines the command Compose runs to check container health. It can be either a string or a list.
• interval: The health check will first run every interval seconds after the container is started, and then again every interval of seconds after each previous check completes.
• timeout: If a single run of the check takes longer than the timeout value, then the check is considered to have failed.
• retries: How many consecutive failures of the health check for the container to be considered unhealthy.
• start_period: Provides initialization time for containers that need time to bootstrap. Probe failure during that period will not be counted towards the maximum number of retries.
  However, if a health check succeeds during the start period, the container is considered started, and all consecutive failures will be counted towards the maximum number of retries.

Recapping the Improvements

name: app

services:
  backend:
    image: gemawardian/app-backend:latest
    build:
      context: ./backend
      dockerfile: Dockerfile
    pull_policy: always
    restart: unless-stopped
    ports:
      - "8000:80"
    networks:
      - app-network
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 128M
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:80"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

  frontend:
    image: gemawardian/app-frontend:latest
    build:
      context: ./frontend
      dockerfile: Dockerfile
    pull_policy: always
    restart: unless-stopped
    ports:
      - "8080:80"
    depends_on:
      backend:
        condition: service_healthy
    networks:
      - app-network
    deploy:
      resources:
        limits:
          cpus: "0.5"
          memory: 256M
        reservations:
          cpus: "0.1"
          memory: 64M
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:80"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 30s

networks:
  app-network:
    driver: bridge

To summarize, we successfully improved the barebones Docker Compose configuration by improving the naming structure, pull & restart policy, resource management, and health check setup.

I hope this guide has given you a good understanding of how to use Docker Compose to the fullest.

Have a good day!