Docker, not just a container of confusion

December 6, 2022

Hello everyone!

I’ve been doing web development for quite a while now, and I’ve seen a lot of tools and technologies come and go. Docker was one of those tools that I didn’t dig far into. For the longest time, I couldn’t confidently tell you the difference between a container and a virtual machine.

But now I can! A virtual machine is an emulated version of a computer system, complete with its own operating system and resources. A container, on the other hand, is just a different namespace on the host operating system, so it’s essentially just another process running on the machine.

Docker Commands

docker run hello-world - downloads the “hello-world” image from dockerhub and executes it
docker run busybox ls - downloads the “busybox” image from dockerhub, executes it, and runs the ls command

busybox is handy because it comes with common tools we might use. By default containers do not have any programs installed that they do not need. So ls and echo or other basic commands are not available on the hello-world image.

docker ps list active docker processes
docker ps --all list all containers that we have run on our machine
docker run busybox ping google.com - long running command to test docker ps

One of the most common uses of docker ps is to get a particular container’s ID to run commands on it.

docker run = docker create + docker start in one command
docker start -a <id> start a stopped docker image by ID. -a flag causes start to list output.
docker system prune delete all stopped or unused containers, images, networks, volumes, etc
docker logs <id> print console log of the container. So if you run docker start <id> without a -a, you can still see the output by docker logs <id>
docker stop <id> shut down but wait for the process to finish.
docker kill <id> force stop the container without allowing the process to finish.
docker exec -it <id> <command> run commands on a container. -it allows us to provide input to the container. -i means pass input to STDIN, and -t to show nicely formatted output.
docker exec -it <id> sh enter the shell on a container

If you are ever stuck in a container and CTRL+C does not escape, try CTRL+d.

docker run -it busybox sh - start up a container and open shell

Instead of using a dockerfile you can use docker commit commands to manually generate an image:

docker run -it alpine sh startup an alpine container and enter the shell
apk add --update redis
(Second terminal): docker ps get container ID
(Second terminal): docker commit -c 'CMD ["redis-server"]' <id> generate an image from the active container and set it to start redis-server

The result is an ID pointing to a new image based on the active container.

docker run -p 8080:8080 <id> start a container and bind port 8080 on the local machine to port 8080 on the container.

Dockerfile

Creating a Dockerfile

Writing a dockerfile is like being given a computer with no OS and being told to install Chrome on it. First you have to install the OS, then tools and libraries you’d need, then you can install Chrome.

Specify a base image
Run some commands to install additional programs
Specify a command to run on container startup

Example Dockerfile:

FROM alpine
RUN apk add --update redis
CMD ["redis-server"]

Creating a Docker Image

Dockerfile -> Docker Client -> Docker Server -> Usable Image

docker build . build the current directory / Dockerfile into a docker image

Dockerfile statements:

CMD defines the primary command on the container once it has been built

Docker build steps in order (or Docker Instructions):

FROM

FROM statement defines the base image:

downloads the alpine image

RUN

RUN statements defines commands to run on the image. Each RUN statement performs the following steps:

get the image from the previous step
create a container out of it
run the command (ie apk add --update redis)
Take snapshot of that container’s filesystem
Shut down that temporary container
Get image ready for the next instruction

CMD

CMD defines the primary command on the container once it has been built:

get the image from the previous step
create a container out of it
Tell container it should run redis-server when started
shut down that temporary container
get image ready for next instruction

Instead of spaces use an array:

CMD ["npm", "run", "start"]

No more steps! The output image is generated from the previous step.

Note that Docker will cache all build steps that had not previously changed. As soon as it reaches an instruction that is new or different, the rest of the instructions will be rebuilt not from cache. This means that if you need to make a change to a Docker image, it is usually better to put that change at the end of the Dockerfile to prevent busting the cache for earlier steps.

Tagging a Docker Image:

docker build -t nimoi/redis:latest . - build the docker image from the current directory and tag it. The tag naming convention is <docker_id>/<repo_or_project_name>:<version_number>

That was a basic example, but there are more Dockerfile commands.

COPY

Copy files from the local machine to the container.

COPY ./ ./ will copy the current directory on your local machine into the current directory in the container.

WORKDIR

Set a working directory for the container. Any other commands will be relative to this directory.

WORKDIR /usr/app - will automatically create this folder if it does not exist in the container

EXPOSE

EXPOSE is only informational telling developers a port needs to be mapped to the container. Some services such as EBS read EXPOSE to map ports automatically.

EXPOSE 80

Dockerfile Caching and Unnecessary Rebuilds

If docker detects a change to one of the files in a COPY instruction then all steps from that point on will be rebuilt. This can take a long time if one of the steps is to npm install from a package.json that was copied over in the directory. A way to deal with this can be to split the copy command into steps.

Instead of:

COPY ./ ./
RUN npm install

Try this:

COPY ./package.json ./
RUN npm install
COPY ./ ./

This way npm install will only be run when package.json is actually changed.

Docker Compose

docker-compose is a separate CLI that gets installed along with Docker. It is used to start up multiple Docker containers at the same time and helps automate some of the long-winded arguments that were passed to docker run.

Example docker-compose.yaml file:

version: '3'
services:
    redis-server:
        image: 'redis'
    node-app:
        build: .
        ports:
            - "8081:8081"

The services section defines different containers. The containers have a hostname that matches the name defined for the service. So to connect to the redis server from the node-app you can set the host to redis-server:

const client = redis.createClient({
    host: 'redis-server'
});

`docker-compose` Commands

docker-compose up = docker run myimage
docker-compose up --build = docker build . and then docker run myimage
docker-composer up -d launch the containers in the background
docker-compose down stop the containers
docker-compose ps show status of all running containers that belong to the docker-compose.yaml
docker-compose -f docker-compose-dev.yml up use the docker-compose-dev.yml file instead of the default path.

Crashed containers just stop. They do not appear in the list of running containers with docker ps. Returning a code of 0 on edit signals that we stopped the process but its OK. Any other number on exit signals something went wrong.

Docker has Restart Policies:

no - Never attempt to restart
always - Always attempt to restart
on-failure - Only attempt to restart on failure
unless-stopped - Always restart unless we stop it

version: '3'
services:
    redis-server:
        image: 'redis'
    node-app:
        restart: on-failure
        build: .
        ports:
            - "8081:8081"

Docker Volumes

Docker Volumes allow us to map files on our local machine to the container so we do not need to rebuild in order to see file changes.

docker run -p 3000:3000 -v $(pwd):/app <image_id> creates a mapping from the current directory to the /app directory on the container.
docker run -p 3000:3000 -v /app/node_modules -v $(pwd):/app <image_id> does the same as above and adds a bookmark on the node_modules file so that it does not get mapped.

When using the -v flag the : will determine whether we are mapping or bookmarking a particular path to not be mapped.

This can be set in the docker-compose.yaml:

version: '3'
services:
    web:
        build: .
        ports:
            - "3000:3000"
        volumes:
            - /app/node_modules
            - .:/app

The build: config can be set to use a nonstandard Dockerfile path:

version: '3'
services:
    web:
        build:
            context: .
            dockerfile: Dockerfile.dev

Multi-step Docker Builds

Dockerfiles can have multi-step build processes. Example of phases:

Build Phase:

Use node:alpine
Copy the package.json file
Install dependencies

Run Phase:

Use nginx
Copy over the result of ‘npm run build’ (from last step in Build Phase)
Start nginx

FROM node:16-alpine as builder
WORKDIR '/app'
COPY package.json .
RUN npm install
COPY . .
RUN npm run build

FROM nginx
COPY --from=builder /app/build /usr/share/nginx/html

Each FROM statement is starting a new container. Above is an example of a multi-step dockerfile with one container that builds the application and the next container copies over the files from the build container and serves them with nginx. The default run command of the nginx container is to start, and nginx serves its /usr/share/nginx/html directory by default.

Docker Development Workflow

Docker Compose Running Tests

Adding a second container with a different run command can allow tests to be automatically rerun for every change.

version: '3'
services:
    web:
        build: .
        ports:
            - "3000:3000"
        volumes:
            - /app/node_modules
            - .:/app
    tests:
        build: .
        volumes:
            - /app/node_modules
            - .:/app
        command: ["npm", "run", "test"]

docker attach <id> will always attach to the primary process of the container.

Attaching to a container with a run command of npm run test would not attach to the running test process, but the npm process. So there is a trade-off because running the tests in an interactive terminal requires getting the container and manually attaching.trade-off because running the tests in an interactive terminal requires getting the container and manually attaching.