Packaging Applications with Docker

Meet the Dockerfile

In order to package up your own Docker image, you write a text document called a Dockerfile that lists all the steps needed to both make the image and use the Docker command line to build it. The Dockerfile uses a very simple domain-specific language that only requires a handful of instructions. Code Listing 10 shows a perfectly valid Dockerfile.

Code Listing 10: A Simple Dockerfile

When you build an image from that Dockerfile and run a container from the image, you’ll be working in an Ubuntu container with the nano package installed. The FROM instruction specifies the base image so that your image will start from there and layer on the changes in the rest of your Dockerfile. In this case, it will run two apt commands to install Nano.

In order to build an image, you use the docker image build command. You need to specify both a repository name to identify the image, and the path Docker should use as the context for building the image. You can also tag images with labels, which explains how you can have multiple image versions in a repository (like ubuntu:12.04 and ubuntu:14.04). Code Listing 11 builds an image using a file called Dockerfile in the local directory.

Code Listing 11: Building the Docker Image

Tip: Dockerfile (with no extension) is the default filename Docker looks for, but you can call your Dockerfile anything and identify it with the --file option. This means you can call your file server.dockerfile and build it with docker image build --file server.dockerfile.

The image build command gets executed by the Docker server—the client simply sends the details through. That’s why you must specify a path (using . for the current working directory in this case). The client sends the contents of the path to the server, and the server stores it in a working folder it uses to build the image. That folder is called the build context, and later in this chapter we’ll see why it’s important.

You can also give your image a repository name. When you build locally you can call your image anything you like, but the convention is to use a format such as {user}/{application}, where the user part is your account ID on Docker Hub.

The tag is the unique identifier for a particular image within a repository, which means that in the public registry on the Docker Hub you’ll see images with repository names like microsoft/azure-cli and sixeyed/hadoop-dotnet that each have many image versions. If you don’t specify a version in a tag, Docker uses the default latest.

Note: Some images on the Hub don’t have a user in the repository name, which means the Ubuntu image is simply called ubuntu rather than canonical/ubuntu. These are from official repositories that are curated, verified, and security scanned. You should use official images for your base image as a preference.

When you successfully build an image, it’s stored in the Docker server’s local cache and you can run containers from it. You can also push it to a shared image registry such as Docker Hub or your own registry (which we’ll cover in Chapter 3, Image Registries and the Docker Hub).

The Docker CLI can list all the images stored in the Docker image cache with the image ls command, as shown in Code Listing 12.

Code Listing 12: Listing Images in the Engine Cache

The output from the docker image ls command tells you the repository name and the tag for each image, its unique image ID, when it was created in the cache, and the size. In this output, I have three images downloaded from the Hub for the containers I ran in Chapter 1, and I have my own newly built image with the dockersuccinctly account name. When you start using larger images, your local cache can use a lot of disk space—we’ll see how to manage that in Chapter 4, Data Storage in Docker.

The main Dockerfile instructions

The only required Dockerfile instruction is FROM, which specifies the base image on top of which a new image will be built. Of course, that doesn’t do much on its own, but to build useful production-grade images you need only a few more instructions:

• RUN: Execute a command.
• ENV: Set environment variables.
• COPY: Copy files from the build context into the image.
• EXPOSE: Expose ports from the container that can be mapped to ports on the host.
• VOLUME: Create a directory inside the image that can be mapped to external storage.
• CMD: Specify the command to run when the container starts.

Here’s a very simple Dockerfile that shows all the main instructions—this image is for a basic app that listens for input on a specific port and echoes out any input it receives to a file. Code Listing 13 shows the Dockerfile in full.

Code Listing 13: Dockerfile for an Echo Application

Note: The full code is on GitHub at SyncfusionSuccinctlyE-Books/Docker-Succinctly, and a built image is available on the Docker Hub in the repository dockersuccinctly/echoserver. The order of instructions is important, as we’ll see soon, but in this example the instructions are ordered to make them easy to follow.

As Docker builds that image, it will process each of the following instructions:

• Download the latest ubuntu image if it’s not already in the local cache.
• Update the package list and install the netcat package.
• Set an environment variable called LOG_FILE with the value echo.out.
• Copy the script called echoserver.sh from the build context into the root folder in the container.
• Make the script executable on the container.
• Expose port 8082 from the container.
• Create a filesystem mount called server-logs.

Note: It’s important to understand that those commands are happening inside containers during the build process, not on your local machine. After you’ve built this image on your machine, the image will have netcat installed and have a value set for the LOG_FILE environment variable, but your local machine won’t.

Everything except the final CMD instruction gets executed during the build. When you run a container from the image, Docker uses the CMD instruction to tell it how to start—in this case by running the echoserver.sh script. That script starts netcat listening on port 8082 and redirects the output from client connections to a file. The file path uses the volume named in the Dockerfile and the log file name from the environment variable. Code Listing 14 shows how to start the echo server container and find the virtual IP address of the container.

Code Listing 14: Running the Echo Server Container

The container is now running netcat listening on port 8082, and Docker is forwarding requests to port 8082 on the local host into the container.

In Code Listing 15, we connect to the container using netcat on the host, specifying the localhost address and the published port 8082. Then we write a string to netcat and exit the connection.

Code Listing 15: Connecting to the Echo Server from the Host

Note: If you’re running on Windows, the netcat utility won’t be available, but you can install a version from https://eternallybored.org/misc/netcat/.

And lastly, in Code Listing 16, we use docker container exec to run a command inside the container and write the output back to the host. In this case, we read the contents of the output file that netcat on the container is using—this is an echo of the string we sent from the client.

Code Listing 16: Viewing the Echo Server’s Log File

By using a volume for the output location and an environment variable for the file name, we can change where the echo data is written for different instances of the container when we run them. We’ll look more closely at Docker volumes in Chapter 4, Data Storage in Docker.

This is a simple example of a standard approach to Dockerfiles. A typical Dockerfile for packaging an application will state a minimal base image, install the application platform, copy in the application source, compile the application, and specify how the application starts.

You can vary that approach. For example, if your platform has an official image, you can use that as the base, and if you can publish your app with all its dependencies, you can compile first, then copy the binaries into the container. There’s a balance between the portability of the Dockerfile, dependencies on third-party resources, and the size of the built image.

How Docker builds an image

Docker uses a layered filesystem for images. Starting from the base image, the Docker server runs a temporary container from the image for each instruction in the Dockerfile, executes the instruction, then saves the temporary container as a new image, adding it to the local image cache. Docker uses the cache during the build process, which means that if it finds an image matching the current instruction stack—that is, one that matches the state you’re asking Docker to create—it will reuse the cached image.

You can write your Dockerfile to make maximum use of the cache by ensuring that the Dockerfile is correctly structured and that the Dockerfiles for different applications each have similar structures. This way, they will use cached images as much as possible. Ideally, when you build apps with similar dependencies, Docker will need only to execute instructions in new layers that are specific to the application. For instance, the Dockerfiles in Code Listing 17 and Code Listing 18 are identical up to the final COPY instruction.

Code Listing 17: Dockerfile ‘A’

Code Listing 18: Dockerfile ‘B’

When you build an image from the first Dockerfile, it will create new layers for both of the RUN instructions and the final COPY instruction. Code Listing 19 builds an image from the first Dockerfile by using the -f flag to specify the source Dockerfile name.

Code Listing 19: Building Dockerfile ‘A’

Let’s note a few things here. The first instruction in Step 1 finds a match in the cache because we’ve already downloaded the ubuntu image, which means Docker simply writes the ID of the cached image it’s going to use (starting f8d).

For Step 2 there is no match, so Docker runs a temporary container from the f8d image, executes the command, and saves the temporary container to a new image with the ID starting e4d. Similarly, for Steps 3 and 4, there is no match in the cache, which means Docker runs a temporary, intermediate container from the image in the previous step, saves the container as a new image, and removes the intermediate container.

The docker image history command displays all the layers in an image, as in Code Listing 20, which shows the layer history for the dockersuccinctly/a image.

Code Listing 20: History of the ‘A’ Image

There is a lot of detail in there. The “missing” layers mean we don’t have the intermediate layers in our cache, because they’re part of the base Ubuntu image. Docker downloaded that image from the Hub—we didn't build it locally, which means we don’t have all the layers. But we can see part of the instructions that went into building the final Ubuntu image, which is the one with ID f8d that our image started with. There are three layers above that, one for each of the instructions in the Dockerfile.

If you now build an image from the second Dockerfile, it will find matching images in the cache for the first two instructions. Docker tells you it’s found a cache hit in the build output, as we see in Code Listing 21.

Code Listing 21: Building Dockerfile ‘B’

The Docker server has run only an intermediate container to execute the COPY instruction in Step 4 because there was no match in the cache for that. Everything up to that point can come from the cache because the Dockerfile instructions are identical to the cached layers. If we look at the layers in the dockersuccinctly/b image, we’ll see the bottom seven layers match the dockersuccinctly/a image and that only the final layer is different, as in Code Listing 22.

Code Listing 22: History of the ‘B’ Image

The image cache saves a huge amount of time when you’re building many images with similar instructions. But you need to be careful with the image cache. Each instruction in a Dockerfile is a candidate for adding to the image cache, which effectively preserves the state at the time the image was built.

If Docker finds a match in the cache, it will use that match, which means you need to be sure you only cache layers that are good candidates for reuse and that don’t have contents that will become stale. For example, in Code Listing 23 we have the apt-get update command in its own RUN instruction.

Code Listing 23: A Dockerfile Which Will Become Stale

When this image gets built, it will cache a layer with an updated package list. If you build an image from a different Dockerfile that starts with the same instructions, it will use the cached image with the saved package list. If we build the new image several months after building the original image, the new image will pick up the old cached layer—it will not run apt-get update for the new image. Any subsequent instructions that install packages will be using an old package list.

Tip: How you structure your Dockerfile impacts the speed of the build process, but the image cache can also have a functional impact on the contents of the image. Docker’s resource Best practices for writing Dockerfiles is worth getting to know. One of its key recommendations is that you combine multiple commands in a single RUN statement in order to prevent unintended cache hits.

Summary

Packaging applications into a Docker image is simple—you specify the base image, install dependencies, copy in your application files, and tell Docker the command to run when a container starts from the image. You will need to consider the workflow for your Dockerfile so that it builds efficiently, and you’ll also need to be aware of the Docker image cache and how it can impact your builds.

The majority of the work that goes into the Dockerfile is about optimizing your image. Typically, for a production image you want the smallest possible image in order to keep your app secure and make it easy to move around. The official base images on the Docker Hub are a good place to start, and in the next chapter we’ll have a closer look at the public Hub and other image registries.