Image Registries and the Docker Hub

About image registries

Packaging an application into a container image is easy with the docker image build command, but the image only gets created in the local image cache on your own machine. In order to make that image useful, you need to be able to ship it to different hosts, and that’s where image registries come in. A registry is simply a shared location for images—you can push images you’ve created and pull images that you or other people have created. The registry is a core concept in Docker, and you use the normal Docker command-line tool to work with registries.

The Docker Hub is a free, public registry for images maintained by Docker, Inc. It is a hugely popular service—at the time of writing there are more than 900,000 images on the Hub, and there have been more than 12 billion image pulls. Anyone can push images to the Hub, and in addition to the community images, there are also official images curated by Docker and supported by their owners.

It’s best to start with the Docker Hub, which is the default source and has a very useful feature set that makes it much more than just an image store. However, if you’re not comfortable storing your own application images in a third-party cloud service, there are many other options. In this chapter, we’ll look at using the Hub, choosing images, and other registry options.

Using the Docker Hub

The Docker Hub is a public image registry provided by Docker Inc. (we’ll look at the relationship between Docker and Docker, Inc. in Chapter 9, Continuing Your Docker Journey). Docker Hub is the default registry for the Docker Engine, and you can pull images without needing an account. So far, when we’ve run containers, the images have been downloaded from the Docker Hub without us having to configure anything.

You will need your own account on the Hub if you want to push images and share them. You can register with the Hub at https://hub.docker.com and create a free account. The username you choose will be the user part of the repository name when you build your own images (my username on the Hub is sixeyed, and I can only push images which are prefixed sixeyed). The Hub can be used for storing private repositories, too, and the free plan lets you store one private repository and unlimited public ones.

Tip: Don't be confused by the terminology here. A “registry” is a service for hosting lots of image repositories, potentially from different users. A “repository” is an image library that potentially holds many different tagged images. For instance, the Docker Hub is a public registry, and one of its users is Microsoft. Microsoft has a repository for .NET Core, microsoft/dotnet, offering many image versions. You can pull a specific .NET Core image from the microsoft/dotnet repository on the Docker Hub registry.

When you have an account registered with the Docker Hub, you will need to add your credentials to the Docker CLI using docker login, as shown in Code Listing 24.

Code Listing 24: Logging in to Docker Hub

You can now push images as well as pull them. My Docker Hub account also has access to the dockersuccinctly account, so I can push images there too. The image push command is very simple, as we see in Code Listing 25.

Code Listing 25: Pushing a Local Image to Docker Hub

The push works intelligently, pushing only layers that aren’t already available in the registry. In this case, the base image layers are from the official Ubuntu repository (technically called library/ubuntu), which means they’re not pushed, and the command flags them as “mounted” instead. Only the new layers created by building my Dockerfile get pushed. If I modify the Dockerfile and rebuild, and if only the top layer changes, when I push a new version only the top layer will be uploaded.

That repository is now created in the Hub, and anyone can run containers from my image with a docker container run command specifying the image tag. As we have seen, when we run a container, Docker checks the image cache to see if it has a copy of the image locally. If not, it will pull the image from the Docker Hub.

Image size can be a few megabytes or many gigabytes, so pulling large images might take a while, which means startup time for a new container can jump from seconds to minutes. In order to prevent that, you can explicitly pull images from the Hub and download them before they are needed, so that when you do run a container it uses the local image, as in Code Listing 26.

Code Listing 26: Pulling an Image

Tags and image versions

So far we’ve mostly referred to images by their repository name without specifying a particular tag. Docker image repositories use the basic format {user}/{app}, but often the tag is used to specify a release version, or a variant of the image, in the format {user}/{app}:{tag}. If you don’t specify a tag when you refer to an image, the default latest tag is used. You can see this when you list out your image cache with the docker image ls. Code Listing 27 shows the various Ubuntu images in my local cache, using the filter flag.

Code Listing 27: Listing Images for a Repository

Note: One image can have multiple tags. At the time of writing, if you pull ubuntu, ubuntu:xenial, or ubuntu:16.04, you’ll get the same image, one with the ID that starts f8d. When Canonical releases a patch to Ubuntu, it uploads a new version with multiple tags—by codename and version. This is useful because you can base your images on ubuntu and you’ll always use the latest patched version when you build. If you’re more specific, your build will use the same image. If you have an old image based on ubuntu:14.04 and rebuild it now, it will still be based on 14.04.

Tags can be version numbers, commit labels, codenames—any useful way to classify images. An established, active image repository will have multiple images that make good use of the tag in order to tell you how the images are different. Table 1 shows some of the images available in the official Elasticsearch repository, with image sizes and age at the time of writing.

Table 1: Image Tags for Elasticsearch

From that output we can clearly see that there is an old version 2.2, there’s a prerelease version 5.0.0-alpha5, and there’s a latest tag for the current release. They’re all the same age, which suggests Elastic has an automated process that builds and pushes all versions at the same time. On the main repository page on the Hub, we can also see that many of the tags point to the same image, as in Figure 3.

Figure 3: Elasticsearch Images with Multiple Tags

We can use docker container run elasticsearch to start a container using the latest image, which is currently version 2.3.5; if we want to try the alpha 5 release of version 5, we specify the tag docker container run elasticsearch:5.0.0-alpha5; and if we want to stick with an old version, we can use docker container run elasticsearch:1.7. The same is true if we’re using that image as a base for our own containers. The FROM instruction in the Dockerfile supports image tags, too.

Tip: It’s a good idea to specify a tag for base images in your FROM instruction. If you don’t, the image tagged as latest will be used. That tag can change to a different image without warning, and there can be breaking changes between image versions, which means your own images might fail to build. If you have an old image built on ubuntu:14.04 but the Dockerfile specifies FROM ubuntu, the base image will be ubuntu:16.04 the next time you build it, which is functionally different.

Automating builds

Manually building and pushing images is fine for apps that don’t change frequently, but if you’re doing regular releases you will want to automate building and pushing your images. Docker Hub supports automated builds for images when the Dockerfile and source is in GitHub or Bitbucket, which means you can set up the Hub to build and tag images whenever you commit changes.

The automated build setup is only available when you’re logged in to the Docker Hub. From the top menu, select Create > Create Automated Build. From here, you can link your GitHub or Bitbucket account and select the source git repository—you’ll see the screen in Figure 4.

Figure 4: Creating an Automated Build in Docker Hub

After the build is set up, any pushes to the git repo will trigger a build on Docker Hub. You can also manually trigger a build, and Docker Hub will pull in the source files, run docker image build on its own servers, and upload the image to your Docker Hub repository.

Automated builds can be set up to create different image tags from different branches or tags from the source git repository. Figure 5 shows the automated build configuration for one of my public images.

Figure 5: An Automated Build with Multiple Image Tags

There are two builds set up, one that uses the GitHub master branch as the source for the latest image tag and one that uses the git tag 2.7.2 for an image with the same tag name. I use that image for Hadoop training, specifying the tagged version 2.7.2 in the material. If I update the image with a new version of Hadoop, I’ll use a new tag. The old image won’t change and people using old content can still work along with it. But if I find a problem with the 2.7.2 image, I can update it with the same tag so that it remains correct for the training.

Choose your base image carefully

Docker Hub is the default registry, and there are thousands of high-quality images available there, but only the official repositories are curated. There are no quality gates on any other images in the Hub, which means anyone can build anything into an image. You need to be careful what you choose to run or to use as a base image.

The official repositories are the best place to start. Not only are they actively supported, but the Docker Hub scans the images against a security vulnerability database and will flag any issues it finds in the image.

You can see the results of the security scan if you’re logged in to the Docker Hub. Figure 6 shows the current status of the official Ubuntu image and some other interesting details.

Figure 6: Security Scan Results for Ubuntu Images

We can see that the image for 16.04 is 16 MB smaller than version 14.04, and it currently has two known security vulnerabilities that you can drill into to find. The image for 14.04 currently has 11 vulnerabilities. If you have images built from ubuntu:14.04, this information tells you it’s worth upgrading and rebuilding your images in order to get a smaller image and close off some attack vectors. The image for 16.10 is smaller still, which means Canonical is actively minimizing the image size with each new release.

Only a small number of official repositories exist, however. In the rest of the Hub, you may find several images that fit the requirements for a base image for your app, but you must be sure they are trustworthy.

If the image is set up as an automated build, you can check the Dockerfile in GitHub and be confident that the image came from that Dockerfile because it was built and pushed by Docker’s servers. For images that aren’t automated, you can’t guarantee that.

Many images have a link to the Dockerfile, but if the image was manually built and pushed by a user, the contents might not match what’s in the Dockerfile. Some images don’t have a Dockerfile link at all. That doesn’t mean they’re poor quality or untrustworthy—the star rating and number of pulls might suggest they’re fine—but pulling the image first is a good precaution. After doing that, you can run docker image history and look through the instructions to get an idea what went into building the image.

Note: If you find the perfect base image and it’s not from an official repository, the publisher is not required to keep that image up-to-date or even available. If you build from another user’s base image on the Hub, that repository can be deleted, leaving you unable to build your own image. If that’s a concern, you should consider cloning the source and building your own version of the base image so that you can control the image stack up to a reliable, official source.

Running your own registry

Nearly all of the images on the Docker Hub are packaged versions of free, open-source software. For commercial or sensitive applications, you might not want your images built and hosted on a public service, even in a private repository. In that case, you can run your own registry on your own hardware with Docker’s registry image.

Yes, it’s a Docker image available on the public Docker Hub that packages an application for running a private image registry you can use on-premise. Code Listing 28 shows how to run a simple local registry.

Code Listing 28: Running Your Own Image Registry

Docker can use different registries from the default Docker Hub, and you need only to specify the registry location as a prefix to the image repository name. The Docker Registry runs on port 5000, which means that when you have a container running locally with port 5000 mapped, you can reference it at localhost:5000. The registry address is actually part of the full image tag, so in order to push an existing image from your cache to your registry, you will need to tag it with the address, as in Code Listing 29.

Code Listing 29: Tagging an Image for a Different Registry

You can now push the image to your registry simply by specifying the full image name, as in Code Listing 30.

Code Listing 30: Pushing to a Local Registry

This gives you a lot of flexibility. You can pull the Ubuntu base image, tag it to give it a completely different name, and push it to your local registry. If you mandate that image as the base for all your in-house container images, they will all use a version of Ubuntu that is in your control. If, in the future, you decide to move to Debian, you can tag the official Debian image with your custom name, replace the version in your registry, and rebuild your containers. They’ll all be rebased to use Debian.

With Docker’s Registry image, you get the basic push and pull functionality but not the command-line search option, and there is no UI, so it’s not feature-equivalent to the Docker Hub. The REST API for the Registry gives you a lot of functionality, but without a friendly interface. You can list all the repositories in your local registry with an HTTP GET to the _catalog endpoint, as in Code Listing 31.

Code Listing 31: Querying the Local Registry with cURL

The Docker Registry is quite a technical option. You can set it up as production-grade, load-balanced, secured cluster, but configuring it correctly and maintaining it well requires a good bit of effort.

Other registry options

Because the Registry API is public, compatible registries are available in other products. You can use the integrated container registry in GitLab as part of a build pipeline so that when you push code, GitLab builds a container and pushes it to the registry. JFrog has a Docker registry add-on for Artifactory. And Docker, Inc. has the Trusted Registry product, which we’ll look at in Chapter 9, Continuing Your Docker Journey.

Summary

We've now seen how to push and pull Docker images using different registries. The most popular registry is the Docker Hub, which has hundreds of thousands of images for all kinds of applications. The Hub has a free service level, and you can push your own images or make use of the automated build system in which the Hub will build images from a GitHub or Bitbucket repository whenever you commit a change.

If you’re packaging applications for internal use, you can host your own registry rather than using the public Docker Hub. Doing so can be as simple as running the registry:2 image on a designated server in your network or running a scalable set of instances on multiple servers or using a commercial registry product.

For cases in which you’re running your image registry inside a container, you’ll obviously want all your data permanently persisted. However, when it comes to saving data, Docker containers may not behave as you expect. In the next chapter, we’ll get a better understanding of Docker volumes.