Docker can now run within Docker

One of the (many!) features of Docker 0.6 is the new “privileged” mode for containers. It allows you

One of the (many!) features of Docker 0.6 is the new “privileged” mode for containers. It allows you to run some containers with (almost) all the capabilities of their host machine, regarding kernel features and device access.

Among the (many!) possibilities of the “privileged” mode, you can now run Docker within Docker itself. First, we will see how to make that happen; next, we will explain what is involved under the hood, and finally, we will show something even more powerful than Docker in Docker!

If you have Docker 0.6, all you have to do is:

This will download my special Docker image (we will see later why it is special), and execute it in the new privileged mode. By default, it will run a local

daemon, and drop you into a shell. In that shell, let’s try a classical “Docker 101” command:

Note how the container ID changes as you transition from the container running Docker, to the innermost container!

Almost nothing! It is built with a regular

. Let’s see what is in that Dockerfile.

First, it installs a few packages:

and

(because Docker needs them), and

(because when communicating with the Docker index and registry, Docker needs to validate their SSL certificates).

The Dockerfile also indicates that

should be a volume. This is important, because the filesystem of a container is an AUFS mountpoint, composed of multiple

; and those branches have to be “normal” filesystems (i.e. not AUFS mountpoints). In other words,

, the place where Docker stores its containers, cannot be an AUFS filesystem. Therefore, we instruct Docker that this path should be a

. Volumes have many purposes, but in this scenario, we use them as a pass-through to the “normal” filesystem of the host machine. The 

directory of the nested Docker will live somewhere in

on the host system.

And of course, the Dockerfile injects the Docker binary in the image, as well as a

. The helper script deals with three things.

In the next section, I’ll tell you why I think that this

environment variable can be

useful.

If you just want to experiment with Docker-in-Docker, just start the image interactively, as shown above. Now, let’s pretend that you want to provide Docker-as-a-Service. I’m not speaking about Containers-as-a-Service here, but whole Docker instances. Well, each time someone wants their own private Docker instance, just run this:

Then use

to retrieve the public port allocated to that container, and give it to your user. They will be able to create containers on this “private Docker” by pointing their Docker client to the IP address and port that you gave them. (See

for a similar example.)

Note, however, that there are serious security implications there: since the private Docker instances run in privileged mode, they can easily escalate to the host, and you probably don’t want this! If you really want to run something like this and expose it to the public, you will have to fine-tune the LXC template file, to restrict the capabilities and devices available to the Docker instances. In the future, Docker will allow fine-grained permission management; but for now, we think that the ability to switch between “locked down” and “privileged” is a great first step.

Can I Run Docker-in-Docker-in-Docker? Yes. When you are inside a privileged container, you can always nest one more level:

And in the resulting container, you can repeat the process,

.

Also, as you exit nested Docker containers, this will happen (note the root prompts):

At that point, you should blast Hans Zimmer’s

on your loudspeakers while twirling a spinning top 😀

While testing Docker-in-Docker in various environments, I found two possible problems.

It looks like the LXC tools cannot start nested containers if the devices control group is not in its own hierarchy. Check the content of

if is standing on a line on its own, you’re good. If you see that another control group is on the same line, Docker-in-Docker won’t work. The wrapper script will detect this situation and issue a warning. To work around the issue, you should stop all running containers, unmount all the control groups, and remount them one by one, each in its own hierarchy. Also, if you use AppArmor, you need a special policy to support nested containers. If Docker-in-Docker doesn’t work, check your kernel log (with ); if you see messages related to AppArmor, you can start Docker in unconfined mode, like this: The Dockerfile, the wrapper, and some extra documentation is available on my github repository: .   Jérôme is a senior engineer at dotCloud, where he rotates between Ops, Support and Evangelist duties and has earned the nickname of “master Yoda”. In a previous life he built and operated large scale Xen hosting back when EC2 was , supervized the deployment of fiber interconnects through the French subway, built a specialized GIS to visualize fiber infrastructure, specialized in commando deployments of large-scale computer systems in bandwidth-constrained environments such as conference centers, and various other feats of technical wizardry. He cares for the servers powering dotCloud, helps our users feel at home on the platform, and documents the many ways to use dotCloud in articles, tutorials and sample applications. He’s also an avid dotCloud power user who has deployed just about anything on dotCloud – look for one of his many custom services on our Github repository.