ccie.tv

Fun with Docker - Part 4: Docker Volumes (and Bind Mounts)

Roddie Hasan -
In Parts 1 & 2, we covered an overview and installation of Docker, and reviewed some basic commands to get you started.  In Part 3, we introduced Docker Compose to help make our Docker experience a bit less annoying.  In the next two posts, we're going to go deeper into two concepts that are pretty much essential with most Docker containers: Volumes & Networking.
Important Note: I've only been messing with Docker for about two months, and even then only in my "spare" time (all of the posts on this blog are being written while I'm on vacation in Greece).  A lot of these concepts are still pretty new to me, and I'm learning on the fly.  Please feel free to correct me in the comments section on anything I've misstated or confused.

The Fun with Docker Series

Links to the entire series are here:

One of the things I've learned about Volumes while researching for this post, is that I've actually been using the term incorrectly this whole time.  To be more accurate, this post will cover Volumes and Bind Mounts.

Do they need to be capitalized, though?  That's the important question. Probably not, but I will continue to do so just to distinguish from the rest of the word salad that I throw at you.

Ok, let's dig in.

Why Volumes/Bind Mounts?

One key characteristic of Docker containers is that, beyond a stop and start, they are not persistent, and neither is the data inside of them.  This feature is by design; to make them flexible, portable, and perhaps most importantly: lightweight.  This is one major difference between VMs and containers.

Storytime: I learned this the hard way, about a month or so before I started playing with Docker. I was setting up Pi-hole on a Raspberry Pi using balenaCloud, which is based on Docker, as I later found out. After I had Pi-hole up and running, I copied over a /etc/hosts file from my network so that I could see hostnames in the Pi-hole dashboard instead of IP addresses. What I noticed was that anytime I upgraded or rebooted my Pi-hole device, the /etc/hosts file reverted to an empty default. This was because the container was being recreated every time. And that, my friends, is how I learned about Docker Volumes before I learned about Docker. This is actually one of the things that inspired me to start learning more.

This is fine if you only want a container to run a few commands and then exit, but what if you're running something like a database or web server in a container that will create or serve data?  Or, what if your container application needs to keep track of configuration settings or state?  What if you need to share data with other containers or the host?  

This is where Volumes and Bind Mounts come in.  As with everything, there is already a ton of deep technical information about these features that you can find with Google, but I'll provide high-level descriptions here and share some examples of their use.  You can find the official documentation for these features here: Volumes - Bind Mounts.

To me, Bind Mounts are the most straightforward way to persist and share data between the container and the host (and also between containers).  That said, everything I've read on the subject says that Volumes are the recommended method to use.  You can decide for yourself once I explain both methods.

Using Bind Mounts

The concept of Bind Mounts is quite simple; at least to me.  It involves mapping an explicit folder or file on the host to one on the container.  That's it.

In fact, Docker uses Bind Mounts automatically for a few files when you start a container. If I issue the mount command inside of a container, I can see entries like this:

/dev/sda2 on /etc/resolv.conf type ext4 (rw,relatime,data=ordered)
/dev/sda2 on /etc/hostname type ext4 (rw,relatime,data=ordered)
/dev/sda2 on /etc/hosts type ext4 (rw,relatime,data=ordered)

You can see from the above that Docker is mapping some of the system files from the host to the container to allow for DNS resolution. If I make a change to /etc/hosts on the host, it will be automatically reflected on /etc/hosts in the container.

Remember my Apache rant from Part 1? In my case, I was moving from a years-old instance of Apache running on my Ubuntu server to Nginx running in a container. But, how do I make the existing websites that live in /var/www/html available to the Nginx container so that the sites will continue to work? More importantly, how do I continue updating websites on the host and still have them be current in the Nginx container? I simply map /var/www/html on the host to /data/www on the container (and inside of the Nginx configuration file, I set /data/www as the root of my website.)

This can be done with the docker run command using the -v option, like this:

docker run --name nginx -v /var/www/html:/data/www -p 80:80 -p 443:443 -d nginx

But why would you use the docker run command when I've shown you Docker Compose? You wouldn't, so here's how we do it in the docker-compose.yml file:

version: '3'
services:
  nginx: 
    image: nginx
    container_name: nginx
    volumes:
      - /var/www/html:/data/www
    ports:
      - 80:80
      - 443:443

Note: You can see that even though we're using Bind Mounts, we still use the -v option or volumes: field to declare them.

We will cover the -p and ports: options in our next post on Networking.

We can also use Bind Mounts for files. Coming back to our Nginx example again, we will map /opt/nginx/nginx.conf file that contains our Nginx configuration to /etc/nginx/nginx.conf:

docker run --name nginx -v /var/www/html:/data/www -v /opt/nginx/nginx.conf:/etc/nginx/nginx.conf -p 80:80 -p 443:443 -d nginx

Or with docker-compose.yml:

version: '3'
services:
  nginx: 
    image: nginx
    container_name: nginx
    volumes:
      - /var/www/html:/data/www
      - /opt/nginx/nginx.conf:/etc/nginx/nginx.conf
    ports:
      - 80:80
      - 443:443

Bind Mounts are also useful for sharing data between containers, as you can map the host folder or file to multiple containers. For example, if you created an FTPd container to allow remote access for users to update their website files, you could map the /var/www/html directory to that container as well. Any updates made by users to files in this directory will also be visable in the Nginx /data/www directory.

The best part? Bind Mounts are persistent. You can start, stop, and destroy containers, and the files on the host will remain in-tact. This allows you to upgrade and change containers, and only need to change the mappings to have them appear in the new containers.

Note: One thing I've been trying to do with Bind Mounts on my system is to put everything that I want to share from my host inside of /opt/docker/<container_name>. This will make things easier to migrate in the future, and the only thing I really have to adjust is the mapping in my docker-compose.yml file to give the container what it expects.

Using Volumes

Volumes provide similar benefits as Bind Mounts do, however there are a few differences:

  • Volumes are managed by Docker and can be manipulated by the Docker CLI
  • files and directories in Volumes, generally speaking, are not intended to be manipulated directly on hosts, though they can be backed-up and migrated
  • Volumes are more portable from a Docker perspective, and Docker provides Volume drivers to allow Volume storage in remote locations such as cloud providers
  • Volumes do not rely on an explicit path, location, or even filesystem type on the host; making them more flexible
  • a container can more easily populate a new Volume with its contents
Note: The last point might be important to you, depending on what kind of containers you use.  I've had inconsistent experiences using Bind Mounts with some containers that need to pre-populate files and I've learned that it depends on how the container is built.

A good write-up on the differences and use-cases for each method can be found here.

Using Volumes is as simple as using Bind Mounts. There is only one extra thing that you have to do in your docker-compose.yml file, which is to declare the volume with a name. Here's an example with Nginx configured similar to above, except that we're going to to declare a Volume, named web_sites: for Nginx to store our websites in, and then map that Volume to /usr/share/nginx/html:

version: '3'
services:
  nginx: 
    image: nginx
    container_name: nginx
    volumes:
      - web_sites:/usr/share/nginx/html
      - /opt/nginx/nginx.conf:/etc/nginx/nginx.conf
    ports:
      - 80:80
      - 443:443
volumes:
  web_sites:

Now let's bring up the container so we can see what happens:

pi@raspberrypi:~/docker $ docker-compose up -d nginx 
Creating volume "docker_web_sites" with default driver
Recreating nginx ... done

We can see that Docker creates the new volume, but where is it? On Linux, Docker stores Volumes in /var/lib/docker/volumes/<volume_name>.

Note: You might notice that the volume name has been given a prefix of docker_. This is derived from the directory we are in and can be overridden if desired.

Let's have a look at the contents, first inside of the container, using the docker-compose run command, which allows us to execute a single command inside a container:

pi@raspberrypi:~/docker $ docker-compose run nginx ls -al /usr/share/nginx/html
total 16
drwxr-xr-x 2 root root 4096 Aug 14 20:14 .
drwxr-xr-x 3 root root 4096 Jul 23 19:31 ..
-rw-r--r-- 1 root root  494 Jul 23 11:45 50x.html
-rw-r--r-- 1 root root  612 Jul 23 11:45 index.html

Nginx has populated the directory with a some default files. Looking at the Volume directory on the host, we see the same contents:

pi@raspberrypi:~/docker $ sudo ls -al /var/lib/docker/volumes/docker_web_sites/
total 12
drwxr-xr-x 3 root root 4096 Aug 14 15:14 .
drwx------ 3 root root 4096 Aug 14 15:14 ..
drwxr-xr-x 2 root root 4096 Aug 14 15:14 _data
pi@raspberrypi:~/docker $ sudo ls -al /var/lib/docker/volumes/docker_web_sites/_data
total 16
drwxr-xr-x 2 root root 4096 Aug 14 15:14 .
drwxr-xr-x 3 root root 4096 Aug 14 15:14 ..
-rw-r--r-- 1 root root  494 Jul 23 06:45 50x.html
-rw-r--r-- 1 root root  612 Jul 23 06:45 index.html

As with Bind Mounts, we can share Volumes between containers. Inside of the same docker-compose.yml, we would just map the same Volume that we already declared, and it will have access to it.

Which one should you use?

As with everything IT related, the answer is: it depends.  

Like I mentioned, I find Bind Mounts a bit easier to work with, but there are pros and cons to each.  As I continue to learn more about Docker, I'll probably tend towards Volumes more and more, but for now I will keep using a mix of both depending on my needs.

Next-up - Networking in Docker (and Docker Compose).  Oh yay, Networking!!

Continue to Part 5.

Do you have a preference between Volumes and Bind Mounts or see any benefits that I've missed?  Feel free to comment below or Tweet at me @eiddor and give me your feedback!