Fun with Docker - Part 4: Docker Volumes (and Bind Mounts)
An explanation of Docker Volumes and Bind Mounts, and how we can achieve data persistence with Docker containers.
In Parts 1 & 2, we covered an overview and installation of Docker, and reviewed some basic commands to get you started. In Part 3, we introduced Docker Compose to help make our Docker experience a bit less annoying. In the next two posts, we're going to go deeper into two concepts that are pretty much essential with most Docker containers: Volumes & Networking.
Important Note: I've only been messing with Docker for about two months, and even then only in my "spare" time (all of the posts on this blog are being written while I'm on vacation in Greece). A lot of these concepts are still pretty new to me, and I'm learning on the fly. Please feel free to correct me in the comments section on anything I've misstated or confused.
The Fun with Docker Series
Links to the entire series are here:
- Part 1 - Docker - An introduction
- Part 2a - Getting started with Docker
- Part 2b - More getting started with Docker
- Part 3 - Docker Compose
- Part 4 - Docker Volumes (and Bind Mounts) (this post)
- Part 5 - Docker Networking
- Part 6 - Monitoring your Docker setup with Portainer
- Part 7 - Setting up this blog with Let's Encrypt, Nginx, and Ghost
One of the things I've learned about Volumes while researching for this post, is that I've actually been using the term incorrectly this whole time. To be more accurate, this post will cover Volumes and Bind Mounts.
Do they need to be capitalized, though? That's the important question. Probably not, but I will continue to do so just to distinguish from the rest of the word salad that I throw at you.
Ok, let's dig in.
Why Volumes/Bind Mounts?
One key characteristic of Docker containers is that, beyond a stop and start, they are not persistent, and neither is the data inside of them. This feature is by design; to make them flexible, portable, and perhaps most importantly: lightweight. This is one major difference between VMs and containers.
Storytime: I learned this the hard way, about a month or so before I started playing with Docker. I was setting up Pi-hole on a Raspberry Pi using balenaCloud, which is based on Docker, as I later found out. After I had Pi-hole up and running, I copied over a
/etc/hostsfile from my network so that I could see hostnames in the Pi-hole dashboard instead of IP addresses. What I noticed was that anytime I upgraded or rebooted my Pi-hole device, the
/etc/hostsfile reverted to an empty default. This was because the container was being recreated every time. And that, my friends, is how I learned about Docker Volumes before I learned about Docker. This is actually one of the things that inspired me to start learning more.
This is fine if you only want a container to run a few commands and then exit, but what if you're running something like a database or web server in a container that will create or serve data? Or, what if your container application needs to keep track of configuration settings or state? What if you need to share data with other containers or the host?
This is where Volumes and Bind Mounts come in. As with everything, there is already a ton of deep technical information about these features that you can find with Google, but I'll provide high-level descriptions here and share some examples of their use. You can find the official documentation for these features here: Volumes - Bind Mounts.
To me, Bind Mounts are the most straightforward way to persist and share data between the container and the host (and also between containers). That said, everything I've read on the subject says that Volumes are the recommended method to use. You can decide for yourself once I explain both methods.
Using Bind Mounts
The concept of Bind Mounts is quite simple; at least to me. It involves mapping an explicit folder or file on the host to one on the container. That's it.
In fact, Docker uses Bind Mounts automatically for a few files when you start a container. If I issue the
mount command inside of a container, I can see entries like this:
/dev/sda2 on /etc/resolv.conf type ext4 (rw,relatime,data=ordered) /dev/sda2 on /etc/hostname type ext4 (rw,relatime,data=ordered) /dev/sda2 on /etc/hosts type ext4 (rw,relatime,data=ordered)
You can see from the above that Docker is mapping some of the system files from the host to the container to allow for DNS resolution. If I make a change to
/etc/hosts on the host, it will be automatically reflected on
/etc/hosts in the container.
Remember my Apache rant from Part 1? In my case, I was moving from a years-old instance of Apache running on my Ubuntu server to Nginx running in a container. But, how do I make the existing websites that live in
/var/www/html available to the Nginx container so that the sites will continue to work? More importantly, how do I continue updating websites on the host and still have them be current in the Nginx container? I simply map
/var/www/html on the host to
/data/www on the container (and inside of the Nginx configuration file, I set
/data/www as the root of my website.)
This can be done with the
docker run command using the
-v option, like this:
docker run --name nginx -v /var/www/html:/data/www -p 80:80 -p 443:443 -d nginx
But why would you use the
docker run command when I've shown you Docker Compose? You wouldn't, so here's how we do it in the
version: '3' services: nginx: image: nginx container_name: nginx volumes: - /var/www/html:/data/www ports: - 80:80 - 443:443
Note: You can see that even though we're using Bind Mounts, we still use the
volumes:field to declare them.
We will cover the
ports:options in our next post on Networking.
We can also use Bind Mounts for files. Coming back to our Nginx example again, we will map
/opt/nginx/nginx.conf file that contains our Nginx configuration to
docker run --name nginx -v /var/www/html:/data/www -v /opt/nginx/nginx.conf:/etc/nginx/nginx.conf -p 80:80 -p 443:443 -d nginx
version: '3' services: nginx: image: nginx container_name: nginx volumes: - /var/www/html:/data/www - /opt/nginx/nginx.conf:/etc/nginx/nginx.conf ports: - 80:80 - 443:443
Bind Mounts are also useful for sharing data between containers, as you can map the host folder or file to multiple containers. For example, if you created an FTPd container to allow remote access for users to update their website files, you could map the
/var/www/html directory to that container as well. Any updates made by users to files in this directory will also be visable in the Nginx
The best part? Bind Mounts are persistent. You can start, stop, and destroy containers, and the files on the host will remain in-tact. This allows you to upgrade and change containers, and only need to change the mappings to have them appear in the new containers.
Note: One thing I've been trying to do with Bind Mounts on my system is to put everything that I want to share from my host inside of
/opt/docker/<container_name>. This will make things easier to migrate in the future, and the only thing I really have to adjust is the mapping in my
docker-compose.ymlfile to give the container what it expects.
Volumes provide similar benefits as Bind Mounts do, however there are a few differences:
- Volumes are managed by Docker and can be manipulated by the Docker CLI
- files and directories in Volumes, generally speaking, are not intended to be manipulated directly on hosts, though they can be backed-up and migrated
- Volumes are more portable from a Docker perspective, and Docker provides Volume drivers to allow Volume storage in remote locations such as cloud providers
- Volumes do not rely on an explicit path, location, or even filesystem type on the host; making them more flexible
- a container can more easily populate a new Volume with its contents
Note: The last point might be important to you, depending on what kind of containers you use. I've had inconsistent experiences using Bind Mounts with some containers that need to pre-populate files and I've learned that it depends on how the container is built.
A good write-up on the differences and use-cases for each method can be found here.
Using Volumes is as simple as using Bind Mounts. There is only one extra thing that you have to do in your
docker-compose.yml file, which is to declare the volume with a name. Here's an example with Nginx configured similar to above, except that we're going to to declare a Volume, named
web_sites: for Nginx to store our websites in, and then map that Volume to
version: '3' services: nginx: image: nginx container_name: nginx volumes: - web_sites:/usr/share/nginx/html - /opt/nginx/nginx.conf:/etc/nginx/nginx.conf ports: - 80:80 - 443:443 volumes: web_sites:
Now let's bring up the container so we can see what happens:
[email protected]:~/docker $ docker-compose up -d nginx Creating volume "docker_web_sites" with default driver Recreating nginx ... done
We can see that Docker creates the new volume, but where is it? On Linux, Docker stores Volumes in
Note: You might notice that the volume name has been given a prefix of
docker_. This is derived from the directory we are in and can be overridden if desired.
Let's have a look at the contents, first inside of the container, using the
docker-compose run command, which allows us to execute a single command inside a container:
[email protected]:~/docker $ docker-compose run nginx ls -al /usr/share/nginx/html total 16 drwxr-xr-x 2 root root 4096 Aug 14 20:14 . drwxr-xr-x 3 root root 4096 Jul 23 19:31 .. -rw-r--r-- 1 root root 494 Jul 23 11:45 50x.html -rw-r--r-- 1 root root 612 Jul 23 11:45 index.html
Nginx has populated the directory with a some default files. Looking at the Volume directory on the host, we see the same contents:
[email protected]:~/docker $ sudo ls -al /var/lib/docker/volumes/docker_web_sites/ total 12 drwxr-xr-x 3 root root 4096 Aug 14 15:14 . drwx------ 3 root root 4096 Aug 14 15:14 .. drwxr-xr-x 2 root root 4096 Aug 14 15:14 _data [email protected]:~/docker $ sudo ls -al /var/lib/docker/volumes/docker_web_sites/_data total 16 drwxr-xr-x 2 root root 4096 Aug 14 15:14 . drwxr-xr-x 3 root root 4096 Aug 14 15:14 .. -rw-r--r-- 1 root root 494 Jul 23 06:45 50x.html -rw-r--r-- 1 root root 612 Jul 23 06:45 index.html
As with Bind Mounts, we can share Volumes between containers. Inside of the same
docker-compose.yml, we would just map the same Volume that we already declared, and it will have access to it.
Which one should you use?
As with everything IT related, the answer is: it depends.
Like I mentioned, I find Bind Mounts a bit easier to work with, but there are pros and cons to each. As I continue to learn more about Docker, I'll probably tend towards Volumes more and more, but for now I will keep using a mix of both depending on my needs.
Next-up - Networking in Docker (and Docker Compose). Oh yay, Networking!!
Continue to Part 5.
Do you have a preference between Volumes and Bind Mounts or see any benefits that I've missed? Feel free to comment below or Tweet at me @eiddor and give me your feedback!