Fun with Docker - Part 6: Monitoring your Docker setup with Portainer

By now, hopefully we've covered enough Docker to get you started with your own learning and exploration.  We wrapped things up in a nice networky bow in Part 5 with an overview of Docker Networking.  In this post, we'll have a look at a tool that allows you to manage and monitor your Docker setup using a GUI.  Bonus: We get to run the tool in a container.

The Fun with Docker Series

Links to the entire series are here:

• Part 1 - Docker - An introduction
• Part 2a - Getting started with Docker
• Part 2b - More getting started with Docker
• Part 3 - Docker Compose
• Part 4 - Docker Volumes (and Bind Mounts)
• Part 5 - Docker Networking
• Part 6 - Monitoring your Docker setup with Portainer (this post)
• Part 7 - Setting up this blog with Let's Encrypt, Nginx, and Ghost

So far, I've shown you two ways to manage and monitor your Docker environment: the Docker CLI and Docker Compose.  As with everything IT related, there are many other ways to do things depending on your requirements. If you are running in a large distributed Docker production environment, you might use things like Docker Swarm or Kubernetes (K8s), but those are out of scope for our  learning purposes (or at least for mine).

There is a tool that we can use to give us a more organized (and prettier) view of our hobby environment: Portainer.  It will even allow us to manipulate Docker elements like containers, Volumes, and networks right from the GUI.

You can launch a live demo directly from the Portainer website, but it's so easy to install it for yourself as a container there's probably no need.

The first thing we should do is get a few containers running so that we have some data to look at.  In this case, we'll work with the blogging environment that we setup in our last post that with Nginx, Ghost, and MySQL containers.

Starting Portainer

As I mentioned in the intro, we can actually run Portainer as a container, which gives us the added benefit of being able to experiment without cluttering up our host with build files and dependancies.  Starting Portainer as a container using the Docker CLI is as simple as this:

docker run -d -p 9000:9000 --name portainer --restart always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce

Using Docker Compose, our docker-compose.yml file would look like this:

version: '3'
volumes:
  portainer_data:
services:
  portainer:
    image: portainer/portainer-ce
    container_name: portainer
    restart: unless-stopped
    volumes:
      - portainer_data:/data
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      - 9000:9000

To review these options:

• create a Volume named portainer_data: for Portainer to use for persistent storage.
• have the container automatically restart unless we manually stop it
• the portainer_data Volume is mapped to /data in the container
• we also use a Bind Mount to map the Docker socket file from our host to a corresponding one in the container so that Portainer can read/manage Docker
• we forward TCP port 9000 from the host to TCP port 9000 in our container for GUI access

After starting it, we can see using docker-compose ps that we have four containers running now:

pi@raspberrypi:~/blog $ docker-compose ps
  Name                 Command              State               Ports            
---------------------------------------------------------------------------------
ghost       docker-entrypoint.sh node       Up      2368/tcp                     
            ...                                                                  
mysql       /entrypoint.sh mysqld           Up      3306/tcp                     
nginx       nginx -g daemon off;            Up      0.0.0.0:443->443/tcp,        
                                                    0.0.0.0:80->80/tcp           
portainer   /portainer                      Up      0.0.0.0:9000->9000/tcp

And that's it! Portainer is now running and ready to configure. You can access the GUI with http://<host_ip>:9000/

Configuring Portainer for the first time

Since this is a new installation, the first screen you will see is the admin user creation page. Enter a password here for the admin user and remember it, since Portainer seems to have a pretty short login timeout.

After clicking "Create user," you'll be take to a screen that allows you to add Docker environments for Portainer to manage.

We'll start with our local Docker instance that we tied to Portainer with our Volume mount earlier, so click the "Local" tab on this screen.  You'll be presented with instructions to activate this connection, but we've already followed those when we started the container, so just click "Connect" (unless you're doing all of this in Windows, in which case, I can't help you).

Note: Portainer can also manage remote Docker environments in a variety of ways.  We'll cover one of these below.

Portainer is now configured and ready for use!

Portainer features

I can't possibly cover every feature and screen in Portainer, but I'll go through a few of the cool ones and highlight some of the things we've learned in this series.

After configuring Portainer, you'll be taken to the Endpoints screen where you will see the local Docker instance that we just added along with some statistics about it.  Note that we have our four containers, four volumes, and images from previous testing.  

Click on the local endpoint to drill down to the Endpoint info screen to see even more data about our Docker environment.

Besides the stuff we already mentioned, you can also see that we have 8 Networks in our Docker environment.  Click through some of the options to get more detailed information on each of the elements, or follow the flow below.

Containers

Click the Containers icon.

On the Containers screen, you'll see each of our running containers along with some basic information about each one.  Notice that you can select one or more containers and then click any of the buttons at the top to quickly Start, Stop, Restart, etc. any of them.  Clicking on a container name will drill down into the Container details screen and give you a lot of information about it while also allowing you to create a new image based on your container or change other settings.

You can even add a new container with Portainer, though I won't cover it because by now we're experts at Docker Compose.  Right?

Also notice the "Quick actions" section where you will see four icons that will save you from having to memorize a few Docker CLI commands:

• Logs - allows you to view realtime logs from each container
• Inspect - allows you to inspect the Docker parameters of each container (similar to the docker inspect command
• Stats - show memory, CPU, and network statistics for each container
• Exec Console - takes you to the CLI of your container, similar to the docker exec -it <container-id> bash command (very handy!!)

Volumes

In the Volumes section of Portainer, we can see a list of our Volumes where we are able to remove them or add new ones.

Clicking on one of the Volumes will take you to the Volume details screen which shows you even more information about the Volume.  Note here that you can see where the Volume lives on the host under "Mount path." At the bottom of the Volume details page, you can see which containers are using the Volume.

Networks

Now let's have a look at the Networks page in Portainer where, as you might guess, we'll see a list of all of the Docker networks that have been created with details about each.

Note: the bridge, host, and none networks can't be removed, as they are created by Docker during installation.

Click on the blog_ghost network to see some details about it. The bottom of this screen shows that our Nginx and Ghost containers are connected to it, which is what we covered in the last post.

Images

Finally, let's go to the Images section, where we'll see a list of images on your Docker system.  We can see how busy we've been testing with Docker (and I've pruned this a few times).

As you might expect, clicking on an image will take you to the Image details screen which will show you a lot of the details about each Image.  This will come in handy if you ever want to build your own images.

Monitoring other Docker hosts

All of the information above came from the same host that Portainer was running on, but what if you wanted to use Portainer to monitor Docker on other hosts in your network?

This is quite simple to do. We just have to enable the remote API for Docker, which is just a matter of overriding one line in your host's startup configuration for Docker.

This configuration change will work with both Ubuntu and Raspberry Pi hosts assuming they use systemd for startup.

Enter sudo systemctl cat docker | grep ExecStart to view the current startup command for Docker. It should look something like this:

pi@RPi4a:~ $ sudo systemctl cat docker | grep ExecStart
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

To this line, we want to insert -H tcp://0.0.0.0:2375, which will bind the Docker daemon to the host's IP address(es) along with the local socket file. The final line should look like this:

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H fd:// --containerd=/run/containerd/containerd.sock

We're going to add this new line to the systemd override file for our Docker daemon.

Create/edit this override file and structure with sudo systemctl edit docker and enter the following:

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H fd:// --containerd=/run/containerd/containerd.sock

Save the file and restart the Docker daemon with the following command:

sudo systemctl restart docker

Important Note: Doing this comes with some security implications, as Docker on this host will now be reachable from your network with full access. Please ensure that your network is secure before opening up this connection. You can also use TLS to secure it. More information on securing this connection can be found here.

Now, let's go back to Portainer so that we can add the new host to it.  Click on "Endpoints" in the sidebar, which will take us to a list of our endpoints:

Click "Add endpoint" to be taken to the Create endpoint screen.  Enter a name for the new endpoint and the IP address with the TCP port number of the Docker daemon (2375 using the above configuration):

Finally, click "Add endpoint" at the bottom to save this new endpoint.  You will be taken back to the Endpoint management screen and you should see your local endpoint along with the new one that we just added.

Clicking on the Home button in the sidebar will now show us both hosts with their summaries:

As you can see, Portainer is a very handy tool to monitor and manage your Docker environment. It can save you a lot of time and typing if you are trying to troubleshoot something and need to quickly restart containers.

In the next (and final!) post of this series, I'm going to give a quick overview of how I setup this blog using Docker, Let's Encrypt, Nginx, and Ghost.  

Continue to Part 7.

How are you using Portainer?  What other Docker tools do you like to use? Feel free to leave your comments below or Tweet them at me @eiddor. I'm also starting to think about other topics to cover in the future (they don't have to be Docker related), so if you have any ideas, let me know!