Setting Up a Docker Swarm Cluster: A Comprehensive Guide

Introduction

Container orchestration has become a fundamental practice in modern software development and operations, especially as applications grow in complexity and scale. While Kubernetes is widely regarded as a robust orchestration tool, Docker Swarm, Docker's native orchestration solution, is simpler to set up and use, making it a great choice for small- to medium-scale containerized environments.

Docker Swarm turns a group of Docker hosts into a single virtual server, providing load balancing, automatic failover, scaling, and service discovery out of the box. In this post, we'll take a deep dive into what Docker Swarm is, why it might be the right choice for your projects, and how to set up a Docker Swarm cluster from scratch.

1. What is Docker Swarm?

Docker Swarm is Docker's native clustering and orchestration tool for managing Docker containers across multiple hosts. By setting up a Swarm, you can combine several Docker hosts to form a unified cluster, distributing workloads automatically across nodes. Docker Swarm provides container scheduling, load balancing, scaling, self-healing, and service discovery.

Swarm mode was integrated into Docker starting from version 1.12, which means you don’t need any extra software to run a Docker Swarm cluster—it’s built right into Docker itself.

2. Why Use Docker Swarm?

Docker Swarm is particularly useful for users and organizations looking for a simple, lightweight, yet powerful container orchestration tool. Here are some reasons to consider Docker Swarm:

• Simplicity: Docker Swarm is easier to set up and configure compared to Kubernetes. If you're already using Docker, Swarm is a natural progression, requiring little additional knowledge.
• Integrated with Docker: Swarm mode is built directly into Docker. This means you can use the same Docker CLI commands (such as docker service, docker stack, etc.) to manage Swarm resources.
• Decentralized Design: Docker Swarm uses a decentralized approach where workloads are spread across multiple nodes. This enhances fault tolerance and ensures high availability.
• Built-In Load Balancing: Docker Swarm automatically distributes traffic to different containers (replicas) of a service across multiple nodes, ensuring optimal load distribution and failover.
• Service Discovery: In Swarm, each service can be automatically discovered using DNS-based service discovery, making it easy to connect different services together.

While Docker Swarm may not offer the extensive feature set of Kubernetes, it strikes a balance between simplicity and powerful orchestration features for those who want to manage containers across multiple hosts without the complexity of Kubernetes.

3. Docker Swarm Architecture: Manager Nodes and Worker Nodes

Before we dive into setting up the cluster, it’s important to understand the architecture of a Docker Swarm. A Docker Swarm cluster consists of two types of nodes:

• Manager Nodes: These nodes handle the orchestration and management of the Swarm cluster. They are responsible for scheduling tasks, maintaining the cluster state, and coordinating the Swarm. Only manager nodes can manage the Swarm (create, remove, update services, etc.), but they can also run containers.
• Worker Nodes: Worker nodes are responsible for running the actual containers. They receive tasks from manager nodes and execute them. Worker nodes cannot control the Swarm and only perform the tasks assigned by the manager nodes.

Swarm clusters can have multiple manager nodes to ensure fault tolerance. A best practice is to have an odd number of manager nodes (e.g., 3 or 5) to maintain quorum in case of failures.

4. Prerequisites for Setting Up a Docker Swarm Cluster

Before setting up a Docker Swarm cluster, ensure that you have the following prerequisites:

1. Docker Installed on All Nodes: Docker Swarm is built into Docker itself, so you need Docker installed on all nodes (manager and worker).
2. Multiple Machines: You need at least two machines (or virtual machines) to set up a Swarm—one for the manager and one for a worker node. These could be physical servers, cloud instances (AWS, GCP, etc.), or virtual machines (VMs).
3. Network Connectivity: All nodes in the Swarm cluster must be able to communicate with each other over the network. Ensure that necessary ports are open between nodes.
   • TCP port 2377 for communication between manager and worker nodes.
   • TCP and UDP port 7946 for control and data plane communications.
   • UDP port 4789 for the container network overlay.

5. Setting Up Docker Swarm Cluster: A Step-by-Step Guide

Step 1: Install Docker on All Nodes

First, ensure that Docker is installed on all the machines that you intend to use in the Swarm cluster. You can install Docker by following the official Docker installation guide for your respective platform:

• Docker installation for Linux
• Docker installation for Windows
• Docker installation for Mac

Or you can checkout my previous post: Installing Docker on Different Platforms (Linux, Mac, Windows).

Once Docker is installed, you can verify it by running the following command on each machine:

docker --version

This should return the version of Docker installed on the machine.

Step 2: Initialize the Docker Swarm on the Manager Node

Once Docker is installed on all machines, choose one machine to be the manager node. SSH into that machine and run the following command to initialize the Docker Swarm:

docker swarm init --advertise-addr <MANAGER-IP>

Replace <MANAGER-IP> with the IP address of the manager node. This command initializes the Swarm cluster and sets the node as the manager.

After successful initialization, you will see output similar to the following:

Swarm initialized: current node (your-node-id) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token <SWARM-TOKEN> <MANAGER-IP>:2377

This output provides you with the token and command needed to join other worker nodes to the Swarm.

Step 3: Adding Worker Nodes to the Cluster

Now, move to each worker node and SSH into the machine. On each worker node, run the docker swarm join command provided in the output from Step 2.

For example:

docker swarm join --token <SWARM-TOKEN> <MANAGER-IP>:2377

Replace <SWARM-TOKEN> with the actual token provided and <MANAGER-IP> with the manager node’s IP address.

Once the worker node successfully joins the Swarm, you’ll see a message confirming the node’s membership in the cluster:

This node joined a swarm as a worker.

Step 4: Verifying the Cluster Status

Once all worker nodes have joined the Swarm, go back to the manager node and run the following command to verify the status of your Swarm cluster:

docker node ls

This command will list all nodes in the Swarm cluster, including their role (Manager or Worker) and status. You should see the manager node and all worker nodes listed.

6. Deploying Services in Docker Swarm

Now that your Swarm cluster is up and running, you can deploy services across the cluster. In Docker Swarm, services are the tasks that run on the cluster nodes.

Step 1: Deploy a Service

To deploy a service, use the docker service create command. For example, let’s deploy an Nginx web server service with three replicas:

docker service create --name nginx-service --replicas 3 -p 80:80 nginx

This command creates a service named nginx-service, runs three replicas of the Nginx container, and exposes port 80 on all nodes.

Step 2: Verify the Service

To verify that the service has been deployed, use the following command:

docker service ls

This will show the deployed services and the number of running replicas.

You can also inspect individual tasks (replicas) with:

docker service ps nginx-service

7. Scaling Services in Docker Swarm

One of the benefits of using Docker Swarm is its ability to easily scale services. You can scale services up or down based on the demand.

To scale a service, use the docker service scale command. For example, to scale the nginx-service to five replicas:

docker service scale nginx-service=5

Docker will automatically distribute the new replicas across the nodes in the Swarm cluster.

8. Docker Swarm Load Balancing and Service Discovery

Docker Swarm provides built-in load balancing and service discovery. When you expose a service, Swarm automatically distributes incoming traffic across all replicas of the service. You don’t need to manually configure load balancers or proxies.

Swarm also provides DNS-based service discovery. Services in a Swarm cluster can be accessed by their service name, and Docker will resolve the service name to the appropriate container(s).

9. Monitoring and Managing the Docker Swarm Cluster

You can monitor the Swarm cluster and the running services using various commands:

• To view node status: docker node ls
• To inspect service details: docker service inspect <service-name>
• To view logs of a service: docker service logs <service-name>

For advanced monitoring, you can integrate third-party tools like Prometheus, Grafana, or Portainer to gain better insights into the performance and status of the cluster.

10. Cleaning Up: Leaving and Removing Nodes from the Swarm

If you need to remove a worker node from the Swarm, SSH into the worker node and run:

docker swarm leave

To force a node to leave the Swarm, use:

docker swarm leave --force

To remove a node from the Swarm cluster from the manager node, run:

docker node rm <node-ID>

Conclusion

Setting up a Docker Swarm cluster is a straightforward and efficient way to manage containers across multiple hosts. By using Docker Swarm, you can automate service deployment, scaling, and load balancing in a simple, native Docker environment.

While Docker Swarm may not offer as many features as Kubernetes, it excels in simplicity and ease of use. This makes it an excellent choice for teams or projects that require basic orchestration without the overhead of more complex tools.

By following this guide, you should now have a fully functional Docker Swarm cluster, ready to deploy and scale containerized applications.