1. Prerequisites
Before we dive into the process, ensure you have the following prerequisites:
- Linux servers: At least three servers (1 master node, 2 worker nodes) running Ubuntu 20.04 LTS.
- Access to the internet: The nodes will need internet access to download and install packages.
2. Environment Setup
Ensure that these 3 virtual or physical machines can see each other. If you don't have 3 static IPs for them, you can set them up on the same LAN as follows:
- Master Node: master-node (IP: 192.168.1.100)
- Worker Node 1: worker-node-1 (IP: 192.168.1.101)
- Worker Node 2: worker-node-2 (IP: 192.168.1.102)
2.1 Update and Upgrade the Servers
Start by updating and upgrading all your servers:
sudo apt-get update
sudo apt-get upgrade -y
2.2 Install Docker
Kubernetes uses Docker as its container runtime. Install Docker on all three nodes:
sudo apt-get install -y docker.io
sudo systemctl enable docker
sudo systemctl start docker
2.3 Install Kubernetes Components
We need to install kubeadm, kubelet, and kubectl on all nodes:
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
2.4 Initialize the Kubernetes Master Node
On the master node, we'll initialize the cluster. This process sets up the control plane:
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
Once the initialization is complete, you'll see a command to join the worker nodes to the cluster. Save this command as we'll need it later.
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/networking/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.1.100:6443 --token abcdef.0123456789abcdef
--discovery-token-ca-cert-hash sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
When I set up the initial master node, I encountered several errors.
container runtime is not running: output: level=fatal msg="validate service connection: CRI v1 runtime API is not implemented for endpoint "unix:///var/run/containerd/containerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"
So I do:
sudo apt remove containerd
sudo apt update
sudo apt install containerd.io
sudo rm /etc/containerd/config.toml
sudo systemctl restart containerd
When
E0616 21:27:18.529602 14900 run.go:74] "command failed" err="failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false.
The kubelet is operating with swap enabled, whereas Kubernetes explicitly discourages the use of swap.
So I do:
sudo swapoff -a
sudo sed -i '/swap/d' /etc/fstab
sudo systemctl restart kubelet
sudo systemctl status kubelet
If you encounter a port error
sudo fuser -k 6443/tcp
sudo fuser -k 10259/tcp
sudo fuser -k 10257/tcp
sudo fuser -k 10250/tcp
sudo fuser -k 2397/tcp
sudo fuser -k 2380/tcp
sudo rm /etc/kubernetes/manifests/kube-apiserver.yaml
sudo rm /etc/kubernetes/manifests/kube-controller-manager.yaml
sudo rm /etc/kubernetes/manifests/kube-scheduler.yaml
sudo rm /etc/kubernetes/manifests/etcd.yaml
sudo rm -r /var/lib/etcd
3.1 Configure kubectl
To start using your cluster, configure kubectl for the current user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
3.2 Deploy a Pod Network
Kubernetes requires a Pod network to allow communication between nodes. We'll use Flannel:
kubectl apply -f https:
4. Join Worker Nodes to the Cluster
On each worker node, run the command provided by the master node after the initialization. It should look something like this:
sudo kubeadm join 192.168.1.100:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>
5. Verify the Cluster Setup
To verify that all nodes have joined the cluster, run:
kubectl get nodes -o wide
You should see all three nodes (1 master and 2 workers) listed as ready.
6. Deploying a Sample Application
Let's deploy a simple Nginx application to verify that the cluster is working correctly.
Create a Deployment
We'll create an Nginx deployment:
kubectl create deployment nginx --image=nginx
Expose the Nginx deployment as a service:
kubectl expose deployment nginx --port=80 --type=NodePort
Verify the Deployment
kubectl get pods
The output should show one Nginx pod running
Find the NodePort assigned to the Nginx service:
kubectl get service nginx
Access the Nginx service using the <node ip=""> : <nodeport> in your browser. You should see the Nginx welcome page.
7. Scaling the Application
Let's scale the Nginx deployment to 3 replicas to see how Kubernetes manages multiple instances:
kubectl scale deployment nginx --replicas=3
Verify the scaling:
kubectl get pods
You should see three Nginx pods running.
8. Cleaning Up
To clean up the deployment and service:
kubectl delete service nginx
kubectl delete deployment nginx
9. Typical installation problems
9.1 When encountering the FORBIDDEN
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get pv
kubectl get deploy
9.2 Couldn't get current server API
E0703 22:36:32.702012 9430 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
E0703 22:36:32.702614 9430 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
E0703 22:36:32.705110 9430 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
E0703 22:36:32.705619 9430 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
E0703 22:36:32.707190 9430 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Solution:
Step 1: On “Worker node/Compute node” look for file "/etc/kubernetes/admin.conf ", If it is not available than “Copy paste/SCP” from your “Controller node/master node”.
Step 2: On “Worker Node/Compute node” create a directory called “.kube” on user home directory, if it is already exist than Not to worry and move inside this directory.
mkdir ~/.kube
or,
mkdir -p $HOME/.kube
cd ~/.kube
Step 3: Create file called “config” and paste the content of “/etc/kubernetes/admin.conf”.
vi config
Note: While Copy the content on config file make sure the below mention parameter has correct master IP mentioned, Some time it is localhost but try to update master IP.
10. Conclusion
In this guide, we've successfully set up a Kubernetes cluster from scratch, deployed a sample application, and verified its functionality. Building a Kubernetes cluster manually gives you a deeper understanding of the underlying infrastructure and how Kubernetes operates. This knowledge is essential for troubleshooting and optimizing your cluster in a production environment.
By following this guide, you're now equipped to create and manage your Kubernetes clusters, deploy applications, and scale them according to your needs. Happy clustering!
