Understanding Kubernetes Networking — For newbies

The aim of this blog is to present a high level concept of the different networking methods in Kubernetes.

What Is Kubernetes?

You can define Kubernetes as an open-source container orchestration tool that provides a portable platform for automating the deployment of containerized applications.

Now, anybody working with Kubernetes must have a clear understanding of Kubernetes Cluster as that will help you in understanding Kubernetes Networking.

Kubernetes Cluster

The Kubernetes platform offers desired state management, which enables the cluster services to run, the fed configuration in the infrastructure. Let me explain with an example.

Consider a YAML file which has all the configuration information which needs to be fed into the cluster services. So, this file is fed to the API of cluster services, and then it will be up to the cluster services to figure out how to schedule pods in the environment. So, suppose there are two container images for pod 1 with three replicas, and one container image for pod 2 with two replicas, it will be up to the cluster services to allocate these pod-replica pairs to the workers.

Refer to the above diagram. Now, as you can see that the cluster services have allotted the first worker with two pod replica pairs, the second worker with a single pod-replica pair, and the third worker with two pod replica pairs. Now, it is the Kubelet process which is responsible for communicating the cluster services with workers.

So, this whole setup of cluster services and the workers themselves makes up this Kubernetes cluster!!

How, do you think these individually allocated pods communicate with each other?

The answer lies in Kubernetes Networking!

There are mainly 4 problems to solve with the networking concepts.

  • Container to container communication
  • Pod to pod Communication
  • Pod to service communication
  • External to service Communication

Now, let me tell you how are the above problems are solved with Kubernetes Networking.

Kubernetes Networking

The communication between pods, services and external services to the ones in a cluster brings in the concept of Kubernetes networking.

So, for your better understanding let me divide the concepts into the following.

  • Pods & Container Communication
  • Services
  • Connecting External to Services via Ingress Network

Pods & Container Communication

Before I tell you how do pods communicate, let me introduce you what are pods?

Pods

Pods are basic units of Kubernetes applications, that consists of one or more containers allocated on the same host to share a network stack and other resources. So, this implies that all containers in a pod can reach other on a local host.

Now, let me brief you on how do these pods communicate?

There are 2 types of communication. The inter-node communication and the intra-node communication.

So, let’s start with intra-node communication, but before that let me introduce to you the components of the pod network.

Intra-node Pod Network

Intra-node pod network is basically the communication between two different nodes on the same pod. Let me explain you with an example.

Assume a packet is going from pod1 to pod2.

  • The packet leaves Pod 1’s network at eth0 and enters the root network at veth0
  • Then, the packet passes onto the Linux bridge(cbr0) which discovers the destination using an ARP request
  • So, if veth1 has the IP, the bridge now knows where to forward the packet.

Now, similarly let me tell you about the inter-node pod communication.

Inter-node pod network

Consider two nodes having various network namespaces, network interfaces, and a Linux bridge.

Now, assume a packet travels from pod1 to a pod4 which is on a different node.

  • The packet leaves the pod 1 network and enters the root network at veth0
  • Then the packet passes on to the Linux bridge (cbr0) whose responsibility is to make an ARP request to find the destination.
  • After the bridge realizes that this pod doesn’t have the destination address, the packet comes back to the main network interface eth0.
  • The packet now leaves the node 1 to find it’s destination on the other node and enters the route table who routes the packet to the node whose CIDR block contains the pod4.
  • So, now the packet reaches node2 and then the bridge takes the packet which makes an ARP request to find out that the IP belonging to veth0.
  • Finally, the packet crosses the pipe-pair and reaches pod4.

So, that’s how pods communicate with each other. Now, lets’ move on and see how services help in the communication of pods.

So, what do you think the services are?

Services

Basically, services are a type of resource that configures a proxy to forward the requests to a set of pods, which will receive traffic & is determined by the selector. Once the service is created it has an assigned IP address which will accept requests on the port.

Now, there are various service types that give you the option for exposing a service outside of your cluster IP address.

Types of Services

There are mainly 4 types of services.

ClusterIP: This is the default service type which exposes the service on a cluster-internal IP by making the service only reachable within the cluster.

NodePort: This exposes the service on each Node’s IP at a static port. Since, a ClusterIP service, to which the NodePort service will route, is automatically created. We can contact the NodePort service outside the cluster.

LoadBalancer: This is the service type which exposes the service externally using a cloud provider’s load balancer. So, the NodePort and ClusterIP services, to which the external load balancer will route, are automatically created.

ExternalName: This service type maps the service to the contents of the externalName field by returning a CNAME record with its value.

So, guys that was all about services. Now, you might be wondering how do external services connect to these networks right?

Well, that’s by none other than Ingress Network.

Ingress Network

Well, Ingress network is the most powerful way of exposing services as it is a collection of rules that allow inbound connections, that can be configured to give services externally through reachable URLs. So, it basically acts as an entry point to the Kubernetes cluster that manages external access to the services in a cluster.

Now, let me explain to you the working of Ingress Network with an example.

We have 2 nodes, having the pod and root network namespaces with a Linux bridge. In addition to this, we also have a new virtual ethernet device called flannel0(network plugin) added to the root network.

Now, we want the packet to flow from pod1 to pod 4.

  • So, the packet leaves pod1’s network at eth0 and enters the root network at veth0.
  • Then it is passed on to cbr0, which makes the ARP request to find the destination and it thereafter finds out that nobody on this node has the destination IP address.
  • So, the bridge sends the packet to flannel0 as the node’s route table is configured with flannel0.
  • Now, the flannel daemon talks to the API server of Kubernetes to know all the pod IPs and their respective nodes to create mappings for pods IPs to node IPs.
  • The network plugin wraps this packet in a UDP packet with extra headers changing the source and destination IP’s to their respective nodes and sends this packet out via eth0.
  • Now, since the route table already knows how to route traffic between nodes, it sends the packet to the destination node2.
  • The packet arrives at eth0 of node2 and goes back to flannel0 to de-capsulate and emits it back in the root network namespace.
  • Again, the packet is forwarded to the Linux bridge to make an ARP request to find out the IP that belongs to veth1.
  • The packet finally crosses the root network and reaches the destination Pod4.

So, that’s how external services are connected with the help of an ingress network.