If there is one thing that network solutions need, it can manage operations seamlessly with little manual input. For proper Kubernetes operations, there are 4 distinct networking problems to address:
Highly-coupled container-to-container communications: this is solved by Pods and localhost communications.
For proper operations, all pods must have the capacity to communicate between themselves (Pod-to-Pod communication).
Pod-to-Service communications
Finally, External-to-Service communications, external traffic into the cluster entities.
There are two types of network setup:
Default k8s network,
Or CNI with its plugins – most frequently used.
Kubernetes default networking provider, kubenet, is a simple network plugin that works with various cloud providers. Kubenet is a very basic network provider, and basic is good, but does not have very many features. Moreover, kubenet has many limitations. For instance, when running kubenet in AWS Cloud, you are limited to 50 EC2 instances. Route tables are used to configure network traffic between Kubernetes nodes and are limited to 50 entries per VPC. Moreover, a cluster cannot be set up in a Private VPC, since that network topology uses multiple route tables. Other more advanced features, such as BGP, egress control, and mesh networking, are only available with different CNI providers.
CNI stands for Container Networking Interface and its goal is to create a generic plugin-based networking solution for containers. It is a library definition and a set of tools under the umbrella of the Cloud Native Computing Foundation project. Kubernetes uses CNI as an interface between network providers and Kubernetes networking.
In this article, we will be taking a look at the Calico network solution and how it helps make your Kubernetes operations better.
Here is a partial list of providers that can be installed out of the box, sorted in alphabetical order.
Calico
Canal (Flannel + Calico)
flannel
kopeio-vxlan
kube-router
romana
Weave Net
Here is a table of different features from each of the CNI providers mentioned:
What are Calico Solutions?
Calico is a scalable, secure, and simple network solution for Kubernetes operations, supporting ipv6 and ipv4. Calico provides simple, scalable networking using a pure L3 approach. It enables native, unencapsulated networking in environments that support it, including environments with L2 adjacency between nodes, or in deployments where it’s possible to peer with the infrastructure using BGP, such as on-premise. Calico also provides a stateless IP-in-IP mode that can be used in other environments, if necessary. Beyond scalable networking, Project Calico also offers policy isolation, allowing you to secure and govern your microservices/container infrastructure using advanced ingress and egress policies. It makes use of Kube-proxy solutions to manage filtering rules. The management process makes use of Linux iptables to make filtering rules on the network and isolate containers.
Better performance
Irrespective of your personal preferences, Calico is built to give you highly optimized standard network pipelines to give you the best network performance. With Calico, you have a high level of flexibility, which allows you to run your operations without the need to use overlays, and also avoiding complications like the overheads of packet encapsulation and decapsulation.
Interoperability
Calico allows workloads, whether Kubernetes or not, to communicate securely and easily. This means that the network and workload communication is done easily without any manual input. Calico extends to secure your host-based workload alongside Kubernetes. This means that only the traffic allowed to flow would flow in the network.
With so many network solutions out there and many promising a lot of solutions for your Kubernetes operations, it is safe to say that information is key to making the right choices. With Calico, you have free-flowing operations you can count on, as highlighted throughout this article.
Motivation
Lately, we needed to deploy IPv6 native Kubernetes clusters, in a few varieties of environments. Starting with Kubernetes version 1.16, it is possible to configure a dual-stack environment. Our mission was to install Kubernetes on IPv6 native environment. Calico takes a more holistic view of networking, concerning itself not only with providing network connectivity between hosts and pods, but also with network security and administration.
Unlike Flannel, Calico does not use an overlay network. Instead, Calico configures a layer 3 network that uses the BGP routing protocol to route packets between hosts. This means that packets do not need to be wrapped in an extra layer of encapsulation when moving between hosts. The BGP routing mechanism can direct packets natively without an extra step of wrapping traffic in an additional layer of traffic.
Besides the performance that this offers, one side effect of this is that it allows for more conventional troubleshooting when network problems arise. While encapsulated solutions using technologies like VXLAN work well, the process manipulates packets in a way that can make tracing difficult. With Calico, the standard debugging tools have access to the same information they would in simple environments, making it easier for a wider range of developers and administrators to understand behavior, and we needed it. I can't imagine us deploying IPv6 on unknown terrain, using other CNI.
In addition to networking connectivity, Calico is well-known for its advanced network features. Network policy is one of its most sought after capabilities. In addition, Calico can also integrate with Istio, a service mesh, to interpret and enforce policy for workloads within the cluster both at the service mesh layer and the network infrastructure layer. This means that you can configure powerful rules describing how pods should be able to send and accept traffic, improving security and control over your networking environment.
Summary
If you do not need the advanced features that a CNI provider delivers, use kubenet. It is stable, and fast. Otherwise, pick one. If you do need IPv6 or other advanced features, make a decision and test with your cluster. This blog is focused on Calico based on our IPv6 experience. I advise you to check and experience some more CNIs to decide what suits your needs. I hope you find this useful to shed light upon Kubernetes networking.