VMware vSphere last summer announced it was going Kubernetes-native. This means that integrated VMware and Kubernetes becomes a reality for all vSphere users. Despite all the benefits this integration promises, running Kubernetes on VMware is not exempt from challenges. This article presents an overview of running Kubernetes on VMware and the challenges it presents.
Virtual Machines (VM) vs Containers: What’s the Difference?
The main difference between VMs and containers is the level at which both systems operate. VMs divide a physical server into a number of virtual servers, working at the hardware level. Containers, on the other hand, package and isolate software, working at the operating system level.
Both VM and containers provide different advantages for organizations. Containerization offers companies a fast way to deploy software. Machine virtualization offers users isolation at the hardware level and the full use of the operating system. Combining both technologies by running Kubernetes on VMware provides companies with the security of VM and the portability of containers.
Kubernetes Overview
Kubernetes is an open-source container orchestration platform that automates the management of contaainerized applications. Software containerization has become increasingly popular in the last years.
Containers are portable and light, which helps developers release software faster. However, manually managing hundreds of containers is a time-consuming task. Kubernetes enables companies to automate container management and deployment processes. For example, scaling apps up or down, or moving workloads across environments.
Kubernetes Pros and Cons
Pros
- Scalability—Kubernetes enables organizations to run billions of containers.
- Broad support—it boasts a large and active community of contributors.
- Multiple deployment options—you can use Kubernetes to run replicated apps, microservices, and stateful apps.
- Built-in security—Kubernetes offers security features such as role-based access control (RBAC) and Secrets API, which enables you to securely manage sensitive information.
- Integration—Kubernetes integrates with all major cloud providers, such as AWS, Google Cloud and Microsoft Azure.
- Open-source—Kubernetes is free and you can download it from the GitHub repository.
Cons
- Hard to learn—Kubernetes is complex to learn and implement. You can find numerous training courses and tutorials online. The Kubernetes documentation is a great start, since it has sections for beginners. Several Kubernetes service providers also offer training.
- Difficult to configure—configuring Kubernetes requires manually configuring several separate components to initialize the cluster. You need to configure the high availability of masters and components, along with other requirements.
- Pods manual restart—sometimes, like when running at full capacity, you need to manually restart the pods.
- It doesn’t have tools compatible with Docker CLI—k8s uses a different configuration and YAML definitions.
VMware Overview
A virtual machine is a digital entity that behaves like a computer. A VM can be a fire, a system, or an entire operating system. You can reboot each VM independently since it runs its own operating system.
VMware vSphere is a cloud virtualization platform. vSphere enables users to allocate application workloads in cloud-based VMs as needed. A Type-1 hypervisor, ESXi runs on the host computers and manages the resource allocation. ESXi assigns resources, such as virtual memory and computing, to each operating system. It does not require an existing operating system because it runs in the bare-metal server.
Advantages and disadvantages of VMware
Pros
- High availability—you can configure redundant servers to achieve high availability in case of a disaster or server failure.
- Visibility—VMware features a service console that lets you know the activity and usage of the VM.
- Built-in security—VMs include security features such as encryption and role-based access control. This restricts access to the data, protecting it.
Cons
- Price—VMware is not free, and the cost of license upcharges can add up. The platform charges per-processor without limiting the number of virtual machines.
- Complex interface—the interface can be confusing because of the wide range of configuration options.
- Lack of detail in error messages—the system only gives general error messages. Often users need to look for support in VM forums to find out the source of the error.
Challenges of Running Kubernetes on VMware
Challenge #1 Complexity of operation
Running open-source software on bare metal is a complex task. One of the challenges is the provisioning time it takes to get physical hardware, which in the US is an average of 86 days. On top of that, moving workloads in a bare-metal environment is akin to killing the workload.
Solution: VMware vSphere extracts the hardware from a Kubernetes cluster via a virtual hardware compatibility list. You can move your workload in VMware using vMotion.
Challenge #2 Slow VM when facing higher loads
Sometimes VM can slow down when having to manage loads higher than normal. This can leave the node flapping or straight down. The problem is that when this happens to manager nodes, the cluster loses its quorum and the components can be unavailable. You can learn more about nodes quorum in Kubernetes here.
Solution: VMware recommends to have less than 5% of CPU ready on the physical nodes.
Challenge #3 Loss of performance advantage
A container platform typically increases the speed of the development process. However, a common issue when running k8s on VM is the pods’s running time is very slow. Some users attribute this to the hypervisor adding another layer of supervision. Another reason is that when running Kubernetes on VMware, you need to allocate resources on multiple levels.
Solution: You can use VM-level reservations to dedicate physical resources to a VM without sharing it with the others running on the same host. Another solution is using a resource pool reservation, which allocates the resources from a node running low to other active nodes in the same resource pool.
The Bottom Line
Despite the challenges, there are inarguable advantages to running K8 on VMware. Kubernetes gains availability and better utilization of the servers by combining the container infrastructure with virtual machines. In addition, VMs offer high availability to container loads. With the integration of vSphere with K8s, we can enjoy an enhanced virtualization environment.
For aNewDomain, I’m Gilad David Maayan.
