Running Containers on Bare Metal vs. VMs: Performance and Benefits

Approximately remain in this minutes read.

Running Containers on Bare Metal vs. VMs: Performance and Benefits

Running Containers on Bare Metal vs. VMs: Performance and Benefits
Written by

According to recent research, Docker is spreading like wildfire, especially in the enterprise and its adoption more than doubles to 27% from 13% percent in 2015; and another 35 percent have plans to use Docker. According to these reports, the main drivers for container adoption are the need to increase the R&D team efficiency and speed and the fact that containers are fundamental building block for microservices.

With these great benefits, it’s only natural to ask if the container is going to kill the VM. This debate gives rise to the relatively trivial question of  why not to run containers directly on irons. In this article, we will not argue that you need to move to bare metal; in fact, we think that large enterprise environments should have a mix of virtualized servers, bare-metal physical hosts, and containers.

As a by-product of researching this post, we are happy to share with you the results of a few tests that show a significant improvement in performance when running containerized workloads on bare metal in comparison to running the same on VMs. We will also discuss both options’ benefits and opportunities.

The Container Benefits

Containers provide a lightweight alternative to virtual machines (VMs). A container isolates your application from wherever you are running it. You install only what you need to run your application and nothing more. Containers allow developers to work with identical dev environments and stacks. You can also develop directly in a container as it gives you a separate network stack and storage without the overhead of building and running a virtual machine. In addition, containers facilitate continuous integration and delivery processes, and encourage the use of stateless designs.

The VM also can be used in same manner as container, but VMs have several significant drawbacks. The most critical one is that virtualization has an overhead;  no matter how thin the operating system (OS) of the guest deployed, you still need to fully copy that OS and its entire configuration when establishing a new VM. Containers run their own init processes, filesystems and network stacks, which are virtualized on top of the VM or a baremetal host OS. By their nature, containers use less memory than VMs. This is due to the fact that they inherently share the OS kernel and, in most cases, they also use the same identical libraries.

To learn about the benefits and challenges with containers and virtualization, watch this webinar.

The Virtualization Overhead

A hypervisor is used to share the hardware infrastructure, allowing multiple tenants, isolated VMs to be run on the same physical machine. A VM emulates a computer system based on computer architectures, and provides the functionality of a physical computer. This allows higher utilization of the underlying physical machine. In comparison, a bare metal server is a single tenant, which means that there is no resource sharing and the available CPU and RAM are dedicated to your processes.

The reported overhead using, for example, Hyper-V is between 9-12%, which means that a guest operating system under Hyper-V typically starts at 88-91% of the CPU available. The memory overhead when the running operating system under Hyper-V was observed to be around 340 MB of main memory. Naturally, running a process on a guest operating system might suffer from lack of resources and performs less efficiently than running the same process directly on a host (physical server) operating system.

Considering this virtualization overhead, the way containers work and their benefits prompt us to check the option to run containers directly on hosts.

Let’s Discuss Performance

The benchmarks we ran used an Amazon EC2 m4.2xlarge instance,  which has 32 GB of RAM and 8vCPUs. We used the same specification for a bare metal machine where the CPU model is Intel Core i7-3770 with 3.40GHz speed (and 8 CPU cores), 32 GB of RAM, and an SSD disk. In our tests, we measured the performance of the file system and CPU operations.

For the first two types of tests,  we used containers that were already built to run simple benchmarks on computational and file system write performance. Once deployed, we used the following command:  docker run d m 256m name=containerbenchmarkvm simplecontainerbenchmarks

As shown below, the performance of containers running on bare metal was 25%-30% better compared to running the same workloads on VMs in both CPU and IO operations. As you can see from the results sample below, the copying operations rate was at around 125MB/s on the VM in comparison to around 165MB/s. The data processing rate on the VM was at almost 13MB/s in comparison to 19MB/s on bare metal

——————————

Filesystem write performance

——————————

1073741824 bytes (1.1 GB) copied, 8.65288 s, 124 MB/s

1073741824 bytes (1.1 GB) copied, 8.44858 s, 127 MB/s

1073741824 bytes (1.1 GB) copied, 8.32321 s, 129 MB/s

1073741824 bytes (1.1 GB) copied, 8.48442 s, 127 MB/s

1073741824 bytes (1.1 GB) copied, 8.47191 s, 127 MB/s

1073741824 bytes (1.1 GB) copied, 8.43977 s, 127 MB/s

1073741824 bytes (1.1 GB) copied, 8.48138 s, 127 MB/s

1073741824 bytes (1.1 GB) copied, 8.45923 s, 127 MB/s

1073741824 bytes (1.1 GB) copied, 8.47802 s, 127 MB/s

1073741824 bytes (1.1 GB) copied, 8.54732 s, 126 MB/s

——————————

CPU performance

——————————

268435456 bytes (268 MB) copied, 21.0134 s, 12.8 MB/s

268435456 bytes (268 MB) copied, 20.9979 s, 12.8 MB/s

268435456 bytes (268 MB) copied, 20.9207 s, 12.8 MB/s

268435456 bytes (268 MB) copied, 21.0908 s, 12.7 MB/s

268435456 bytes (268 MB) copied, 21.0547 s, 12.7 MB/s

268435456 bytes (268 MB) copied, 20.9105 s, 12.8 MB/s

268435456 bytes (268 MB) copied, 20.8068 s, 12.9 MB/s

268435456 bytes (268 MB) copied, 20.8462 s, 12.9 MB/s

268435456 bytes (268 MB) copied, 20.9656 s, 12.8 MB/s

268435456 bytes (268 MB) copied, 20.8076 s, 12.9 MB/s

Benchmark results for VM

When running the same command on a bare metal machine, performance of the file system and CPU for same amount of data showed:

——————————

FS write performance

——————————

1073741824 bytes (1.1 GB) copied, 6.63242 s, 162 MB/s

1073741824 bytes (1.1 GB) copied, 6.55013 s, 164 MB/s

1073741824 bytes (1.1 GB) copied, 6.6611 s, 161 MB/s

1073741824 bytes (1.1 GB) copied, 6.42406 s, 167 MB/s

1073741824 bytes (1.1 GB) copied, 6.88436 s, 156 MB/s

1073741824 bytes (1.1 GB) copied, 6.39726 s, 168 MB/s

1073741824 bytes (1.1 GB) copied, 6.52477 s, 165 MB/s

1073741824 bytes (1.1 GB) copied, 6.61627 s, 162 MB/s

1073741824 bytes (1.1 GB) copied, 6.95134 s, 154 MB/s

1073741824 bytes (1.1 GB) copied, 6.56434 s, 164 MB/s

——————————

Filesystem performance

——————————

268435456 bytes (268 MB) copied, 13.789 s, 19.5 MB/s

268435456 bytes (268 MB) copied, 14.1166 s, 19.0 MB/s

268435456 bytes (268 MB) copied, 13.6356 s, 19.7 MB/s

268435456 bytes (268 MB) copied, 13.9786 s, 19.2 MB/s

268435456 bytes (268 MB) copied, 13.6349 s, 19.7 MB/s

268435456 bytes (268 MB) copied, 14.397 s, 18.6 MB/s

268435456 bytes (268 MB) copied, 13.7385 s, 19.5 MB/s

268435456 bytes (268 MB) copied, 14.5623 s, 18.4 MB/s

268435456 bytes (268 MB) copied, 14.6485 s, 18.3 MB/s

268435456 bytes (268 MB) copied, 13.9463 s, 19.2 MB/s

Benchmark results for bare metal machine

In another benchmark report running the same tests, the difference in performance reached 7x-9x running Docker containers on bare metal in comparison to running the same on virtual machines.

Questions about the arrangement, coordination and management of containers ? Discover 12 Kubernetes key features here.

Run Kubernetes on Bare Metal

Another interesting performance test was done by CenturyLink running Kubernetes for cluster creation, where one cluster was made up of bare metal servers and another was made up of VMs. This test measured network latency for both types of clusters using netperf-tester. You  can see the results for both cases in the following tables:

As we can see from the table above, running Kubernetes and containers on the bare metal machines achieved significantly lower latency — around 3x lower than running Kubernetes on VMs. We can also see that in several cases the CPU utilization can be pretty high when running on VMs in comparison to bare metal.

Looking at these results, there is no doubt that if your systems are sensitive, for example, to page loading time or web service response time, running your containers on bare metal merits consideration.  

Applications and workloads that require direct access to physical hardware, such as enterprise databases and calculation-intensive applications, can greatly benefit from the performance of bare metal clouds. The results above are significant, for example, when running analytics and BI processes, where this translates into faster results and higher throughput of data processing. This also applies to machine learning (ML) algorithms that require heavy computation when fed large amounts of data to train the ML models.

Is container monitoring important? Find out here.

VMs vs. Bare Metal

The results are interesting but not really surprising. Nonetheless, assessing performance using bare metal resources should be part of the management efforts, and subject to resource cost and utilization considerations.

In addition to benefits mentioned above, VMs enable users to easily move workloads (i.e., containers) between hosts using guest images, whereas bare metal machines are more difficult to upgrade or move. A good example for that is rollback. With bare metal servers, rolling back a machine state is a challenging task. Versioning and rollback capabilities supported in modern cloud platforms, such as Amazon cloud, includes the ability to take a point-in-time snapshot of a VM periodically, and when required easily roll back to that snapshot.

Another example accepts that containers have constraints, such as the fact that official Docker installation requires Windows 10 Pro, and other Windows Server versions,  such as 2012 or even 2008, aren’t supported. This might make using bare metal servers a pain when you need to upgrade and configure the operating system.

On the other hand, single-tenant, bare metal servers can provide a better option for organizations that are restricted by compliance measures, which might require strict data security and privacy controls.

Final Note

At the end of the day, every IT organization should be able to easily mix and match a resource to a workload. But this is easier said than done. Enterprises should look for cloud solutions that support running VMs side-by-side with containers, and be able to create an hybrid cloud environment which can easily bridge the gaps between the on-premises enterprise irons and the use of virtualized, scalable public cloud environments.

Download the Ultimate Guide to deploying, managing and scaling Kubernetes

We respect your privacy

January 11, 2017

Simple Share Buttons
Simple Share Buttons