While preparing for the VCAP Deploy exam, I came across an insightful on-demand course from Broadcom: VMware vSphere: Advanced Administration [V8] – On Demand.
One of the topics covered was CPU vulnerabilities related to the L1TF vulnerability, Microarchitectural Data Sampling (MDS), including CVE-2018-12126, CVE-2018-12127, CVE-2018-12130, and CVE-2019-11091, along with VMware Security Advisory VMSA-2019-0008.
This is a complex subject, so I highly recommend reading this excellent article by Bob Plankers at VMware. My post aims to reinforce that information and share my own insights while keeping it as brief and to the point as possible.
What is a CPU Scheduler?
The vSphere CPU scheduler in ESXi manages how virtual machines access the processors. Its goal is to maximize CPU utilization while ensuring fair resource distribution across all VMs.
L1TF vulnerability exposed a risk where Intel CPUs could leak information between two threads running on the same core. To mitigate this, the original ESXi Side-Channel Aware Scheduler v1 (SCAv1) disabled Intel Hyper-Threading, effectively closing the vulnerability but at the cost of reduced CPU capacity.
For example, a hosting provider with multiple customers sharing the same physical hardware, two VMs running on the same physical core simultaneously could potentially access and discover each other’s data – this is the risk VMware aim to mitigate using SCAV1 or SCAV2.
Similarly, the MDS vulnerability also enables data leakage between processes, making SCAv1 a strong mitigation option. However, in certain scenarios where security requirements allow, you might still want to leverage Hyper-Threading within a virtual machine. This is where SCAv2 comes in, offering a balance between security and performance by allowing Hyper-Threading while still applying protections.
How to choose the CPU Scheduler?
There are three schedulers to choose from in ESXi 6.7 Update 2: the default, SCAv1, and SCAv2.
Default Scheduler
The default scheduler has no inherent security awareness but is the most performant. It should be assumed that all workloads running on a host with the default scheduler can see each other’s data.
VMs are distributed across CPU cores dynamically. They can run on the same core, leading to fast context switching and less isolation. However, this means VMs can interfere with each other, and data could leak between them.
Side-Channel Aware Scheduler v1 (SCAv1)
SCAv1 implements per-process protections which assist with L1TF and MDS and is the most secure but can be the slowest.
Disables Hyper-Threading (HT) to ensure that VM processes don’t share threads on the same physical core. This provides better isolation at the cost of performance.
Side-Channel Aware Scheduler v2 (SCAv2)
SCAv2 implements per-VM protections but needs considerations for the risk involved in using it.
Hyper-Threading is enabled, but each physical core’s threads are dedicated to a single VM at a time. This ensures that threads used by one VM remain isolated, preventing data leakage between VMs. In other words, even though a physical core has multiple threads, they won’t be shared by two VMs simultaneously, allowing for better security while still optimizing core usage.
To implement one of the options, simply edit the host’s advanced system settings. Change the values of hyperThreadingMitigation and hyperThreadingMitigationIntraVM to true or false depending on whether you want SCAv1 or SCAv2 (refer to the settings above for the exact values for each option).
The CPU scheduler is a fascinating feature, and I personally find it very interesting. If you want to explore it in more detail and deepen your knowledge, I highly recommend starting with the article mentioned at the beginning of this blog post.