Blog
As you may know, we've recently released Constellation, the first confidential Kubernetes engine, as open source. Constellation keeps all data always encrypted --- even at runtime in memory --- and shields entire clusters from the infrastructure. Thereby, it ensures that no one, not even cloud admins, hackers or foreign governments, can get access to your data.
Naturally, increased security with additional layers of encryption comes with lower performance. So, how is the performance of Constellation? In our analysis, we first look at the general impact of using runtime memory encryption with Confidential Virtual Machines (CVMs) vs. standard VMs. Second, we benchmark Constellation against non-confidential managed Kubernetes offerings Azure Kubernetes Service (AKS) and Google Kubernetes Engine (GKE).
To ensure confidentiality, the nodes in a Constellation cluster run inside CVMs. This means that the performance of the CVMs affects the performance of Constellation. Based on a performance analysis conducted by Azure and AMD, we can say that the performance is slightly lower than normal VMs. However, this highly depends on the application you're running on the CVMs. Depending on how CPU-heavy an application is, a performance overhead of 2% to 8% is possible. For most microservice applications, this is expected stay on the lower end and with newer generations of confidential computing hardware, the overhead is expected to decrease further over time.
Besides runtime memory encryption, Constellation comes with numerous other security features, like network encryption and adding encrypted persistent storage, node attestation/verification, and much more (discover all the security benefits here.) Since it's nearly impossible to benchmark on a feature level, we benchmarked the overall performance of Constellation using K-Bench. K-Bench is a configurable framework to measure Kubernetes clusters in terms of storage input/output, network performance, and creating/scaling resources.
Below you can find different graphs that compare the performance between Constellation and other non-confidential managed Kubernetes offerings, such as the Azure Kubernetes Service (AKS) and the Google Kubernetes Engine (GKE).
Kubernetes API
The Kubernetes API is the front-end of the Kubernetes control plane, which is used to modify the cluster's state. It's critical that the delay before a transfer of data begins to follow instructions, is as short as possible. Therefore, we tested it with K-Bench, which calls the API to create, update, and delete cluster resources.
As you can see in the three graphs below, which show a comparison of the latencies for pods, services and deployments, Constellation performs overall faster than the other engines. The only exceptions are for the pod update call, for scaling deployments on GKE and creating deployments on AKS deployments on GKE, and creating deployments on AKS.
API latency for Pods:
API latencies for services:
API latencies for deployments:
Network
There are two main indicators for network performance: intra-node and inter-node transmission speed. Intra-node refers to the communication that happens between pods running on the same node, while inter-node indicates the communication between different Kubernetes nodes.
The graphs below compare the throughput in megabits per second. As you can see, Constellation provides a fast networking speed, and where the speed is slower, it can largely be attributed to Constellation's network encryption that protects data-in-transit.
Storage Input/Output
Azure and GCP offer persistent storage for their Kubernetes services AKS and GKE via the Container Storage Interface (CSI). Constellation also provides persistent storage on Azure and GCP that's encrypted on the CSI layer. As the last graph, also the following graph shows the throughput in megabits per second, and Constellation on GCP speed is similar in all scenarios. However, the performance of Constellation on Azure and AKS partially differs. In read-write mixes, Constellation on Azure outperforms AKS.
As mentioned in the introduction, security through encryption and performance are a general trade-off. However, the performance benchmarks highlight that Constellation outperforms other Kubernetes engines in API latency, and it performs at the same level regarding networking and storage throughput. Where it has slightly lower performance, it's mainly because Constellation automatically encrypts all data that is sent over the network or written to storage, which prevents both hackers and the cloud admins from accessing it. Considering the greatly minimized attack surface and security gains, the performance impact, if any, should be acceptable for most workloads.
Constellation is also easy to use, with just a few commands on your CLI, you are ready to embark on your confidential journey. As we just released Constellation 2.1.0, you can now run Constellation on single hosts, and you won't need a cloud subscription to test it!
Check out the repo here!
For a more detailed review on how we performed these benchmarks, check out the documentation.
