News
How Nitro System Became the Key to AWS' AI Security Efforts
Amazon Web Services is taking steps to position its Nitro hypervisor, which already underpins its Elastic Compute Cloud (EC2) solution, as the center of its AI security strategy.
"As customers move quickly to implement generative AI in their organizations, you need to know that your data is being handled securely across the AI lifecycle, including data preparation, training, and inferencing," AWS said in a blog post on Tuesday. "The security of model weights -- the parameters that a model learns during training that are critical for its ability to make predictions -- is paramount to protecting your data and maintaining model integrity."
The Nitro System, AWS went on to explain, is the cornerstone of AWS' efforts to provide customers with that security.
AWS took years to develop Nitro before fully rolling it out to EC2 in 2017 with the launch of the C5 instance family. Then-chief of AWS Global Infrastructure, Peter DeSantis, described the Nitro project's goal as "[making] the EC2 instance indistinguishable from bare metal."
In terms of security, Nitro was designed to reduce EC2 users' attack surfaces by offloading virtualization resources onto dedicated hardware and software. Notably, according to the product page, "Nitro System's security model is locked down and prohibits administrative access, eliminating the possibility of human error and tampering."
It's this point that AWS said makes Nitro the ideal infrastructure for enforcing security around generative AI data.
"By design, there is no mechanism for any Amazon employee to access a Nitro EC2 instance that customers use to run their workloads, or to access data that customers send to a machine learning (ML) accelerator or GPU," the company said. "This protection applies to all Nitro-based instances, including instances with [machine learning] accelerators like AWS Inferentia and AWS Trainium, and instances with GPUs like P4, P5, G5, and G6."
The blog listed AWS' three goals regarding securing AI workloads for its customers. They are:
- Complete isolation of the AI data from the infrastructure operator: The infrastructure operator must have no ability to access customer content and AI data, such as AI model weights and data processed with models.
- Ability for customers to isolate AI data from themselves: The infrastructure must provide a mechanism to allow model weights and data to be loaded into hardware, while remaining isolated and inaccessible from customers’ own users and software.
- Protected infrastructure communications: The communication between devices in the ML accelerator infrastructure must be protected. All externally accessible links between the devices must be encrypted.
As mentioned, Nitro prevents AWS employees from accessing customer instances running on Nitro, so the system inherently meets the first of these goals.
It also meets the second by way of AWS' native key management service (KMS) and the Nitro Enclaves feature. Combined, these capabilities enable users to "encrypt your sensitive AI data using keys that you own and control, store that data in a location of your choice, and securely transfer the encrypted data to an isolated compute environment for inferencing," according to AWS. "Throughout this entire process, the sensitive AI data is encrypted and isolated from your own users and software on your EC2 instance, and AWS operators cannot access this data."
As for the third goal, AWS announced that it plans to extend Nitro's encryption capabilities beyond CPUs to include AI accelerators and GPUs. It will do this by integrating Nitro with the newly announced Blackwell processors from Nvidia. The two companies are currently co-developing "a joint solution...including NVIDIA's new NVIDIA Blackwell GPU 21 platform, which couples NVIDIA's GB200 NVL72 solution with the Nitro System and EFA technologies to provide an industry-leading solution for securely building and deploying next-generation generative AI applications."
The second generation of AWS' Trainium chip will also support this "end-to-end encrypted flow," the company said.
AWS pointed out that Nitro's security architecture has been vetted by researchers from information security firm NCC Group. More information about Nitro's security design is available here.