“What would happen if a commercial cloud could guarantee the capture of malware, no matter how expensive or exotic, in volatile memory?”
Microsoft has built an absolute behemoth of a cloud virtual machine (VM) security tool from scratch in Rust* called Project Freta, and it is rather exciting.
The stated aim: automating cloud-based Linux VM forensics at staggering scale, e.g. for enterprises spinning up thousands of virtual machines in the cloud. (Freta automatically supports 4,000 Linux kernel versions).
In short, the service (classed as a technology demonstration and currently available for free) allows “full system memory inspection” of live Linux systems to take place without attackers knowing, so that previously unseen malware and rootkits from sophisticated attackers can sniffed out.
As one earlier adopter in the aerospace and defence sector told Computer Business Review: “The existing method for detecting malware in a running Linux virtual machine involves VM introspection, where the virtualisation host (Azure/Hyper-v, ESXi, KVM, etc) tracks system events happening inside of the guest virtual machine. Unfortunately, that kind of live-tracking can be detected by sophisticated malware using timing or monitoring the cache.
“So the Project Freta method is to take a whole-system snapshot, and analyse that frozen image offline. Any running malware would be frozen in the snapshot and Freta can run any kind of analysis it wants to on it.” (Users can pull analysis data via REST or Python API, or see it in a portal).
Mike Walker of Microsoft Research’s “NExT” Security Ventures team says the tool was built to work at a huge scale for organisations with large cloud workloads. As he puts it: “The ability to programmatically audit 100,000 machines in a short, cost-bounded timeframe was a minimum requirement.
“This meant architecting from the beginning for batch processing in the cloud… [including for] VMs with 100+ gigabytes of RAM.”
Project Freta: Why Should I Care?
As Walker notes: “Snapshot-based memory forensics is a field now in its second decade, [but] no commercial cloud has yet provided customers the ability to perform full memory audits of thousands of VMs without intrusive capture mechanisms and a priori forensic readiness.”
Using Freta, his team claims that Hyper-V checkpoint files grabbed from thousands of VMs can be searched for “everything from cryptominers to advanced kernel rootkits… transitioning [cloud users] to automated malware discovery built into the bedrock of a commercial cloud.”
There’s nothing comparable out there that we have seen.
The behind-the-scenes engineering that went into the tool has clearly been colossal.** Azure users and those who trust Microsoft implicitly may feel comfortable taking Freta for a spin. It’s also available for non-Azure users. Whether they’d want to try it out is an open question, particularly since the analysis engine itself is something of a black box at the moment.
Thank you. Just started playing with Freta and the demo images. This is SERIOUSLY cool and powerful. I see that you report UNIX sockets, is there any interest in reporting other types of IPC like Netlink or shared memory?
— Josh Avraham (@josh_avraham) July 6, 2020
As one user told us: “That’s a big concern certainly, since the data you’re uploading to Freta could contain passwords, customer data, etc. Non-Azure customers would definitely avoid uploading their data to a black box.
“If they allowed us to run the analysis ourselves without uploading the data, it would decrease the risk of giving Microsoft potentially sensitive data.”
Microsoft’s rhetorical question, meanwhile: “What would happen if a commercial cloud could guarantee the capture of malware, no matter how expensive or exotic, in volatile memory?” It’s answer: expensive reinvention cycles would render the cloud “an unsuitable place for cyberattacks.”
It’s a big dream, but it’s also a big and clever project that could prove invaluable in shining some sunlight on sophisticated threats. Given its invisibility to attackers (or any actor other sitting in the VM), and its powerful ability to view everything happening across thousands of VMs, Azure users will no doubt also be wanting clear reassurances that it can’t be abused.
* As Walker puts it in a Microsoft blog: “We knew that any system designed to hunt for tools fielded by the most well-resourced attackers would itself become a target. Given the history and preponderance of memory-corruption exploits, we made the choice as a team to embrace Rust at the beginning, architecting the entire capability from scratch in Rust from line one and building upon no existing software. This has yielded a high-performance analysis engine for memory images of arbitrary size that also has memory safety properties”.
**“Many existing forensic approaches execute clarifying instructions on the guest, such as copying KASLR [Editor’s note: our link] keys. Unfortunately, these instructions can tip off malware to a capture event. The requirement not to interact with the target OS, needed to ensure the element of surprise, mandated a forensic imaging technology that was completely ‘blind.’ As a consequence, memory scrambled by security mechanisms such as ASLR needed to be decoded without keys or context. This task is complex enough for one operating system, and it’s a templating nightmare to support any operating system.