Can virtualization be trusted for security?
Some of our clients have started using VMWare, Xen and MS Virtual Server to provide either a contained environment for accessing the Internet from the desktop, or to isolate Internet services such as web and e-mail servers from the rest of a physical server.
The idea is that if the virtual machine gets infected with malware or infiltrated by an attacker, it’s not possible to escape to the host operating system. Getting rid of the malware and undoing damage by an attacker is as easy as reloading the VM from a known good image.
This has become a lot more widespread now that VMWare has released no-cost versions of their desktop and server products. Even Microsoft has started giving away their Virtual Server product. The problem is virtual machine products are not designed for security. They are designed for resource partitioning and running multiple OSs on one physical computer.
People simply assume that since each VM looks like separate physically hardware, it provides the same level of isolation as physical hardware. But appearances are deceiving… that’s simply not what most of these products are designed for. An attacker can easily detect they are inside a virtual machine, and with some effort could escape into the host OS or into other VMs on the same physical server.
We dealt with one organization who had constructed an entire Internet server farm on one physical server using VMWare’s ESX server product. The one physical box had been segregated into multiple guest VMs running Linux and Windows: a firewall, a database server, a web server, an e-mail server and a Java application server. The VMs were interconnected using VMWare’s virtual LAN capability.
It was a cute idea and one that would potentially save a lot of administration headaches and costs compared to setting up and maintaining four separate physical servers. The flaw was the assumption of perfect isolation of each VM from each other, and isolation from the host OS.
Using the emulated VLANs for network separation between the VMs was also a big assumption. Like virtual machines, virtual LANs are often abused for security, though they were never designed for that purpose. There are several published techniques for escaping “real” VLANs and settings the administrator of a hardware switch can use to reduce that vulnerability. Can the emulated VLANs in ESX server be hardened in the same way? Does the emulation code that creates the VLAN illusion add additional methods for escape?
Detecting the presence of most VMs is easy. VMWare Workstation and VMWare Player can be detected by looking at the name of the IDE device (”VMware Virtual IDE” or vendor string of the SCSI driver (”VMWare”). MS Virtual Server can be detected by looking for “Microsoft Corporation” in the manufacturer string of the motherboard. One generic approach to VM detection is discussed at http://invisiblethings.org/papers/redpill.html
Once an attacker determines they are inside a virtual machine, breaking out can be accomplished multiple ways. With VMWare Workstation, for example, you can do the following:
- via the “hidden” I/O port vmware uses to provide host communication such as clipboard and drive sharing (details at http://chitchat.at.infoseek.co.jp/vmware/backdoor.html)
- via known bugs, especially when Linux is the host OS (for example, http://securitytracker.com/alerts/2005/Dec/1015401.html and http://secunia.com/advisories/13871/)
The above are well known published exploits. You can be sure that many more unpublished methods to escape specific virtual machines exist and are circulating.
A knowledgable attacker can detect they are inside a virtual machine, and can probably escape it to get at the host operating system (or the linux-based “Console OS” in ESX Server). The fundamental problem is that even if the hardware virtualization of the environment was perfect, the host OS or management console must use some form of hole to communicate with the VM, such as the undocumented I/O port in Vmware workstation. Any flaw in that channel can potentially be exploited.
That means such VMs should not be used in roles such as for honeypots or separation of Internet-facing services from more sensitive ones.
Marketing of various virtualization products often imply they can be used for security, but a little digging reveals how much the vendors really stand by the claim. For example, Vmware’s ESX Server product has undergone Common Criterial evaluation, but only to the EAL 2 level of assurance for “protection against casual breach of TOE security by attackers possessing a low attack potential”.
So what about using VMs as a security container for web browsing on desktops? After all, the first pre-built virtual environment offered with VMWare Player is the “Browser Appliance”, a version of Ubuntu Linux intended specifically for that task. Surfing the web using a VM can help stop simple viruses and spyware from gaining a foothold on your “real” desktop. When the OS in the VM becomes compromised you can just reset it to a known good image. However there are a problems with that:
- Users want to share files between their real host desktop and the virtual machine (for example, when a user downloads a PDF they probably want to put it in their normal home directory, not keep it on the VM’s virtual hard drive). VMware Workstation provides “Shared Folders” for that purpose. However, any file moved from the virtual machine can contaminate the host OS. Having a writable shared folder on the host also allows malware to write files directly to the host.
- You probably won’t know when they virtual machine has been compromised until it’s too late. If you access your bank using a browser in the VM, how do you know the VM doesn’t have a keylogger installed? You would have to purposely reset the VM to the” known good” image at the beginning of each session.
- Nothing prevents a compromised VM from becoming part of a botnet or being controlled remotely to attack other computers. Most VMs also allow the virtual NIC to be placed into promiscuous mode to sniff local network traffic.
- There is evidence that malware are starting to check whether they are running inside a virtual machines and either refuse to run, or automatically attempt to escape to the host OS.
So, are virtual machines completely useless for security purposes? No. While not as trustworthy as physical hardware for containment, a VM does provide isolation against casual intrusion and the ability to quickly recover from compromise. Denial of service attacks are also mitigated since a crashed or overloaded VM can usually just be restarted without adversely affecting the host or other VMs.
As long as the limitations of the containment offered by virtualization product are recognized, they are valuable tools. Personally I would not use a VM for tasks such as a honey pot, analysing malware, or running a firewall.
On a server, I would make sure all VM on a physical server are used for tasks within the same security domain (e.g. one physical server hosting VMs for Internet services in a DMZ, another physical server hosting VMs for intranet applications, etc) and separate each physical server with a physical firewall.
If using VM on a workstation to surf web sites, a good practice is to reset the VM to the known good snapshot each time the VM is started, and to disable host interoperability features such as clipboard sharing and shared drives.
By the way, containment environments designed for security do exist. In 2001 VMware started working with the National Security Agency in the U.S to develop a virtual machine intended specifically for security containment. That project eventually resulted in “NetTop” (http://www.trustedcs.com/products/1products1_1_4.html)
NetTop is intended for classified environments where users typically have one physical workstation for each classified network they need to access. NetTop is designed to isolate each network within hardened virtual machines running on one physical workstation.
VMware’s ACE (http://www.vmware.com/products/ace/) is a specialized version of VMWare Workstation that is designed for security, though it’s designed to protect the VM from compromise from the host (i.e. to keep malware out of the VM rather than in, the opposite of what we’ve been discussing).
In the Unix world there are also “private execution environments” (OS Virtualization) such as FreeBSD Jail and Solaris Containers. For Linux, similar execution evironments can be added using either OpenVZ or Linux-Vserver. All these are designed for security containment but share the same underlying kernel OS. You cannot use them to run different OSs like in VMware, but can have reasonable assurance that attackers cannot escape.
Further reading on detecting virtual machines:http://www.phrack.org/fakes/p62/p62-0×07.txt
Update 1: Recently the SANS Handler’s Diary (recommended daily reading, by the way) discussed how more malware is detecting VMWare, and linked to a presentation on methods of avoiding detection.
Update 2: A researcher at Symantec has released a great little paper Attacks on Virtual Machine Emulators
Related posts:
- Security of virtualization
- XenServer virtualization to go free today
- CIS releases Vmware ESX security guide
Tags: Virtualization, virtualization security, vmware
4 Responses to “Can virtualization be trusted for security?”:
August 22nd, 2006 at 4:13 am
Do you have a source for your statement that Virtual LAN’s aren’t designed for security purposes? This is the first time I see that statement. I don’t know that much about virtual machines, but I know network well. And the “escape techniques” that are in the source article that supports your VLAN statement, are quite well known under networking professionals. It’s always best practice to leave the native VLAN unused, or use it for management purposes only.
Having said that, you cannot “hop VLAN’s” on a network that doesn’t use the Native VLAN, or use Cisco’s propriatary ISL instead of IEEE 802.1q to communicate between switches. Because “Native VLAN” exists on a 802.1q link only.
In my opinion, a virtual machine can be as “save” as you need it to be. You just need to understand the technique very well.
August 23rd, 2006 at 1:03 pm
Tommy:
Thanks for the comment. You’re right that a virtual machine can be “safe” for a particular purpose. Anything can be acceptable as long as you understand the issues and accept the risk. Using virtual machines all within the same security domain (to emulate all the servers housed within one DMZ, for example), isn’t much riskier than physical hardware since the assumption of a DMZ is always that the machines within it can attack each other. Spanning security domains is higher risk, since much less is known about vulnerabilities in these entirely software-based emulations than is known about physical hardware like a firewall appliance.
The point of the article is that people are assuming virtual machines are *identical* security-wise to physical machines, and that the emulated VLANs provided by products like vmware are *identical* to physical LANs or hardware VLANs. They’re not.. and shouldn’t blindly be used as drop-in substitutions for physical hardware.
Regarding VLANs, in the article I was talking about the emulated VLANs implemented within VMWare. These are software simulations of physical LANs, just like the NICs in each guest OS is a software emulation of a physical network card. I don’t think VMWare VLANs use 802.1q internally… but that’s something to look into. The point is that there are ways to escape VLANs when implemented in hardware… assuming that simulated LANS implemented entirely in software are just as strong is a mistake IMHO.
By the way, take at look at the “aims and benefits” section in the original IEEE 802.1Q spec at http://standards.ieee.org/getieee802/download/802.1Q-1998.pdf. Physical VLANs were originally designed to limit ethernet collision domains and to allow physically separate hosts to be placed logically on the same LAN. As far as I can see, security wasn’t a specific design goal. Doesn’t mean VLANs can never be used to provide traffic separation for security purposes… it depends on what you’re protecting and the level risk you’re willing to accept.
December 15th, 2006 at 2:14 pm
I would think that the level of security would vary quite a bit depending on the host (ESX, Windows, Linux). Are there really any exploits to break out of a VM guest into an ESX 3 server? Is any exploit at all likely?
Vmware says that the kernel hosting the VM guest is purely VMware. The implication is that any Linux vulnerabilities in the Service Console would not be reachable by the guest OSs.
December 18th, 2006 at 11:28 am
If someone tells you that goats are living on the moon, the onus is on them to prove it, not on you to disprove it.
The folks using virtualization to contain server instances for security purposes are assuming that VM isolation is just as robust as separation of physical servers. That’s a "goats on the moon" assumption… which is odd because not even the vendors are making that claim (read the TOE of the ESX server common criteria evaluation).
Are exploits against virtual machines likely? Of course. All software has bugs, especially complex software like virtualization products. Do I know of exploits specifically for ESX server? Not at the moment, but then I’m not connected to any of the underground black markets where such exploits are most likely to show up first.