During the planned downtime that occurred on the 27th of February, we worked hard on improving our physical security and reducing packet loss. It’s now time to talk about how the physical security has been updated and what it means in terms of improvement of our services.
As you all know, apart from our uptime guarentee and a fast connection, our main focus continues to be the security of our users. We strive for the highest level of security, both physically and in terms of software.
We wrote about the importance of strong physical security and what kind of physical security we have been using since October of 2014. We are now taking the physical security one step further.
Earlier, every server in Stockholm had a USB flash drive loading the operating system into the RAM. You can see this on one of the many pictures that we uploaded in January.
Since the update on the 27th of February, our servers no longer have any kind of permanent storage installed. In other words, there are no hard drives, no USB flash drives, and no CDs in our servers. There is no storage on the server, so there's no way of storing data that can remain after a reboot. This makes our servers completely secure in the case of someone trying to tinker with them physically.
How do the servers work without storage?
In every datacenter, we have a server that we call ‘boot server’. The boot server contains all of our live images. These contain no sensitive information at all, but the partition where the live images are being stored is still encrypted using XTS-AES with 512-bit keys.
The live images are tailored versions of Debian. In order to increase the speed and security of our service, we have removed everything unnecessary and tailored the kernel to minimize the number of kernel modules. A lot of weaknesses derive from unnecessary modules that come with the default version of kernels.
The default version of Debian reaches about 1 GB. Our trimmed-down version is at 188 MB (about 210 MB with kernel and modules). In other words, we have removed about 80% of all packages, significantly minimizing the risk of vulnerabilities. This means that all SATA controllers and USB ports are inactive since our tailored version of Debian doesn’t include that functionality.
Every night, critical security updates are being run automatically. These updates are being stored in the RAM since there is no HDD/SSD. In order to improve our security, effectiveness, and performance, we continuously review our live images.
The servers are booted in PXE (Preboot Execution Environment) and download the correct live image through NFS (Network File System) from our boot server. Since the live images are being booted locally and verified through a checksum, the traffic never leaves our rack cabinets. By doing this, we are preventing all tinkering of the operating system, even if someone would be attempting to perform a MITM-attack.
When the live images have been downloaded and verified, the entire operating system is copied into the RAM memory. Everything that happens in the operating system stays in the RAM and never reaches any media that can store information after a reboot. Since we believe that it's better to be safe than sorry, we are still employing a high level of software security, even though no information can be stored.
Our servers also use the latest version of Tresor and grsecurity. Tresor stores the secret key inside the CPU register, making sure that no AES states ever touches the RAM memory. This means that it's impossible for anyone to access sensitive data in the case of a CBC (Cold Boot Attack).
Pictures of the servers
As we mentioned earlier, we uploaded pictures of our servers in January. As you can see in the pictures, we hadn’t removed the chassis from the servers. We had no hard drives in these chassis, but now we have removed the chassis and snapped a couple of new photos.
The front of the servers in Stockholm
A closer look at the front of the servers
The back of the servers in Stockholm
As you can see, there are no hard drives or USB flash drives connected to the servers. As far as we know, no other VPN provider can deliver this kind of physical security. We have put a lot of effort into ensuring such a high level of security and are extremely happy knowing that we probably have the best security among all VPN services out there.
With the combination of physical and software security and our insurance for legal expenses, it is our opinion that there is no other option than OVPN when your personal integrity is at stake.
Make sure to check out our infrastructure in Frankfurt, Germany as well, where we've deployed a blade system without any hard drives.