END-TO-END ENCRYPTION, THE GOLD STANDARD

End-to-End Encryption is an implementation of encryption. It is a method of encrypting data at the absolute soonest possible point in time and decrypting the data only at the very last possible moment. The encryption algorithm used when encrypting/decrypting is irrelevant to the definition of End-to-End Encryption. Using End-to-End Encryption means that the data is encrypted as it travels through the internet (in transit), to application servers, into databases (at rest), and out of databases, to be processed. This insures that the data is encrypted and the plain-text version is not available to DBA’s, Developers, Network Engineers, Managers, Unix/Windows Administrators, Help Desk Personnel, Storage Administrators, Security Personnel, etc. End-to-End Encryption is considered the best and most secure implementation of encryption.

A very simple example… data is encrypted by the sender as soon as the message is composed. It is encrypted as it flows to an email application on the sender’s computer. It is encrypted as it flows through the sender’s router, through the internet (and all that that entails), to the recipient’s router. It’s encrypted as it rests within the recipient’s email application. It is only decrypted just prior to the recipient reading the message.

Here’s an example that one may find in an enterprise… A credit card number (let’s call it a PAN for Primary Account Number) is encrypted by javascript in the browser just after the Submit button is pressed. The cyphertext (encrypted data) is http posted to an app server, and the app server updates a column in a database. The PAN is encrypted ASAP in this example.

To use the PAN in the above example, the cyphertext is copied from the column in the database to a server that has the single purpose of decrypting the cyphertext just prior to sending it to the credit card processor through a leased line connected directly to that server. The PAN is decrypted at the last possible moment.

Or is it? It would be better if the credit card processor would decrypt the PAN just prior to posting the charge on their system. That would be ideal, but in 2015… in the era of vintage 1928 mag stripes still on our credit cards, the credit card processors are accepting all kinds of risk paid for by their 1.5-4% fees. The processors simply will not undertake the extra effort to do decryption, manage keys, etc. They receive credit cards that are only encrypted via TLS or SSL. TLS/SSL encryption is to encrypt data ‘in transit’. It does not encrypt ‘data at rest’ like End-to-End encryption does.

Note in our enterprise example, with the given situation, data is decrypted at the last possible moment. It’s not possible to decrypt on the credit card processor’s system, so it’s decrypted on a dedicated server, just prior to sending to the processor.

End-to-End encryption is considered the best encryption because the data is encrypted while in transit, and at rest, from the moment it is sent, to the moment it is received. It provides better protection than all the other encryption methods including the ‘at rest’ database specific methods (ie, columnar encryption), ‘in transit’ network specific methods (ie, TLS/SSL), etc.

Troy Frericks.
blog 27-Dec-2015
=
Copyright 2015 by Troy Frericks, http://dba.frericks.us/.
#

IS TDE WORTH THE EFFORT

Transparent Data Encryption (TDE) is an encryption method that encrypts database containers (usually a file) of Oracle or SQL Server databases. Containers are “data at rest” (ie, the data is not moving from one place to another). Because database backup programs copy blocks from the (encrypted) database container, TDE causes backups to be encrypted. This is important because backups are frequently moved off-site. If a backup tape/disk were to be lost, any attempt to restore the database would require the keys/certificates that were used to encrypt the data. Without them, the database would not be usable. It’d look like random bits & bites. The engine probably would not even mount the container.

Another feature of TDE is that if a disk becomes unusable, it can simply be discarded. There is no need to destroy the disk if it only contained database containers as any data that might later be recovered would be encrypted.

TDE is “transparent” because it does not change the way the database is used by applications. A SELECT will return data just as it would if TDE were not implemented. Everything works the same. This means that, with TDE, confidential information can be queried (ie, by a DBA), but could not be obtained if a backup tape were restored to another system (assuming it did not have copies of the proper keys & certificates).

When evaluating how to protect confidential information, one needs to carefully consider the goals of the encryption. For example, if you want to protect against this situation: “If an intruder circumvents other security controls and gains access to encrypted data, without the proper cryptography keys, the data is unreadable and unusable to that person” [PCI 3.1, Summary of Section 3], maybe End to End Public Key cryptography should be used (in addition to, or) rather than TDE. Another option might be columnar encryption. Judging by the chatter on the internet, a PCI audit has been blessed because the auditors are simply unaware of what TDE really is. I’ll talk about columnar and End to End Public Key encryption in the near future.

Is TDE worth the Effort? I assert not. Here’s why. If the disk fails, simply destroy it by dropping it on a concrete floor until you hear little pieces of what sounds like broken glass rattling around inside. If you are concerned about backups, simply encrypt the backups as they are taken. Adding a script to do that is much simpler than the hassles of setting up, maintaining, and using TDE.

Setting up TDE involves creating certificates & keys, and understanding how each works.

Maintaining TDE involves things like regularly backing up those certificates & keys and storing them separately from your normal backups (in PCI environments, implementing proper key management).

So, what is involved with USING TDE? If you’ve implemented TDE, the database just works, correct? Yes, that’s what’s cool about TDE… But, suppose you want to perform DBA level tasks like coping a (sanitized) database from production (with TDE) to development?

The process should be: restore the database to production server with a new name, sanitize the new database, back it up, restore it to development. But, with TDE on the source database, the restore will not work unless you have the keys available. I flat-out could not get it to work. Here’s a second opinion. See the bottom of the page, section “Known Issue with TDE”: second opinion.

I ended up having to move keys to the destination system in order to restore the non-encrypted database backup. It’s simply not worth the hassle for zero benefit (other than to say that your database is encrypted, and maybe snow your PCI auditor into believing your data is secure against commonly known attacks, which it’s not. I’ll blog about that next month).

On the other hand, if TDE were not involved, but the backup is explicitly encrypted, one could specify the key when restoring the copy to production, and skip the encryption when backing up the copy… without the possibility of leaving TDE keys behind on development as no keys would be required to restore the unencrypted backup.

Because the problems that are solved with TDE are so easily solved by other means, and because of the problems CAUSED by TDE, I discourage its use.

Troy Frericks.
blog 13-Nov-2015
updated 17-Feb-2016, 7-Mar-2016
=
Copyright 2015 by Troy Frericks, http://dba.frericks.us/.
#