Decrypting the LVM-Cryptsetup

Ok, once i have posted the Installing BackBox Linux with LVM and nearly complete encryption, i felt that it is not complete/sufficient. The desire to dig deeper into it and learn what happens behind the scenes is there...so lets just do it :)

Note: I can't give you all the backgrounds and can't explain everything - this would (and actually does) fill books. But a little bit of information will be here.

The whole system encryption

The disk encryption article won't encrypt your whole disk. /boot is still unencrypted. Ok, its just the kernel with ramfs (mini-root). But it is unecrypted. Ubuntu and grub2 doesn't offer you the ability to boot an encrypted partiton directly at the moment. There are programs (mostly windows based) that will offer you the complete disk encryption but i have never seen nor used one.

Side note - yes, you can encrypt also the kernel and all other things from /boot with a custom built grub2. But that is not easy to setup, i may write an article someday. In the meantime you can look at this nice article: full system encryption for linux.

Another alternative for full disk encryption is possible with hardware encryption (adapters) or if you separate your /boot and the boot-loader to an thumb-drive or similar. But i won't go through that now.
Live with it, /boot stays unencrypted. But it's nothing in there that you can't find on the Iso and repositories coming with the distribution. Sure someone could compromise/manipulate the kernel to put in a backdoor on your system. For me this was no decision for disk encryption. But it is indeed a real security thread that should be kept in mind.

Data manipulation brings me to the next point: Encryption offers you another benefit. You have a data integrity check for free. You encrypted data can not be manipulated from the outside. If someone boots the system with an external medium (like CD, USB stick etc.) no access to the data is possible. BUT if you are running the system and someone breaks in (through a network exploit, no screen lock and so on) you data is available (and can be manipulated/deleted) like on a normal filesystem. So if you want to keep real sensitive data on your disk like credit card numbers or your plans to rule the world, you should encrypt it directly with a tool like ccrypt, a suite like truecrpyt or any other program which comes to your mind.

Encrypted disks also won't help you with not-encrypted network traffic, data copied to another medium and so on. It is strictly bound to the devices you encrypted, nothing else. But it runs transparent for you on your system after you entered the correct passphrase. Good enough for most people :)

So lets explore what we have done

fdisk

fdisk /dev/sda


Used to create and manipulate the partition table, in our case on disk sda. It works directly at the physical layer of a disk, so it is a program that should be known well and used with care. You can also use parted or gparted (the graphical interface) to interact with the disk layout, but i got used to fdisk.
I decided to use just 2 primary partitions, sda1 and sda2. This is the easiest way and should be used in this layout because after the creation of sda2 there is no space to create another partition. When you thinking about a more complex example (which is usually not necessary when working with LVM, see below) you will have to keep in mind that a physical disk can only have 4 primary partitions or 3 primary and then one extended which then is divided into logical partitions (starting from sda5). But for todays needs, complex physical partitioning is really ancient.

cryptsetup

It's used to create and manage encrypted partitions - period. Or not?
The first step to create the encryption actually was

cryptsetup luksFormat /dev/sda2

So what we have here

• cryptsetup - well, thats easy, the command actually doing somethings with encryption
• luksFormat - this means basically 2 "operations". luks for handling LUKS (Linux Unified Key Setup, an encryption standard under Linux) and Format, which doesn't format the device in the common sense. It doesn't even delete anything on that partition except a small part to mark it as a LUKS partition and creates key-slots. From the man-Page:
  initializes  a  LUKS  partition  and  sets  the initial key, either via prompting or via <key file>.
  So if you want to really blank the device (maybe you had confidential data on it before) you should wipe it with either a program or just do a 
  dd if=/dev/urandom of=/dev/sda2 bs=4096
  The latter is NOT compliant to any data erasing standard but should be sufficient enough for most people. Anyway it will take a long time to complete.
• /dev/sda2 is the device being initialized.

So the command runs and you will be asked to type in an uppercase YES and a passphrase. You could also support a key file (where you can use e.g. a 2048 byte, random generated key which is currently not breakable in a human measurable amount of time) which is stored on a thumb drive. Or if you have more encrypted partitions with different keys you can encrypt one partition with a passphrase and store the keyfiles on it to decrypt the other partitions. Tons of possibilities.
You have up to 8 keyslots - so you can use different mechanisms to unlock the device later (keyfiles or passphrases). The more keys you assign to the device, the more likely it is that someone may gather one of those keys (theoretically).

Then the second step

cryptsetup luksOpen /dev/sda2 crypt

opens the device. But why open?

There is a device /dev/sda2 which supports encryption but the kernel doesn't directly know how to handle it. You shouldn't put a filesystem on it because then the encryption would be lost. You need an additional layer between the device and the kernel. This is done with luksOpen. It creates a mapper device /dev/mapper/crypt (or any other name you give this after the device in the command above) which then can be used to  put a filesystem or something else on it. The informations written to that device will be handled by the Device Mapper and then dm_crypt (for the encryption) and the encryption related kernel modules (aes_generic, aes_x86_64, crypt and sha256_generic since the default encryption is aes-cbc-essiv:sha256).

If you want to learn more about the possible encryptions i suggest you read the homepage of cryptsetup and some wikipedia. This is the real hard/mathematical/technical stuff. Looking at the homepage you will also see which encryption has which performance impact on your system/disk I/O. This should be also considered...the stronger the encryption is, the slower your write and read speed will be. Remember, with great encryption comes great performance decrease (taken from Spidercrypt Part I). Honestly, if you really need top level encryption you should go and buy hardware encryption devices. This will make your life much easier and your system faster (and you can encrypt your whole disk). But be sure you find one that is compatible with linux...

cryptsetup offers some additional actions which should be looked up in the manual. Most of them (like luksClose) are straightforward. Be especially careful with the header-dump actions since they will give out informations which can be used to directly access the device without knowledge of the key. But those informations can also help you to recover an accidential overwritten partition information (maybe you did an pvcreate on the device /dev/sda2 before using luksOpen).

LVM

LVM is great. It offers a lot of flexibility, especially with grow-and shrinkable filesystems like ext[234] and others. But LVM is also an additional layer. This is used to avoid the physical limitations with disks. You can only have one or more partitions per disk and only one filesystem on one partition (well, this restriction is logical and applies to LVM too). So if you have a couple of disks with 500GB and you need a filesystem with 1TB you can't use traditional partitioning, you are limited to the physical disk size. In some OSes you also have a limitation of partitions (slices in Solaris) per physical disks.
With LVM you can combine one or more partitions to one Volume group. So lets say, you have 3 disks sda, sdb and sdc, all 100GB. You use (traditionally) sda as a system partition with 50 GB for /, 5GB Swap and the rest for /home. sdb and sdc are data disks for your large Volume /data. So you combine sdb and sdc into one VG datavg and create a logical volume lv_data (like a partition) onto that VG. Names should be descriptive but don't have to be. If you don't provide a name during LV creation it will be names lvol0, lvol1 and so on.
If you have 2 physical disks as one volumegroup you can also stripe logical volumes onto it - this will increase the performance. Also you could mirror it and can avoid using mdadm. But this is beyond the scope of this writing. Also keep in mind that you can mirror/stripe on the same disk on different partitons (which is useless for the idea itself but can be used to test).
You can add a lot VGs with different physical disks/partitions and on those VGs you can create a lot of LVs.

Note: All of the commands i used can be also used with the high level command lvm and then the subcommands. So the man page for lvm is a great starting point to find out what it will offer to you.

So back the tutorial

pvcreate /dev/mapper/crypt

This create a physical volume on /dev/mapper/crypt. Nice, huh? You could say this by looking at the command. Well, basically it will mark /dev/mapper/crypt as a physical volume to use with LVM. There is not much done here, just the headers for LVM are written to the (in our case) encrypted device. Without encryption you would need to use /dev/sda2.

vgcreate cryptvg /dev/mapper/crypt

This creates the Volume group cryptvg with the physical volume /dev/mapper/crypt. There could be more than one physical volume (e.g. /dev/mapper/crypt1 /dev/mapper/crypt2 or /dev/sda2 /dev/sdb2 without encryption and so on).
In our case we used a cryptvolume, so any data written on that device will be handled through dm_crypt with the appropriate encryption and written onto the physical partition /dev/sda2. So if you don't have the key to unlock the device /dev/sda2 you won't even be able to see what kind of data is on it (well, some people may be able to, but i am not). The physical disk is "splitted" into physical extents (PE), which are small parts of data, usually 4MB. So the smallest unit that can be allocated for a logical volume is 4MB=1PE. But all your fears come true, a logical Volume is almost ever a multiple of this ;) Now we have some kind of container with the ability to store small chunks, to combine them and so on. Which brings us to the next step

lvcreate -L 5G -n rootlv cryptvg

So, now the logical volume is created by allocation those 4MB chunks to build a large "partition". The name is rootlv (for root logical volume - could also be lv_root or just root) and is created on the volume group cryptvg. Wow, life can be so easy. Actually linux don't care much about the partition type, windows is here more strict. For linux the type of "filesystem" is looked up when the filesystem is used. So you can easily put the partition for swap as a logical volume on that volume group. Or another LV for an ext2 filesystem, one for JFS and so on.
For the size you can use -L and give it "human readable" parameters like 5G, 10M or even 1T/P/E (Terra-/Peta-/Exabyte if you have the space). Or you can use -l (lowercase L) to size it from a physical extent perspective (so 100 extents means 400MB).
Logical volumes can be accessed through 3 ways:

• /dev/mapper/vgname-lvname which uses the device mapper notation
• /dev/vgname/lvname which is more like a direct association to the device
• /dev/dm-1 which is the LVM created block device, allocated through a major and a minor number which is noted behind the dm-. The 2 cases above are just links to this one.

LVM offers a lot of functionality besides what i showed you. You can resize a logical volume, you can add or delete physical volumes from the volume group and so on. One nice feature is snapshots volumes, which will keep the same state of the filesystem from the time when you took the snapshot. So you can backup the snapshot LV in a consitent state and work normal with the system at the same time. Or you can take a snapshot, upgrade the system and if anything goes wrong, you can revert the volume back to the time before the upgrade/snapshot state (remember /dev/sda1!).

The created LVs could now be used as a "raw device" which means, that the application (usually databases) will take care of the allocation of data in it. Or, suited for our purpose of installing an OS to it, put a filesystem on top of it. So here we go

Filesystems

mkfs.ext4 /dev/cryptvg/rootlv

Not much to say here. An ext4 filesystem gets created, using the normal attributes. I used /dev/vgname/lvname, but /dev/mapper/vgname-lvname will work too. You can also use /dev/dm-1 but this is not really straightforward.
The filesystem is most important from the user perspective. It is the thing that gives programs the ability to write, read and manipulate data. I forgot to test it (and i am a bit tired to do now) if you can install BB without creating those filesystems. Actually i got them and this was my focus. But the installer should recognize the volumegroup and the logical volumes itself, so the creation of the filesystems should be optional. Maybe you want to test this, i want to go to bed now :)

Is that all real?


Also i wanted to give you a good oversight about what happened and what means what (for now until the installation) this  whole article is still not the reality and some informations maybe even wrong (sorry for that). But it should give enough informations to understand what will be done to your poor disks when using LVM and cryptsetup :) I touched some deeper internals but if you want to really understand whats going on, you will have to get the informations yourself. Starting from physical disk design (heads, sectors and cylinders) over to Bus protocols like SATA or IDE up to the kernel into the userspace. That is a nice travel...which fills thousands of pages i will never write. Its just 1s and 0s - but they change a lot during their trip to your monitor.


So have fun exploring.