Saturday, February 18, 2012

Upcoming stuff

So, the next posts i plan to do is really the video of the crypt-install. I have problems finding (and using) a good video editor under linux. Maybe i am doing something really wrong. Besides it takes its time that i don't have too much. But i hope i will finish it until the end of the month.
If this goes well, i will do another vid playing around with LVM, i need to make up my mind what to show and how.
The decrypt post will have a follow up. Some things still need to be explained, especially what happened after the installation completed.

If i have a way to make videos i think i will have a lot more stuff to show. Sometimes a videos is better than 1000 words :)

Keep your heads clear.

Wednesday, February 15, 2012

Decrypting the LVM-Cryptsetup

Ok, once i have posted the Installing BackBox Linux with LVM and nearly complete encryption, i felt that it is not complete/sufficient. The desire to dig deeper into it and learn what happens behind the scenes is there...so lets just do it :)

Note: I can't give you all the backgrounds and can't explain everything - this would (and actually does) fill books. But a little bit of information will be here.

The whole system encryption

The disk encryption article won't encrypt your whole disk. /boot is still unencrypted. Ok, its just the kernel with ramfs (mini-root). But it is unecrypted. Ubuntu and grub2 doesn't offer you the ability to boot an encrypted partiton directly at the moment. There are programs (mostly windows based) that will offer you the complete disk encryption but i have never seen nor used one.

Side note - yes, you can encrypt also the kernel and all other things from /boot with a custom built grub2. But that is not easy to setup, i may write an article someday. In the meantime you can look at this nice article: full system encryption for linux.

Another alternative for full disk encryption is possible with hardware encryption (adapters) or if you separate your /boot and the boot-loader to an thumb-drive or similar. But i won't go through that now.
Live with it, /boot stays unencrypted. But it's nothing in there that you can't find on the Iso and repositories coming with the distribution. Sure someone could compromise/manipulate the kernel to put in a backdoor on your system. For me this was no decision for disk encryption. But it is indeed a real security thread that should be kept in mind.

Data manipulation brings me to the next point: Encryption offers you another benefit. You have a data integrity check for free. You encrypted data can not be manipulated from the outside. If someone boots the system with an external medium (like CD, USB stick etc.) no access to the data is possible. BUT if you are running the system and someone breaks in (through a network exploit, no screen lock and so on) you data is available (and can be manipulated/deleted) like on a normal filesystem. So if you want to keep real sensitive data on your disk like credit card numbers or your plans to rule the world, you should encrypt it directly with a tool like ccrypt, a suite like truecrpyt or any other program which comes to your mind.

Encrypted disks also won't help you with not-encrypted network traffic, data copied to another medium and so on. It is strictly bound to the devices you encrypted, nothing else. But it runs transparent for you on your system after you entered the correct passphrase. Good enough for most people :)

So lets explore what we have done

fdisk

fdisk /dev/sda


Used to create and manipulate the partition table, in our case on disk sda. It works directly at the physical layer of a disk, so it is a program that should be known well and used with care. You can also use parted or gparted (the graphical interface) to interact with the disk layout, but i got used to fdisk.
I decided to use just 2 primary partitions, sda1 and sda2. This is the easiest way and should be used in this layout because after the creation of sda2 there is no space to create another partition. When you thinking about a more complex example (which is usually not necessary when working with LVM, see below) you will have to keep in mind that a physical disk can only have 4 primary partitions or 3 primary and then one extended which then is divided into logical partitions (starting from sda5). But for todays needs, complex physical partitioning is really ancient.

cryptsetup

It's used to create and manage encrypted partitions - period. Or not?
The first step to create the encryption actually was

cryptsetup luksFormat /dev/sda2

So what we have here

  • cryptsetup - well, thats easy, the command actually doing somethings with encryption
  • luksFormat - this means basically 2 "operations". luks for handling LUKS (Linux Unified Key Setup, an encryption standard under Linux) and Format, which doesn't format the device in the common sense. It doesn't even delete anything on that partition except a small part to mark it as a LUKS partition and creates key-slots. From the man-Page:
    initializes  a  LUKS  partition  and  sets  the initial key, either via prompting or via <key file>.
    So if you want to really blank the device (maybe you had confidential data on it before) you should wipe it with either a program or just do a 
    dd if=/dev/urandom of=/dev/sda2 bs=4096
    The latter is NOT compliant to any data erasing standard but should be sufficient enough for most people. Anyway it will take a long time to complete.
  • /dev/sda2 is the device being initialized.
So the command runs and you will be asked to type in an uppercase YES and a passphrase. You could also support a key file (where you can use e.g. a 2048 byte, random generated key which is currently not breakable in a human measurable amount of time) which is stored on a thumb drive. Or if you have more encrypted partitions with different keys you can encrypt one partition with a passphrase and store the keyfiles on it to decrypt the other partitions. Tons of possibilities.
You have up to 8 keyslots - so you can use different mechanisms to unlock the device later (keyfiles or passphrases). The more keys you assign to the device, the more likely it is that someone may gather one of those keys (theoretically).

Then the second step

cryptsetup luksOpen /dev/sda2 crypt

opens the device. But why open?
There is a device /dev/sda2 which supports encryption but the kernel doesn't directly know how to handle it. You shouldn't put a filesystem on it because then the encryption would be lost. You need an additional layer between the device and the kernel. This is done with luksOpen. It creates a mapper device /dev/mapper/crypt (or any other name you give this after the device in the command above) which then can be used to  put a filesystem or something else on it. The informations written to that device will be handled by the Device Mapper and then dm_crypt (for the encryption) and the encryption related kernel modules (aes_generic, aes_x86_64, crypt and sha256_generic since the default encryption is aes-cbc-essiv:sha256).
If you want to learn more about the possible encryptions i suggest you read the homepage of cryptsetup and some wikipedia. This is the real hard/mathematical/technical stuff. Looking at the homepage you will also see which encryption has which performance impact on your system/disk I/O. This should be also considered...the stronger the encryption is, the slower your write and read speed will be. Remember, with great encryption comes great performance decrease (taken from Spidercrypt Part I). Honestly, if you really need top level encryption you should go and buy hardware encryption devices. This will make your life much easier and your system faster (and you can encrypt your whole disk). But be sure you find one that is compatible with linux...

cryptsetup offers some additional actions which should be looked up in the manual. Most of them (like luksClose) are straightforward. Be especially careful with the header-dump actions since they will give out informations which can be used to directly access the device without knowledge of the key. But those informations can also help you to recover an accidential overwritten partition information (maybe you did an pvcreate on the device /dev/sda2 before using luksOpen).

LVM

LVM is great. It offers a lot of flexibility, especially with grow-and shrinkable filesystems like ext[234] and others. But LVM is also an additional layer. This is used to avoid the physical limitations with disks. You can only have one or more partitions per disk and only one filesystem on one partition (well, this restriction is logical and applies to LVM too). So if you have a couple of disks with 500GB and you need a filesystem with 1TB you can't use traditional partitioning, you are limited to the physical disk size. In some OSes you also have a limitation of partitions (slices in Solaris) per physical disks.
With LVM you can combine one or more partitions to one Volume group. So lets say, you have 3 disks sda, sdb and sdc, all 100GB. You use (traditionally) sda as a system partition with 50 GB for /, 5GB Swap and the rest for /home. sdb and sdc are data disks for your large Volume /data. So you combine sdb and sdc into one VG datavg and create a logical volume lv_data (like a partition) onto that VG. Names should be descriptive but don't have to be. If you don't provide a name during LV creation it will be names lvol0, lvol1 and so on.
If you have 2 physical disks as one volumegroup you can also stripe logical volumes onto it - this will increase the performance. Also you could mirror it and can avoid using mdadm. But this is beyond the scope of this writing. Also keep in mind that you can mirror/stripe on the same disk on different partitons (which is useless for the idea itself but can be used to test).
You can add a lot VGs with different physical disks/partitions and on those VGs you can create a lot of LVs.

Note: All of the commands i used can be also used with the high level command lvm and then the subcommands. So the man page for lvm is a great starting point to find out what it will offer to you.

So back the tutorial

pvcreate /dev/mapper/crypt

This create a physical volume on /dev/mapper/crypt. Nice, huh? You could say this by looking at the command. Well, basically it will mark /dev/mapper/crypt as a physical volume to use with LVM. There is not much done here, just the headers for LVM are written to the (in our case) encrypted device. Without encryption you would need to use /dev/sda2.

vgcreate cryptvg /dev/mapper/crypt

This creates the Volume group cryptvg with the physical volume /dev/mapper/crypt. There could be more than one physical volume (e.g. /dev/mapper/crypt1 /dev/mapper/crypt2 or /dev/sda2 /dev/sdb2 without encryption and so on).
In our case we used a cryptvolume, so any data written on that device will be handled through dm_crypt with the appropriate encryption and written onto the physical partition /dev/sda2. So if you don't have the key to unlock the device /dev/sda2 you won't even be able to see what kind of data is on it (well, some people may be able to, but i am not). The physical disk is "splitted" into physical extents (PE), which are small parts of data, usually 4MB. So the smallest unit that can be allocated for a logical volume is 4MB=1PE. But all your fears come true, a logical Volume is almost ever a multiple of this ;) Now we have some kind of container with the ability to store small chunks, to combine them and so on. Which brings us to the next step

lvcreate -L 5G -n rootlv cryptvg

So, now the logical volume is created by allocation those 4MB chunks to build a large "partition". The name is rootlv (for root logical volume - could also be lv_root or just root) and is created on the volume group cryptvg. Wow, life can be so easy. Actually linux don't care much about the partition type, windows is here more strict. For linux the type of "filesystem" is looked up when the filesystem is used. So you can easily put the partition for swap as a logical volume on that volume group. Or another LV for an ext2 filesystem, one for JFS and so on.
For the size you can use -L and give it "human readable" parameters like 5G, 10M or even 1T/P/E (Terra-/Peta-/Exabyte if you have the space). Or you can use -l (lowercase L) to size it from a physical extent perspective (so 100 extents means 400MB).
Logical volumes can be accessed through 3 ways:

  • /dev/mapper/vgname-lvname which uses the device mapper notation
  • /dev/vgname/lvname which is more like a direct association to the device
  • /dev/dm-1 which is the LVM created block device, allocated through a major and a minor number which is noted behind the dm-. The 2 cases above are just links to this one.
LVM offers a lot of functionality besides what i showed you. You can resize a logical volume, you can add or delete physical volumes from the volume group and so on. One nice feature is snapshots volumes, which will keep the same state of the filesystem from the time when you took the snapshot. So you can backup the snapshot LV in a consitent state and work normal with the system at the same time. Or you can take a snapshot, upgrade the system and if anything goes wrong, you can revert the volume back to the time before the upgrade/snapshot state (remember /dev/sda1!).

The created LVs could now be used as a "raw device" which means, that the application (usually databases) will take care of the allocation of data in it. Or, suited for our purpose of installing an OS to it, put a filesystem on top of it. So here we go

Filesystems

mkfs.ext4 /dev/cryptvg/rootlv

Not much to say here. An ext4 filesystem gets created, using the normal attributes. I used /dev/vgname/lvname, but /dev/mapper/vgname-lvname will work too. You can also use /dev/dm-1 but this is not really straightforward.
The filesystem is most important from the user perspective. It is the thing that gives programs the ability to write, read and manipulate data. I forgot to test it (and i am a bit tired to do now) if you can install BB without creating those filesystems. Actually i got them and this was my focus. But the installer should recognize the volumegroup and the logical volumes itself, so the creation of the filesystems should be optional. Maybe you want to test this, i want to go to bed now :)

Is that all real?


Also i wanted to give you a good oversight about what happened and what means what (for now until the installation) this  whole article is still not the reality and some informations maybe even wrong (sorry for that). But it should give enough informations to understand what will be done to your poor disks when using LVM and cryptsetup :) I touched some deeper internals but if you want to really understand whats going on, you will have to get the informations yourself. Starting from physical disk design (heads, sectors and cylinders) over to Bus protocols like SATA or IDE up to the kernel into the userspace. That is a nice travel...which fills thousands of pages i will never write. Its just 1s and 0s - but they change a lot during their trip to your monitor.


So have fun exploring.

Tuesday, February 14, 2012

Installing BackBox Linux with LVM and (nearly) complete encryption

Abstract

The idea is to install BackBox with full disk encryption and LVM.
BackBox Linux is a Penetration Testing Distribution based on Ubuntu Linux. Or better, Xubuntu Natty Narwhal (11.4 LTS) so its based on the XFCE window manager and not on Unity or KDE.
Ubuntu doesn't offer LVM nor encrypted installation out of the box. Sure, with BackBox this is the same. So its time for some tricks.

Background

I own a netbook and i prefer to encrypt it. I used to install Backtrack 5 R1 back when i got the netbook. BT is mainly a great penetration testing distribution. But as a matter of fact it is also great for general network analysis/troubleshooting and related stuff which i like to use it for (you know those friends "can you help me getting my WLAN working?").
Since you can lose a netbook faster than a normal PC i decided to setup disk encryption so when i lose it or someone needs my equipment more than i do, i won't also lose all my informations. I followed Backtrack 5 – Bootable USB Thumb Drive with “Full” Disk Encryption to setup my little Acer.
Now don't get me wrong, BT is great. It's a tested and well-made distribution initially based on Ubuntu. But after a release there is not much done with the OS, the applications receive updates but no security fixes for the OS. From the penetration testing perspective this is ok - run it just for your tests via Live-CD or USB and after this go back to you usual OS. But if you use it as installed OS you would like to have this also patched, especially against critical (local) bugs like http://www.exploit-db.com/exploits/18411/. Debian patches those bugs, Ubuntu does then too. Backtrack won't do it because of their philosophy. I can understand this but i need something else.
So one day i stumled across the Securitytube Tools site and found BackBox. I was first thinking of a BlackBox/FluxBox based distribution but learned its XFCE! That was my first Desktop Environment when i started using Ubuntu 5 years back (before that i used almost every other no-tiling WM/DE available). BackBox is packed in Italy, so basically my 2nd italien distribution. The first one was (only as live medium since it is for forensics) DEFT Linux.
After playing around a bit with BackBox i found it nice to handle. As written above it doesn't offer disk encryption and no LVM as basic install routine. Also i got some data on my disk already which i don't want to destroy and also won't want to backup before (i know, i know...lazy guy sometimes). Lukily /home is a separate logical volume and i got some data volumes too. So just install BackBox to / and /boot and i should be done. But thats needs some testing before to ensure everything will run fine.

Preferences

  • A virtual machine powered by Virtual Box with 8GB disk and 768MB RAM
  • An ISO of BackBox 2.0.1 amd64
  • some background knowledge about cryptsetup, LVM under Linux, filesystems and ubiquity (the Ubuntu installer)
  • recordmydesktop for making a video 
  • some "still looking for a working one" video editing program under linux and some time to work on that video. Also patience and some more learning.
Note from the real world: NetBooks have no CD drive usually so i needed to put the ISO onto an USB stick. This made some stress since my 2 free sticks won't boot directly after the preparation via unetbootin. It seems that one stick is really broken somehow and i needed to format the other before i was able to boot it directly and install all.

Documentation


After the BackBox boot you should (if it applies) change your keyboard layout and choose the best mirror. You will need a working network connection. Then open up a terminal window.


Note: The following steps can easily damage/break your system. If you are not sure what the step means just don't do it and don't start before you are sure you understood every single step in the document. For this writing I assume you have a completely blank system. If you are doing this on a live system, plesae be very careful and be sure your backups are working! Additionally test test test before.


Lets start. Become root with

sudo -s

lvm2 is not included in the base install, so we need to install it

apt-get update
apt-get install lvm2

After this we need to create some partitions using fdisk on sda (my hard disk). The installer is not capable of doing this the way we need it, but it will see the existing partitions/filesystems:

fdisk /dev/sda

I created a small primary partition sda1 (300MB is enough) and used the rest for another primary partition sda2. The partition type on sda1 is 83 (Linux) and on sda2 8e (Linux LVM). sda1 is marked as active partition.

Now we can save this and start with the activation of the encrypted device. 
First format the partition (use uppercase YES and a secure passphrase - please remember that following my instructions you will end up with an english keyboard at boot time, which maybe different from you native keyboard).

cryptsetup luksFormat /dev/sda2

Now open the device as a mapper to crypt:

cryptsetup luksOpen /dev/sda2 crypt

So the encryption is running on that device, next we need to create a Volumegroup (i used cryptvg but any other name is good too) and Logical Volumes with filesystems. The installer will tell you that you need at least 4.2GB of space. Good thing is that you can extend all partitions in LVM on the fly later. Please note that you don't create the physical disk on /dev/sda2 (like without encryption) but on the mapper device /dev/mapper/crypt!

pvcreate /dev/mapper/crypt
vgcreate cryptvg /dev/mapper/crypt
lvcreate -L 5G -n rootlv cryptvg
lvcreate -L 500M -n swaplv cryptvg
lvcreate -L 500M -n homelv cryptvg


Now create the filesystems:

mkfs.ext4 /dev/sda1
mkfs.ext4 /dev/cryptvg/rootlv
mkfs.ext4 /dev/cryptvg/homelv
mkswap -f /dev/cryptvg/swaplv

Thats it! Now launch the installer and follow the normal instructions.


Important addition: Once you come to the step "Allocate drive space", choose "Something else"! I hope this is clear, the other disk partitioning options won't help you! Proceed with the steps below.
Use /dev/mapper/cryptvg-rootlv as you / (check format partition) and /dev/mapper/cryptvg-homelv as /home. swaplv will be used automatically. /dev/sda1 is /boot (check format partition).

After the install do NOT click reboot! Your system isn't aware of encryption and LVM now! Use the "Continue testing" button.

Lets setup a chroot environment for the further installation. You need to mount sys, proc and /dev into the chroot to be aware of your Live-CD system.

mount /dev/cryptvg/rootlv /mnt
mount -t proc proc /mnt/proc
mount -t sysfs sys /mnt/sys
mount -o bind /dev /mnt/dev
mount /dev/sda1 /mnt/boot
chroot /mnt

In the chroot, you need to enable grub to recognize and decrypt your cryptdevice. Use any texteditor you like (e.g. vi, nano) to edit /etc/default/grub. Change the line

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
to
GRUB_CMDLINE_LINUX_DEFAULT="cryptdevice=/dev/sda2:crypt quiet splash"

Now we need to change the /etc/crypttab to use the UUID of /dev/sda2 as cryptdevice:

ls /dev/disk/by-uuid/$(ls -l /dev/disk/by-uuid | grep sda2 | grep cut -d" " -f8) >> /etc/crypttab

Edit the /etc/crypttab to look like this (inserted/added content is bold)

# <target name> <source device> <key file> <options>
crypt /dev/disk/by-uuid/564606ec-3c60-4141-833e-463a67307141 none luks

The /dev/disk/by-uuid/... should be added automatically from the command above. Later you can change this to have a key on an USB stick or something else...

Now update the system (no dist-upgrade) and install lvm2 and cryptsetup:


apt-get update
apt-get upgrade
apt-get install lvm2 cryptsetup

This is the "lazy way" to update the initramfs and also update grub. Benefit is that you system will be upgraded, but after the reboot you will have to do another update. Somehow it isn't getting all the stuff.

Now you can enter exit or press Ctrl-D to leave the chroot and reboot the system. Once the system comes up you should be greeted with a Prompt to enter you passphrase. Remember the english keyboard, type your passphrase and the boot should proceed with a nice message saying that the cryptdevice was successfully unlocked. Once you are logged in, the encryption works completely transparent.

Looking forward

This tutorial should also work with other Debian/Ubuntu based distributions. I tested it with Ubuntu 11.10 and it works.

I hope that i manage to prepare my screencast on this tutorial, which will also show the installation itself. It will also clarify some points i just touched without deeper explanation. Sorry for that but 1. as mentioned above i am a lazy guy sometimes and 2. i would like to press you to understand what you do.

If you have an existing Ubuntu 11.4 installation you can also add the BackBox repository easily to your installation, look at the BackBox wiki. If you haven't set it up with encryption, you will need to follow this tutorial doing a complete new installation.

References

Friday, February 10, 2012

Blog will open soon

Ok,

finally i made myself a blog. Not too hard with Tools from Google and Co.

Usually i prefer to run such stuff myself, but i don't want to rent a webserver just to tell the world why i am up to. At least not until i get payed for it :)

So have a nice day, some posts will follow.

SaThaRiel