Well, what a lazy poster i am. But i can explain it, i have a family, some projects running at the moment and not too much time to test out a nice lvm setup and other interesting stuff...but read on.
Hacking Hacking-Lab
Trying to become more involved with the world of penetration testing i made myself an account for Hacking Lab. A swiss based online training lab setup, partly for free.
To use the lab you will either need a live cd which can be downloaded from their homepage (after registering) or setup your PC to match their VPN environment. I am working on the latter a bit to have BackBox as my core platform. But the live cd includes some nice tools which should be looked up too.
After you registered you can choose an event. Inside this event you find different challenges inside a lab which can only be reached through VPN.
The labs themselves are made with different virtual hosts that needs to be hacked, mostly in a capture-the-flag style. So become root and get a file from the system. But there are also more riddle-like hacking activities like notpron or application relevant todos. Some stuff can be done locally without the access to the lab. From what i have done until now (which isn't that much) the challenges seem to be more mis-configuration relevant, i was not able to exploit a system.
Once you made a challenge you can send your solution together with some proof/logfiles (don't forget the gold nuggets) to the jury/teacher and if you have done well, you will receive points based on the difficulty and your solution.
Sounds fun? Then go on, register and try it out. A better (and more important - visual) introduction was made by the hacking lab guys themselves and can be watched here.
Become MSFE
Yeah, i registered for SMFE. While it is focused "only" on the famous Metasploit Framework it is still a great opportunity, the price is affordable for a private person and super cheap for an IT sec related course.
I have choosen the Exam plus 30 day lab access (which i haven't enrolled at the moment). But you can also do all the things on a private lab setup and just choose to do the exam. Look up the options on the website.
As usual Securitytube offers you all training videos for free. Vivek does a great job explaining and showing you the internals of Metasploit. Sadly i find the lab questions a bit confusing and apart from all courses i took in the past, they seem to apply for the next chapter, not the current. But to be honest i haven't read them all.
If you are a professional you sure should choose another training, especially since you will use Metasploit Pro which has some extended capabilities (best and most important thing sure is reporting) and is usually handled through the webinterface. But everyone must decide this for himself.
If i really find the time to do the lab and finally make the exam i will maybe go for the PWB certification. This is a more general course from the tools included with Backtrack Linux and from what i have read doesn't allow Metasploit during the certification at all.
I know, i switched to BackBox but the toolbox from Backtrack is much bigger. BackBox only want's one tool for a task while BackTrack includes almost all famous tools. Question of philosophy, i can understand both.
Bashing kdump
Well, actually what every sysadmin ever dreamed about - crashing a server system the whole day without any trouble. But the reason is, as usual, not the fun.
After some unexpected crashes of our systems we need to get a crash dump for our RHEL 6 installations (RH: Looks like a hardware failure, do you have a dump?). Usually an easy task - install kexec-tools, configure /etc/kdump.conf, start kdump and you're done. Well, reserve some memory for the crashkernel...
kdump will insert an additional initrd image inside RAM. Once the "main kernel" crashes, this will dump the memory contents at time of the crash to a device, network share or via ssh to another system. So RH will have something to look at.
The memory for the extra kernel gets allocated during boot time via the crashkernel=X@Y parameter (or auto for the lucky and lazy). X means the size of the region and Y is the offset.
But then why do i have to enter the prominent echo c > /proc/sysrq-trigger all the day?
As it turned out the HP servers we are using have a controller that the crash kernel can't handle. Nice! NFS is not an option so we choose ssh. Works with a fresh install in an VM out of the box, dumps just nice.
But not in our environment - we have to use the prior kernel version because the net performance with the current kernel is too slow and additionally we have a huge amount of RAM, which doesn't seem to be handled well by the kexec-tools and our kernel. The crashkernel allocation is limited to something between 820MB and 1024MB (still testing - looks like 850MB - but that is really not for sure). With the current kernel in RH 6.1 this seems to be sufficient - or at least i think so. But bug hunting is sometimes a task where you crawl all day behind those little creatures without really catching one of them, they just slip through your fingers. So crashing, waiting (takes around 6 minutes before you know that the crash dump will be written), changing the crashkernel size, installing different versions of kdump and so on.
Well, we have a support contract with RH, so why not ask them? As it turns out, they also have to play the old trial and error game. Not really helpful with all of our problems so far. Just do it yourself.
But i had no filesystem or any other problem during my current tests which are more than 30! So crashing RH systems work without a problem.
Enough moaned...
See you soon!
Thursday, March 15, 2012
Tuesday, March 6, 2012
Decrypting the LVM-Cryptsetup - part 2
Review
Some little things have been done until now.
A disk has been partitioned to support a /boot filesystem (holding the kernel, initram image and bootloader configuration) and an LVM volume.
Before applying the LVM setup, we created an encrypted device /dev/mapper/crypt. Data written to that device is handled by the dm_crypt module and writes the raw data encrypted onto the physical disk. On this layer the LVM setup was made, including the logical volumes and filesystems/swap for the installation
Installation
Well, personally i don't really like Ubuntu and its derivates because they tend to be to macroscopic. Additionally it seems that they use just the Debian development resources to update their systems and don't develop/fix much on their own. But that is just my view.
Ubuntu did indeed one really great step for the linux community. They helped a lot of people getting a linux up and running with a superb and easy installer. I would say that installing Ubuntu is easier than installing Windows.
Drawback: Because the installer is so easy it won't offer you all install options that could be possible. That's why we needed to take the round on the command line and do some things manually. If you just want to install BackBox (or Ubuntu), throw in the DVD, boot and then click install. After some clicks and minutes you're done. But LVM (which should be default in Ubuntu soon) and cryptsetup, since it's a transparent layer, will be honored by the installer.
The steps:
As mentioned above, because it's easy it won't offer you many install options. You can choose to install it beneath another operating system, you can install it over another operating system, on a blank disk and do something else. Something else offers a bit more but needs some preparations.
The chroot
chroot (Change Root) is mainly mainly to devide applications from the rest of the system. The application runs in its own area of the system and has just the programs it needs for itself. Maybe those programs can also be from a different operating system level. Read more about the basic functionality at wikipedia.
In our case we need the chroot to emulate the later running system. We sure need the root filesystem and /boot is also needed because the initramfs needs to be updated to include lvm2 and the crypt-modules at boot time.
The first step is self explanatory - just mount the root-filesystem of the machine to /mnt. You could create your own mount point for this if you want to, but /mnt is wiedly known as standard mount point and available on every Linux/Unix system. Just stick to the basics.
After this we mount some virtual filesystems onto /mnt:
Now /boot is mounted. Since we mounted the relevant system filesystems on /mnt, this step could be done after chrooting to /mnt. But i did it before, thinking that the steps are more straighforward.
chroot /mnt
Works as designed, we are now "inside" the newly installed system. All changes done inside the chroot will be persistent. And that's great. So we can install and update software and also change configuration files. Just what we need, because lvm2 and cryptseup is not installed on the default install and additionally there is no configuration for LUKS.
Note: Changes made to the desktop at this time won't be persistent because it still runs from the Live-CD system. We just use it to ensure that the relevant configuration files are in place and correct and the needed packages are installed, setup and included in the initrd image.
Just 1 mile to go
Inside the chroot the grub default configuration has to be extended to make grub aware of our new cryptdevice. This can be done by manually editing the GRUB_CMDLINE_LINUX_DEFAULT and add the parameter cryptdevice=/dev/sda2:crypt. This way update-grub will automatically add it to the kernel line and grub is aware of the device.
The next part is a bit tricky - or at least it looks like it. Actually you just tell luks how which device should be handled and where to find the key.
So the file /etc/crypttab needs to be edited. There are 4 parameters:
ls /dev/disk/by-uuid/$(ls -l /dev/disk/by-uuid | grep sda2 | grep cut -d" " -f8) >> /etc/crypttab
This cuts out the UUID for /dev/sda2 and appends it to /etc/crypttab. After this you have to edit /etc/crypttab to make it work.
A better way could be
ls -l /dev/disk/by-uuid | awk '/sda2/ { print "crypt\t\tUUID="$9"\tnone\tluks" }' >> /etc/crypttab
but this is up to you.
And thats it - just update your newly installed system inside the chroot and don't forget to install lvm2 and cryptsetup:
apt-get update
apt-get upgrade
apt-get install lvm2 cryptsetup
Don't do a dist-upgrade, this should be run later when the machines comes up again.
Good thing when installing lvm2 and cryptsetup, it will automatically update the initrd image for you. So the needed drivers and modules will be included in the initrd mini-root. Oh, and besides...you are running in a chroot, the package database is also updated with the necessary informations. Nice!
Conclusion
Well, installing an Ubuntu based system is really easy. And if you make up your mind a bit its not much harder with encryption and lvm2.
If you leave away the cryptsetup you can install the system with LVM only - maybe using the internal home-only-encryption offered by the installer. Or you can just partition your system like you always do (maybe /dev/sda1 for /boot and an extended partition for / on /dev/sda5) and just encrypt it without LVM.
If you understand what happened it is also really easy to install BackBox next to another existing operating system. Keep in mind that the boot loader of any operating system might break the ability to boot into another one - but must work for its own system.
Hope you enjoy this article, see you the next time.
Some little things have been done until now.
A disk has been partitioned to support a /boot filesystem (holding the kernel, initram image and bootloader configuration) and an LVM volume.
Before applying the LVM setup, we created an encrypted device /dev/mapper/crypt. Data written to that device is handled by the dm_crypt module and writes the raw data encrypted onto the physical disk. On this layer the LVM setup was made, including the logical volumes and filesystems/swap for the installation
Installation
Well, personally i don't really like Ubuntu and its derivates because they tend to be to macroscopic. Additionally it seems that they use just the Debian development resources to update their systems and don't develop/fix much on their own. But that is just my view.
Ubuntu did indeed one really great step for the linux community. They helped a lot of people getting a linux up and running with a superb and easy installer. I would say that installing Ubuntu is easier than installing Windows.
Drawback: Because the installer is so easy it won't offer you all install options that could be possible. That's why we needed to take the round on the command line and do some things manually. If you just want to install BackBox (or Ubuntu), throw in the DVD, boot and then click install. After some clicks and minutes you're done. But LVM (which should be default in Ubuntu soon) and cryptsetup, since it's a transparent layer, will be honored by the installer.
The steps:
- click on the install icon (you guessed it)
- choose an installation language. I recommend choosing English US for all Linux installations.
- let away the update during installation (the vid shows something different but who cares) and optional choose 3rd party software. Please check the informations on the top of the window, net should be available (actually you installed software from the network), you system should not run on battery power (just to be sure) and need at least 4.4GB free disk space (BackBox 2.0.1)
- next comes the interesting step - disk partitioning. Well, sure we need to do "Something else" than just click install entire system. Inside the advanced partitioning window, everything looks familiar. Just use /dev/sda1 as /boot with ext4 filesystem, /dev/mapper/cryptvg-rootlv as / (choose format here, otherwise the installer will complain about this) and /dev/mapper/cryptvg-homelv as /home (if you want to, format is optional). /dev/mapper/cryptvg-swaplv is already recognized as swap area.
- now its time to install. During the installation you will be asked which time zone you live in, which keyboard layout you want to have (doesn't matter which installation language you choose) and you should create a user. If you create the user with automatic login (which i don't recommend, especially hen using an encrypted installation - be completely paranoid, not just a bit), you still have to remember the password. sudo (for administrative tasks) will ask for it, to confirm that you are you. A user account can be hacked remotely, a password compromised (over net) is not so easy if you follow the usual best practices for passwords (long and strong).
You are offered to encrypt you home folder, but that is more for a not-encrypted installation or the paranoids with people tracking them. - after the installation: Continue Testing!
As mentioned above, because it's easy it won't offer you many install options. You can choose to install it beneath another operating system, you can install it over another operating system, on a blank disk and do something else. Something else offers a bit more but needs some preparations.
The chroot
chroot (Change Root) is mainly mainly to devide applications from the rest of the system. The application runs in its own area of the system and has just the programs it needs for itself. Maybe those programs can also be from a different operating system level. Read more about the basic functionality at wikipedia.
In our case we need the chroot to emulate the later running system. We sure need the root filesystem and /boot is also needed because the initramfs needs to be updated to include lvm2 and the crypt-modules at boot time.
The first step is self explanatory - just mount the root-filesystem of the machine to /mnt. You could create your own mount point for this if you want to, but /mnt is wiedly known as standard mount point and available on every Linux/Unix system. Just stick to the basics.
After this we mount some virtual filesystems onto /mnt:
- /proc is for processes and kernel related informations that must be known in the chroot. I would say that it is the most interesting virtual filesystem, you can lookup and do a lot of things inside. Look here.
- /sys is the system filesystem - which holds the driver and device informations about your running system.
- /dev is the device filesystem which holds all devices and is used to access them - like /dev/sda1 or /dev/cryptvg/rootlv
Now /boot is mounted. Since we mounted the relevant system filesystems on /mnt, this step could be done after chrooting to /mnt. But i did it before, thinking that the steps are more straighforward.
chroot /mnt
Works as designed, we are now "inside" the newly installed system. All changes done inside the chroot will be persistent. And that's great. So we can install and update software and also change configuration files. Just what we need, because lvm2 and cryptseup is not installed on the default install and additionally there is no configuration for LUKS.
Note: Changes made to the desktop at this time won't be persistent because it still runs from the Live-CD system. We just use it to ensure that the relevant configuration files are in place and correct and the needed packages are installed, setup and included in the initrd image.
Just 1 mile to go
Inside the chroot the grub default configuration has to be extended to make grub aware of our new cryptdevice. This can be done by manually editing the GRUB_CMDLINE_LINUX_DEFAULT and add the parameter cryptdevice=/dev/sda2:crypt. This way update-grub will automatically add it to the kernel line and grub is aware of the device.
The next part is a bit tricky - or at least it looks like it. Actually you just tell luks how which device should be handled and where to find the key.
So the file /etc/crypttab needs to be edited. There are 4 parameters:
- target name: Defines the mapper target for the device, in our case crypt
- source device: This should be the UUID of the device, using /dev/sda2 caused some problems in the past.
- key file: This could be a path to a keyfile (like /etc/key) or a usb device which stores the key (this is out of scope). In our case we want to enter our overly long passphrase (i really recommend 20+ characters) everytime at boot time, so we say none.
- options: If you choose something different or another encryption method you could add this here. We just enter luks.
ls /dev/disk/by-uuid/$(ls -l /dev/disk/by-uuid | grep sda2 | grep cut -d" " -f8) >> /etc/crypttab
This cuts out the UUID for /dev/sda2 and appends it to /etc/crypttab. After this you have to edit /etc/crypttab to make it work.
A better way could be
ls -l /dev/disk/by-uuid | awk '/sda2/ { print "crypt\t\tUUID="$9"\tnone\tluks" }' >> /etc/crypttab
but this is up to you.
And thats it - just update your newly installed system inside the chroot and don't forget to install lvm2 and cryptsetup:
apt-get update
apt-get upgrade
apt-get install lvm2 cryptsetup
Don't do a dist-upgrade, this should be run later when the machines comes up again.
Good thing when installing lvm2 and cryptsetup, it will automatically update the initrd image for you. So the needed drivers and modules will be included in the initrd mini-root. Oh, and besides...you are running in a chroot, the package database is also updated with the necessary informations. Nice!
Conclusion
Well, installing an Ubuntu based system is really easy. And if you make up your mind a bit its not much harder with encryption and lvm2.
If you leave away the cryptsetup you can install the system with LVM only - maybe using the internal home-only-encryption offered by the installer. Or you can just partition your system like you always do (maybe /dev/sda1 for /boot and an extended partition for / on /dev/sda5) and just encrypt it without LVM.
If you understand what happened it is also really easy to install BackBox next to another existing operating system. Keep in mind that the boot loader of any operating system might break the ability to boot into another one - but must work for its own system.
Hope you enjoy this article, see you the next time.
Saturday, March 3, 2012
Open Curtain - Cryptsetup and LVM with BackBox
Finally progress
After a lot of trouble i made progress in video editing - please don't expect too much. It's the content that counts :)
Movie
Credits
After a lot of trouble i made progress in video editing - please don't expect too much. It's the content that counts :)
Movie
Credits
- Installation done with BackBox 2.0.1 on Virtual Box (host Arch Linux with spectrwm window manger - thats what caused the red borders)
- Shot was taken using recordmydesktop
- Converted for editing using mencoder: mencoder install_backbox_with_crypt_and_lvm.ogv -ovc lavc -lavcopts vcodec=mjpeg -nosound -o output.dv
- Video editing was done with OpenShot (great tool but really needs some time to get used to it - or better: RTFM ;) )
- Music: "Dont' touch" by SunIce - had to be an italian artist ;) Support them please if you like their music.
Saturday, February 18, 2012
Upcoming stuff
So, the next posts i plan to do is really the video of the crypt-install. I have problems finding (and using) a good video editor under linux. Maybe i am doing something really wrong. Besides it takes its time that i don't have too much. But i hope i will finish it until the end of the month.
If this goes well, i will do another vid playing around with LVM, i need to make up my mind what to show and how.
The decrypt post will have a follow up. Some things still need to be explained, especially what happened after the installation completed.
If i have a way to make videos i think i will have a lot more stuff to show. Sometimes a videos is better than 1000 words :)
Keep your heads clear.
If this goes well, i will do another vid playing around with LVM, i need to make up my mind what to show and how.
The decrypt post will have a follow up. Some things still need to be explained, especially what happened after the installation completed.
If i have a way to make videos i think i will have a lot more stuff to show. Sometimes a videos is better than 1000 words :)
Keep your heads clear.
Wednesday, February 15, 2012
Decrypting the LVM-Cryptsetup
Ok, once i have posted the Installing BackBox Linux with LVM and nearly complete encryption, i felt that it is not complete/sufficient. The desire to dig deeper into it and learn what happens behind the scenes is there...so lets just do it :)
Note: I can't give you all the backgrounds and can't explain everything - this would (and actually does) fill books. But a little bit of information will be here.
The whole system encryption
The disk encryption article won't encrypt your whole disk. /boot is still unencrypted. Ok, its just the kernel with ramfs (mini-root). But it is unecrypted. Ubuntu and grub2 doesn't offer you the ability to boot an encrypted partiton directly at the moment. There are programs (mostly windows based) that will offer you the complete disk encryption but i have never seen nor used one.
Side note - yes, you can encrypt also the kernel and all other things from /boot with a custom built grub2. But that is not easy to setup, i may write an article someday. In the meantime you can look at this nice article: full system encryption for linux.
Another alternative for full disk encryption is possible with hardware encryption (adapters) or if you separate your /boot and the boot-loader to an thumb-drive or similar. But i won't go through that now.
Live with it, /boot stays unencrypted. But it's nothing in there that you can't find on the Iso and repositories coming with the distribution. Sure someone could compromise/manipulate the kernel to put in a backdoor on your system. For me this was no decision for disk encryption. But it is indeed a real security thread that should be kept in mind.
Data manipulation brings me to the next point: Encryption offers you another benefit. You have a data integrity check for free. You encrypted data can not be manipulated from the outside. If someone boots the system with an external medium (like CD, USB stick etc.) no access to the data is possible. BUT if you are running the system and someone breaks in (through a network exploit, no screen lock and so on) you data is available (and can be manipulated/deleted) like on a normal filesystem. So if you want to keep real sensitive data on your disk like credit card numbers or your plans to rule the world, you should encrypt it directly with a tool like ccrypt, a suite like truecrpyt or any other program which comes to your mind.
Encrypted disks also won't help you with not-encrypted network traffic, data copied to another medium and so on. It is strictly bound to the devices you encrypted, nothing else. But it runs transparent for you on your system after you entered the correct passphrase. Good enough for most people :)
So lets explore what we have done
fdisk
fdisk /dev/sda
Used to create and manipulate the partition table, in our case on disk sda. It works directly at the physical layer of a disk, so it is a program that should be known well and used with care. You can also use parted or gparted (the graphical interface) to interact with the disk layout, but i got used to fdisk.
I decided to use just 2 primary partitions, sda1 and sda2. This is the easiest way and should be used in this layout because after the creation of sda2 there is no space to create another partition. When you thinking about a more complex example (which is usually not necessary when working with LVM, see below) you will have to keep in mind that a physical disk can only have 4 primary partitions or 3 primary and then one extended which then is divided into logical partitions (starting from sda5). But for todays needs, complex physical partitioning is really ancient.
cryptsetup
It's used to create and manage encrypted partitions - period. Or not?
The first step to create the encryption actually was
cryptsetup luksFormat /dev/sda2
So what we have here
You have up to 8 keyslots - so you can use different mechanisms to unlock the device later (keyfiles or passphrases). The more keys you assign to the device, the more likely it is that someone may gather one of those keys (theoretically).
Note: I can't give you all the backgrounds and can't explain everything - this would (and actually does) fill books. But a little bit of information will be here.
The whole system encryption
The disk encryption article won't encrypt your whole disk. /boot is still unencrypted. Ok, its just the kernel with ramfs (mini-root). But it is unecrypted. Ubuntu and grub2 doesn't offer you the ability to boot an encrypted partiton directly at the moment. There are programs (mostly windows based) that will offer you the complete disk encryption but i have never seen nor used one.
Side note - yes, you can encrypt also the kernel and all other things from /boot with a custom built grub2. But that is not easy to setup, i may write an article someday. In the meantime you can look at this nice article: full system encryption for linux.
Another alternative for full disk encryption is possible with hardware encryption (adapters) or if you separate your /boot and the boot-loader to an thumb-drive or similar. But i won't go through that now.
Live with it, /boot stays unencrypted. But it's nothing in there that you can't find on the Iso and repositories coming with the distribution. Sure someone could compromise/manipulate the kernel to put in a backdoor on your system. For me this was no decision for disk encryption. But it is indeed a real security thread that should be kept in mind.
Data manipulation brings me to the next point: Encryption offers you another benefit. You have a data integrity check for free. You encrypted data can not be manipulated from the outside. If someone boots the system with an external medium (like CD, USB stick etc.) no access to the data is possible. BUT if you are running the system and someone breaks in (through a network exploit, no screen lock and so on) you data is available (and can be manipulated/deleted) like on a normal filesystem. So if you want to keep real sensitive data on your disk like credit card numbers or your plans to rule the world, you should encrypt it directly with a tool like ccrypt, a suite like truecrpyt or any other program which comes to your mind.
Encrypted disks also won't help you with not-encrypted network traffic, data copied to another medium and so on. It is strictly bound to the devices you encrypted, nothing else. But it runs transparent for you on your system after you entered the correct passphrase. Good enough for most people :)
So lets explore what we have done
fdisk
fdisk /dev/sda
Used to create and manipulate the partition table, in our case on disk sda. It works directly at the physical layer of a disk, so it is a program that should be known well and used with care. You can also use parted or gparted (the graphical interface) to interact with the disk layout, but i got used to fdisk.
I decided to use just 2 primary partitions, sda1 and sda2. This is the easiest way and should be used in this layout because after the creation of sda2 there is no space to create another partition. When you thinking about a more complex example (which is usually not necessary when working with LVM, see below) you will have to keep in mind that a physical disk can only have 4 primary partitions or 3 primary and then one extended which then is divided into logical partitions (starting from sda5). But for todays needs, complex physical partitioning is really ancient.
cryptsetup
It's used to create and manage encrypted partitions - period. Or not?
The first step to create the encryption actually was
cryptsetup luksFormat /dev/sda2
So what we have here
- cryptsetup - well, thats easy, the command actually doing somethings with encryption
- luksFormat - this means basically 2 "operations". luks for handling LUKS (Linux Unified Key Setup, an encryption standard under Linux) and Format, which doesn't format the device in the common sense. It doesn't even delete anything on that partition except a small part to mark it as a LUKS partition and creates key-slots. From the man-Page:
initializes a LUKS partition and sets the initial key, either via prompting or via <key file>.
So if you want to really blank the device (maybe you had confidential data on it before) you should wipe it with either a program or just do a
dd if=/dev/urandom of=/dev/sda2 bs=4096
The latter is NOT compliant to any data erasing standard but should be sufficient enough for most people. Anyway it will take a long time to complete. - /dev/sda2 is the device being initialized.
You have up to 8 keyslots - so you can use different mechanisms to unlock the device later (keyfiles or passphrases). The more keys you assign to the device, the more likely it is that someone may gather one of those keys (theoretically).
Then the second step
cryptsetup luksOpen /dev/sda2 crypt
opens the device. But why open?
There is a device /dev/sda2 which supports encryption but the kernel doesn't directly know how to handle it. You shouldn't put a filesystem on it because then the encryption would be lost. You need an additional layer between the device and the kernel. This is done with luksOpen. It creates a mapper device /dev/mapper/crypt (or any other name you give this after the device in the command above) which then can be used to put a filesystem or something else on it. The informations written to that device will be handled by the Device Mapper and then dm_crypt (for the encryption) and the encryption related kernel modules (aes_generic, aes_x86_64, crypt and sha256_generic since the default encryption is aes-cbc-essiv:sha256).
If you want to learn more about the possible encryptions i suggest you read the homepage of cryptsetup and some wikipedia. This is the real hard/mathematical/technical stuff. Looking at the homepage you will also see which encryption has which performance impact on your system/disk I/O. This should be also considered...the stronger the encryption is, the slower your write and read speed will be. Remember, with great encryption comes great performance decrease (taken from Spidercrypt Part I). Honestly, if you really need top level encryption you should go and buy hardware encryption devices. This will make your life much easier and your system faster (and you can encrypt your whole disk). But be sure you find one that is compatible with linux...
cryptsetup offers some additional actions which should be looked up in the manual. Most of them (like luksClose) are straightforward. Be especially careful with the header-dump actions since they will give out informations which can be used to directly access the device without knowledge of the key. But those informations can also help you to recover an accidential overwritten partition information (maybe you did an pvcreate on the device /dev/sda2 before using luksOpen).
LVM
LVM is great. It offers a lot of flexibility, especially with grow-and shrinkable filesystems like ext[234] and others. But LVM is also an additional layer. This is used to avoid the physical limitations with disks. You can only have one or more partitions per disk and only one filesystem on one partition (well, this restriction is logical and applies to LVM too). So if you have a couple of disks with 500GB and you need a filesystem with 1TB you can't use traditional partitioning, you are limited to the physical disk size. In some OSes you also have a limitation of partitions (slices in Solaris) per physical disks.
With LVM you can combine one or more partitions to one Volume group. So lets say, you have 3 disks sda, sdb and sdc, all 100GB. You use (traditionally) sda as a system partition with 50 GB for /, 5GB Swap and the rest for /home. sdb and sdc are data disks for your large Volume /data. So you combine sdb and sdc into one VG datavg and create a logical volume lv_data (like a partition) onto that VG. Names should be descriptive but don't have to be. If you don't provide a name during LV creation it will be names lvol0, lvol1 and so on.
If you have 2 physical disks as one volumegroup you can also stripe logical volumes onto it - this will increase the performance. Also you could mirror it and can avoid using mdadm. But this is beyond the scope of this writing. Also keep in mind that you can mirror/stripe on the same disk on different partitons (which is useless for the idea itself but can be used to test).
You can add a lot VGs with different physical disks/partitions and on those VGs you can create a lot of LVs.
Note: All of the commands i used can be also used with the high level command lvm and then the subcommands. So the man page for lvm is a great starting point to find out what it will offer to you.
So back the tutorial
pvcreate /dev/mapper/crypt
This create a physical volume on /dev/mapper/crypt. Nice, huh? You could say this by looking at the command. Well, basically it will mark /dev/mapper/crypt as a physical volume to use with LVM. There is not much done here, just the headers for LVM are written to the (in our case) encrypted device. Without encryption you would need to use /dev/sda2.
vgcreate cryptvg /dev/mapper/crypt
This creates the Volume group cryptvg with the physical volume /dev/mapper/crypt. There could be more than one physical volume (e.g. /dev/mapper/crypt1 /dev/mapper/crypt2 or /dev/sda2 /dev/sdb2 without encryption and so on).
In our case we used a cryptvolume, so any data written on that device will be handled through dm_crypt with the appropriate encryption and written onto the physical partition /dev/sda2. So if you don't have the key to unlock the device /dev/sda2 you won't even be able to see what kind of data is on it (well, some people may be able to, but i am not). The physical disk is "splitted" into physical extents (PE), which are small parts of data, usually 4MB. So the smallest unit that can be allocated for a logical volume is 4MB=1PE. But all your fears come true, a logical Volume is almost ever a multiple of this ;) Now we have some kind of container with the ability to store small chunks, to combine them and so on. Which brings us to the next step
lvcreate -L 5G -n rootlv cryptvg
So, now the logical volume is created by allocation those 4MB chunks to build a large "partition". The name is rootlv (for root logical volume - could also be lv_root or just root) and is created on the volume group cryptvg. Wow, life can be so easy. Actually linux don't care much about the partition type, windows is here more strict. For linux the type of "filesystem" is looked up when the filesystem is used. So you can easily put the partition for swap as a logical volume on that volume group. Or another LV for an ext2 filesystem, one for JFS and so on.
For the size you can use -L and give it "human readable" parameters like 5G, 10M or even 1T/P/E (Terra-/Peta-/Exabyte if you have the space). Or you can use -l (lowercase L) to size it from a physical extent perspective (so 100 extents means 400MB).
Logical volumes can be accessed through 3 ways:
The created LVs could now be used as a "raw device" which means, that the application (usually databases) will take care of the allocation of data in it. Or, suited for our purpose of installing an OS to it, put a filesystem on top of it. So here we go
Filesystems
mkfs.ext4 /dev/cryptvg/rootlv
cryptsetup offers some additional actions which should be looked up in the manual. Most of them (like luksClose) are straightforward. Be especially careful with the header-dump actions since they will give out informations which can be used to directly access the device without knowledge of the key. But those informations can also help you to recover an accidential overwritten partition information (maybe you did an pvcreate on the device /dev/sda2 before using luksOpen).
LVM
LVM is great. It offers a lot of flexibility, especially with grow-and shrinkable filesystems like ext[234] and others. But LVM is also an additional layer. This is used to avoid the physical limitations with disks. You can only have one or more partitions per disk and only one filesystem on one partition (well, this restriction is logical and applies to LVM too). So if you have a couple of disks with 500GB and you need a filesystem with 1TB you can't use traditional partitioning, you are limited to the physical disk size. In some OSes you also have a limitation of partitions (slices in Solaris) per physical disks.
With LVM you can combine one or more partitions to one Volume group. So lets say, you have 3 disks sda, sdb and sdc, all 100GB. You use (traditionally) sda as a system partition with 50 GB for /, 5GB Swap and the rest for /home. sdb and sdc are data disks for your large Volume /data. So you combine sdb and sdc into one VG datavg and create a logical volume lv_data (like a partition) onto that VG. Names should be descriptive but don't have to be. If you don't provide a name during LV creation it will be names lvol0, lvol1 and so on.
If you have 2 physical disks as one volumegroup you can also stripe logical volumes onto it - this will increase the performance. Also you could mirror it and can avoid using mdadm. But this is beyond the scope of this writing. Also keep in mind that you can mirror/stripe on the same disk on different partitons (which is useless for the idea itself but can be used to test).
You can add a lot VGs with different physical disks/partitions and on those VGs you can create a lot of LVs.
Note: All of the commands i used can be also used with the high level command lvm and then the subcommands. So the man page for lvm is a great starting point to find out what it will offer to you.
So back the tutorial
pvcreate /dev/mapper/crypt
This create a physical volume on /dev/mapper/crypt. Nice, huh? You could say this by looking at the command. Well, basically it will mark /dev/mapper/crypt as a physical volume to use with LVM. There is not much done here, just the headers for LVM are written to the (in our case) encrypted device. Without encryption you would need to use /dev/sda2.
vgcreate cryptvg /dev/mapper/crypt
This creates the Volume group cryptvg with the physical volume /dev/mapper/crypt. There could be more than one physical volume (e.g. /dev/mapper/crypt1 /dev/mapper/crypt2 or /dev/sda2 /dev/sdb2 without encryption and so on).
In our case we used a cryptvolume, so any data written on that device will be handled through dm_crypt with the appropriate encryption and written onto the physical partition /dev/sda2. So if you don't have the key to unlock the device /dev/sda2 you won't even be able to see what kind of data is on it (well, some people may be able to, but i am not). The physical disk is "splitted" into physical extents (PE), which are small parts of data, usually 4MB. So the smallest unit that can be allocated for a logical volume is 4MB=1PE. But all your fears come true, a logical Volume is almost ever a multiple of this ;) Now we have some kind of container with the ability to store small chunks, to combine them and so on. Which brings us to the next step
lvcreate -L 5G -n rootlv cryptvg
So, now the logical volume is created by allocation those 4MB chunks to build a large "partition". The name is rootlv (for root logical volume - could also be lv_root or just root) and is created on the volume group cryptvg. Wow, life can be so easy. Actually linux don't care much about the partition type, windows is here more strict. For linux the type of "filesystem" is looked up when the filesystem is used. So you can easily put the partition for swap as a logical volume on that volume group. Or another LV for an ext2 filesystem, one for JFS and so on.
For the size you can use -L and give it "human readable" parameters like 5G, 10M or even 1T/P/E (Terra-/Peta-/Exabyte if you have the space). Or you can use -l (lowercase L) to size it from a physical extent perspective (so 100 extents means 400MB).
Logical volumes can be accessed through 3 ways:
- /dev/mapper/vgname-lvname which uses the device mapper notation
- /dev/vgname/lvname which is more like a direct association to the device
- /dev/dm-1 which is the LVM created block device, allocated through a major and a minor number which is noted behind the dm-. The 2 cases above are just links to this one.
The created LVs could now be used as a "raw device" which means, that the application (usually databases) will take care of the allocation of data in it. Or, suited for our purpose of installing an OS to it, put a filesystem on top of it. So here we go
Filesystems
mkfs.ext4 /dev/cryptvg/rootlv
Not much to say here. An ext4 filesystem gets created, using the normal attributes. I used /dev/vgname/lvname, but /dev/mapper/vgname-lvname will work too. You can also use /dev/dm-1 but this is not really straightforward.
The filesystem is most important from the user perspective. It is the thing that gives programs the ability to write, read and manipulate data. I forgot to test it (and i am a bit tired to do now) if you can install BB without creating those filesystems. Actually i got them and this was my focus. But the installer should recognize the volumegroup and the logical volumes itself, so the creation of the filesystems should be optional. Maybe you want to test this, i want to go to bed now :)
Is that all real?
Also i wanted to give you a good oversight about what happened and what means what (for now until the installation) this whole article is still not the reality and some informations maybe even wrong (sorry for that). But it should give enough informations to understand what will be done to your poor disks when using LVM and cryptsetup :) I touched some deeper internals but if you want to really understand whats going on, you will have to get the informations yourself. Starting from physical disk design (heads, sectors and cylinders) over to Bus protocols like SATA or IDE up to the kernel into the userspace. That is a nice travel...which fills thousands of pages i will never write. Its just 1s and 0s - but they change a lot during their trip to your monitor.
So have fun exploring.
The filesystem is most important from the user perspective. It is the thing that gives programs the ability to write, read and manipulate data. I forgot to test it (and i am a bit tired to do now) if you can install BB without creating those filesystems. Actually i got them and this was my focus. But the installer should recognize the volumegroup and the logical volumes itself, so the creation of the filesystems should be optional. Maybe you want to test this, i want to go to bed now :)
Is that all real?
Also i wanted to give you a good oversight about what happened and what means what (for now until the installation) this whole article is still not the reality and some informations maybe even wrong (sorry for that). But it should give enough informations to understand what will be done to your poor disks when using LVM and cryptsetup :) I touched some deeper internals but if you want to really understand whats going on, you will have to get the informations yourself. Starting from physical disk design (heads, sectors and cylinders) over to Bus protocols like SATA or IDE up to the kernel into the userspace. That is a nice travel...which fills thousands of pages i will never write. Its just 1s and 0s - but they change a lot during their trip to your monitor.
So have fun exploring.
Subscribe to:
Posts (Atom)