Archive

Posts Tagged ‘lvm’

Setting up an LVM for Storage

December 30th, 2009 5 comments

Recently, I installed Kubuntu on my PC. Under Windows, I had used RAID1 array to create a storage volume out of two extra 500GB hard drives that I have in my system. Under Linux, I’ve decided to try creating a 1TB LVM out of the drives instead. This should be visible as a single drive, and allow me to store non-essential media files and home partition backups on a separate physical drive, the better to recover from catastrophic failures with. The only problem with this plan: documentation detailing the process of creating an LVM is sparse at best.

The Drive Situation
My machine contains the following drives, which are visible in the /dev directory:

  • sdc: root drive that contains three partitions; 1, 2, and 5, which are my boot, root, and swap partitions respectively
  • sda: 500GB SATA candidate drive that I’d like to add to the LVM
  • sdb: 500GB SATA candidate drive that I’d like to add to the LVM

First Try
Coming from a Windows background, I began by searching out a graphical tool for the job. I found one in my repositories called system-config-lvm 1.1.4.

The graphical tool that I found to create LVMs

I followed the buttons in this tool and created a 1TB LVM spanning sda and sdb, then formatted it with ext3. The result of these steps was an uninitialised LVM that refused to mount at boot. In response, I wrote the following script to activate, mount, and assign permissions to the drive at boot:

#!/bin/bash
sudo whoami
sudo lvchange -a y /dev/Storage/Storage
sudo mount /dev/Storage/Storage /home/jon/Storage
sudo chown jon /home/jon/Storage
sudo chmod 777 /home/jon/Storage

It worked about 50% of the time. Frustrated, I headed over to the #kubuntu IRC channel to find a better solution.

Second Try
On the #kubuntu channel, I got help from a fellow who walked me through the correct creation process from the command line. The steps are as follows:

  1. Create identical partitions on sda and sdb:
    1. sudo fdisk /dev/sda
    2. n to create a new partition on the disk
    3. p to make this the primary partition
    4. 1 to give the partition the number 1 as an identifier. It will then appear as sda1 under /dev
    5. Assign first and last cylinders – I simply used the default values for these options, as I want the partition to span the entire drive
    6. t toggle the type of partition to create
    7. 8e is the hex code for a Linux LVM
    8. w to write your changes to the disk. This will (obviously) overwrite any data on the disk
    9. Repeat steps 1 through 8 for /dev/sdb
    10. Both disks now have partition tables that span their entirety, but neither has been formatted (that step comes later).
  2. Make the partitions available to the LVM:
    1. sudo pvcreate /dev/sda1
    2. sudo pvcreate /dev/sdb1
    3. Notice that the two previous steps addressed the partitions sda1 and sdb1 that we created earlier
  3. Create the Volume Group that will contain our disks:
    1. sudo vgcreate storage /dev/sda1 /dev/sdb1 will create the volume group that spans the two partitions sda1 and sdb1
    2. sudo vgdisplay /dev/storage queries the newly created volume group. In particular, we want the VG Size property. In my case, it is 931.52 GB
  4. Create a Logical Volume from the Volume Group:
    1. sudo lvcreate -L $size(M or G) -n $name $path where $size is the value of the VG Size property from above (G for gigabytes, M for megabytes), $name is the name you’d like to give the new Logical Volume, and $path is the path to the Volume Group that we made in the previous step. My finished command looked like sudo lvcreate -L 931G -n storage dev/storage
    2. sudo lvdisplay /dev/storage queries our new Logical Volume. Taking a look at the LV Size property shows that the ‘storage’ is a 931GB volume.
  5. Put a file system on the Logical Volume ‘storage':
    1. sudo mkfs.ext4 -L $name -j /dev/storage/storage will put an ext4 file system onto the Logical Volume ‘storage’ with the label $name. I used the label ‘storage’ for mine, just to keep things simple, but you can use whatever you like. Note that this process takes a minute or two, as it has to write all of the inode tables for the new file system. You can use mkfs.ext2 or mkfs.ext3 instead of this command if you want to use a different file system.
  6. Add an fstab entry for ‘storage’ so that it gets mounted on boot:
    1. sudo nano /etc/fstab to open the fstab file in nano with root permissions
    2. Add the line /dev/storage/storage    /home/jon/Storage       ext4    defaults        0       0 at the end of the file, where all of the spaces are tabs. This will cause the system to mount the Logical Volume ‘storage’ to the folder /home/jon/Storage on boot. Check out the wikipedia article on fstab for more information about specific mounting options.
    3. ctrl+x to exit nano
    4. y to write changes to disk
  7. Change the owner of ‘storage’ so that you have read/write access to the LVM
    1. sudo chown -R jon:jon /home/jon/Storage will give ownership to the disk mounted at /home/jon/Storage to the user ‘jon’

Time for a Beer
Whew, that was a lot of work! If all went well, we have managed to create a Logical Volume called storage that spans both sda and sdb, and is formatted with the ext4 file system. This volume will be mounted at boot to the folder Storage in my home directory, allowing me to dump non-essential media files like my music collection and system backups to a large disk that is physically separate from my system partitions.

The final step is to reboot the system, navigate to /home/jon/Storage (or wherever you set the boot point for the LVM in step 6), right-click, and hit properties. At the bottom of the properties dialog, beside ‘Device Usage,’ I can see that the folder in question has 869GB free of a total size of 916GB, which means that the system correctly mounted the LVM on boot. Congratulations to me!

Much thanks to the user ikonia on the #kubuntu IRC channel for all the help.

This piece has been mirrored at Index out of Bounds




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

Getting Up and Running with Debian

September 1st, 2009 3 comments

Considering that it was my first experience with Linux, the installation of Debian actually went rather smoothly. I popped in the Debian live disc, rebooted my machine, and said goodbye to Windows XP.

The Basics:

I chose the graphical installer, because I’m a big wimp, and because it makes screenshots. You might ask how the installer saves screenshots to a hard drive that is as of yet unpartitioned, and will be wiped/encrypted during the coming hours… I don’t know either, but I guess that’s a part of the Linux magic. (You don’t get any of these pictures, because I lost them when I reinstalled, and forgot to take more the second time around).

The first few settings were fairly straightforward. Debian asked me to choose my default language, geographical location, and keyboard layout. Debian appears to support some 44 different keyboard layouts on install, including Dvorak, and Canadian Multilingual, which is perhaps the most awful layout ever conceived by man. Is it as bad for the Quebecois as it is for us Ontarians?

Next, the installer attempted to detect my hardware settings, and scanned the live disc for required drivers. At this point, the installer notified me that my system would require non-free firmware files to get my wireless card working. In Linux-speak, non-free simply means that the firmware is distributed as a compiled binary, and that the source code is not available. It is, however, free in the sense that I don’t have to pay a dime to use it, although I have to agree to a license to do so. Given the option to load the firmware files from a disc, or to wait and deal with the problem once the desktop was up and running, I chose the latter.

The next step was for the Debian installer to attempt to auto-configure my DHCP settings, and to use my ethernet card to connect to the internet. Since the laptop wasn’t plugged in to an ethernet cable, it didn’t really surprise me that this step failed. I chose to configure the network later, and moved on to giving my machine a name and choosing my timezone instead.

Partitions and Full-Disc Encryption:

When it came time to partition my disks, I chose to take a shot at full-disc encryption. The most basic Linux drive has two partitions – one called /boot that is generally formatted with ext2 and takes the place of Window’s boot sector, and another called / that contains the rest of your data, including the OS. Once the BIOS has finished all of it’s startup checks and initializations, it hands off to GRUB, which is stored on the /boot partition. GRUB does some other stuff, and then boots the operating system, which is stored on the secondary partition, usually formatted with one of the many available file systems that Tyler covered in detail in a previous post. There is a great explanation of the entire boot process available here.

To allow for multiple partitions, Linux utilizes some fancy software called the LVM (logical volume manager), which virtualizes any partitions that you create within the big main one. When enabling full-disc encryption, everything inside of the LVM (all of the partitions except for /boot, because the machine needs to be able to start) is encrypted as it is written to disc, and decrypted as it is read from disc. This method of protecting your data is extremely secure, as the encryption is transparent to the user and operating system, while every file on the system remains encrypted until the correct password is provided by the user.

Debian allows me to put each of the important parts of my root directory on separate drive partitions. For example, I can separate the /home partition from the /usr, /var, and /tmp partitions within the LVM. This would be extremely handy if my machine were running multiple physical discs, and I wanted to put my install on a separate disk from my data so that backups and reinstalls are less painful. Because I’m new at this and have only one disc, I chose to put all of my files in a single partition.

Before proceeding with installation, Debian tried to zero all existing data on my drive. Since that data wasn’t at all sensitive, my hard drive is small, and I don’t care to wait years for the wiping process to finish, I hit the cancel button in the bottom-right hand corner of the screen, which allowed me to skip to the next step. I actually found this out by accident, fully expecting the cancel button to boot me right out of the install process. Silly UI design, that.

The last step in partitioning my drives was to provide a password for the full-disc encryption, and to choose the file system for each of my newly created partitions. As previously noted, my /boot partition is formatted with ext2, and the LVM is using a filesystem called crypto, which I assume is just the name of the encrypted partition container. Linux also creates a root partition for me (located at /), which I’ve chosen to format with the ext3 filesystem, since ext4 does not appear to be supported by my installer. Finally, a partition called /swap is created (the equivalent of the Window’s swap file), that is formatted with the (what else?) swap file system.

It should be noted that the partition manager screen also had a strange UI bug in it – the continue button that had been my friend and companion thus far throughout the install process ceased to have any meaningful functionality. I had to choose to ‘finish partitioning and write changes to disc’ from the partition manager menu before I could continue with the installation.

Just About There:

With all of the setup options behind me, the Debian installer helpfully finished the install all on it’s own, pausing only to demand that I enter a root password, a default user account name and user account password. It should be noted that if you intend to become a l337 system administrator, your root password should be hard to guess but easy to type, as you’ll be forced to enter it whenever you do an action that is outside of the user account security privileges (or in other words, essentially anything of consequence).

Lastly, the installer asked if I wanted to enable the Debian package popularity contest (popcon), and which default software I wanted to install. I chose to add a web server, file server, and SQL database to the default install. That done, the installer went on it’s merry way and actually got down to the business of installing my distribution.

Adding the Tubes (Or Not):

Remember how the Debian installer failed to auto-detect my DHCP settings because my laptop wasn’t plugged into an ethernet cable? Well it also “forgot” to install my ethernet card driver at the same time. Since the machine doesn’t have a network connection, I have no access to the Debian repositories from which I can get the required drivers, but I can’t seem to get them without access to those repositories. I found the driver in question here, but have no idea what to do with the driver once I get it, because it is distributed as a *.rpm package, which is the Fedora package format, and unsupported by Debian. I’ve found various discussions on the Debian website that reveal that tg3, the driver for my network card, was removed from the Debian package, because it is not “free” in the sense that it is distributed as a compiled binary, and not as source.

After spending a half hour scouring the GOOG for instructions on how to install this driver, only to come up empty handed, I’ve decided to simply reinstall, but to plug the ethernet cable in this time, and hope that it works better than it did last time. If anybody knows what the hell happened, I’d love to hear an explanation, and perhaps a method by which I can fix the problem.

Back from the Reinstall:

So after waiting for Debian to reinstall, I got back up and running, and just needed my wireless card active so that I could put my ethernet cable back in my other computer. Luckily, the steps to get it working are very straightforward:

  1. Open the synaptic package manager
  2. Under the settings menu, select repositories, and select the DFSG and Non-DFSG checkboxes. This will allow synaptic to install packages that are non-free.
  3. Search for a package called firmware-ipw2x00, and install it
  4. Agree to the firmware license
  5. Open a root terminal
  6. Type “modprobe ipw2200″ to load the firmware kernel module
  7. Navigate to the etc/ directory, and open the file “modules” in the nano editor
  8. add the line “ipw2200″ to the end of the file
  9. Hit ctrl-x to close the file, and Y to save the changes.
  10. Restart your machine

Once your machine restarts, the wireless firmware will be loaded, and you should be able to click on the network icon in the taskbar and select an available wireless network to connect to.

So Everything is Cool, Right?

Well, not quite. My system is up and running, supports full-disc encryption, wired and wireless networking; but the GNOME desktop bugs the hell out of me, and so far as I can tell, I don’t have a working sound card yet… More on that one tomorrow. The moral of this story is as follows: NEVER install Debian without a network connection present. It’s just not worth your time.