Archive

Posts Tagged ‘Linux’

Compile Windows programs on Linux

September 26th, 2010 No comments

Windows?? *GASP!*

Sometimes you just have to compile Windows programs from the comfort of your Linux install. This is a relatively simple process that basically requires you to only install the following (Ubuntu) packages:

To compile 32-bit programs

  • mingw32 (swap out for gcc-mingw32 if you need 64-bit support)
  • mingw32-binutils
  • mingw32-runtime

Additionally for 64-bit programs (*PLEASE SEE NOTE)

  • mingw-w64
  • gcc-mingw32

Once you have those packages you just need to swap out “gcc” in your normal compile commands with either “i586-mingw32msvc-gcc” (for 32-bit) or “amd64-mingw32msvc-gcc” (for 64-bit). So for example if we take the following hello world program in C

#include <stdio.h>
int main(int argc, char** argv)
{
printf(“Hello world!\n”);
return 0;
}

we can compile it to a 32-bit Windows program by using something similar to the following command (assuming the code is contained within a file called main.c)

i586-mingw32msvc-gcc -Wall “main.c” -o “Program.exe”

You can even compile Win32 GUI programs as well. Take the following code as an example

#include <windows.h>
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow)
{
char *msg = “The message box’s message!”;
MessageBox(NULL, msg, “MsgBox Title”, MB_OK | MB_ICONINFORMATION);

return 0;
}

this time I’ll compile it into a 64-bit Windows application using

amd64-mingw32msvc-gcc -Wall -mwindows “main.c” -o “Program.exe”

You can even test to make sure it worked properly by running the program through wine like

wine Program.exe

You might need to install some extra packages to get Wine to run 64-bit applications but in general this will work.

That’s pretty much it. You might have a couple of other issues (like linking against Windows libraries instead of the Linux ones) but overall this is a very simple drop-in replacement for your regular gcc command.

*NOTE: There is currently a problem with the Lucid packages for the 64-bit compilers. As a work around you can get the packages from this PPA instead.

Originally posted on my personal website here.




I am currently running a variety of distributions, primarily Linux Mint 18.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Feel free to visit me at my personal website here.
Categories: Tyler B, Ubuntu Tags: , , , ,

Trying out the Chakra Project

August 24th, 2010 1 comment

After a little bit of pressure from the people responding to my previous post (My search for the best KDE Linux distribution), I have finally given in and tried out Chakra. The Chakra Project starts with Arch Linux as a base but, instead of forcing you to build your own distro piece of piece, Chakra comes more or less pre-packaged.

Installation

The installation was one of the best I’ve ever seen. For alpha software this distribution’s first point of interaction is already very polished – even warning me that it is not stable software and might therefore eat my hamster.

The install process even let me decide to install some very useful packages, like Microsoft Core TTF Fonts and Adobe Flash, right away. Even the Language & Time step was incredible, offering a rotating globe that I could drag around and manipulate.

The only issue I had was trying to create a disk partition to install the OS to. This was because I was trying this out inside of VirtualBox, and the virtual hard disk did not have any partitions on it whatsoever. There is a bug and (thankfully) work-around for this known issue with their Tribe installer, and after reading a quick walk-through I was once again ready to install.

The Desktop

The desktop is standard KDE version 4.4.2 after install. Opening up Pacman (or is it Shaman?) showed me a list of brand new software that I could install, including the newest KDE 4.5. One of Project Chakra’s great strengths will be in this rolling release of new software updates. The concept of installing once and always having the most up-to-date applications is very intriguing.

Unfortunately, as with most alpha software, Shaman is still pretty buggy and often crashed whenever I tried to apply the updates. Also unfortunate is that Shaman started a trend of applications simply crashing for no reason. I don’t want to give this distribution a bad reputation, because it is still pre-release software, but I think it goes without saying that the developers have some bug squashing to do before a stable release will be ready. Something I found rather strange is that the current default software selection that Chakra ships with includes two different browsers, Konqueror and rekonq, but no office software whatsoever.

Google Chrome much?

Final Thoughts (for now!)

The Chakra Project looks very promising, albeit very unpolished at the moment. If they can manage to fix up the rest of the distribution, getting it just as polished feeling as the installer, this will definitely be one to look out for. I look forward to trying it out again once it hits a stable release.




I am currently running a variety of distributions, primarily Linux Mint 18.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Feel free to visit me at my personal website here.
Categories: KDE, Linux, Tyler B Tags: , , ,

A Matter of Opinion

July 19th, 2010 No comments

Tonight I installed VirtualBox, an incredibly handy virtualization program that lets me run instances of Windows and other Linux distributions from the comfort of my Linux Mint 9 Isadora desktop. Upon installing the latest version in my repositories, I launched the program, only to be confronted by a dialog box offering a link to a newer version of the program available on its website. So I clicked the link, and downloaded the *.deb of the new version. My package manager started up, tried to install the new package, and complained that it conflicted with the existing VirtualBox install. So I opened synaptic, uninstalled the version of VirtualBox that I got from my repositories, and finally installed the most recent version from the website.

So here’s my question, and please feel free to leave your opinion in the comments below: Should Linux applications warn the user about updates that are not available from their repositories?

On one hand, I like having up to date software, but on the other, package maintainers work hard to ensure that everything that ships with a stable distribution plays well together, and probably don’t appreciate these apps leading users outside of their carefully curated repositories. From a security-oriented point of view, this is also bad practice, as much of the security that is inherent in Linux comes from the fact that the vast majority of the software that you install has been vetted by the package maintainers who work to ensure that your distribution is safe and stable. And surely the guys who program VirtualBox, being the insanely awesome ninja-powered pirate wizards that they are, could have come up with a way to update my install without my having to uninstall and re-install an entirely new version. Just sayin’

Chime in with your opinion in the comments below.




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

PulseAudio: Monitoring your Line-In Interface

July 11th, 2010 22 comments

At home, my setup consists of three machines –  a laptop, a PC, and an XBOX 360. The latter two share a set of speakers, but I hate having to climb under the desk to switch the cables around, and wanted a better way to switch them back and forth. My good friend Tyler B suggested that I run the line out from the XBOX into the line-in on my sound card, and just let my computer handle the audio in the same way that it handles music and movies. In theory, this works great. In practice, I had one hell of a time figuring out how to force the GNOME sound manager applet into doing my bidding.

After quite a bit of googling, I found the answer on the Ubuntu forums. It turns out that the secret lies in a pulse audio module that isn’t enabled by default. Open up a terminal and use the following commands to permanently enable this behaviour. As always, make sure that you understand what’s up before running random commands that you find on the internet as root:

pactl load-module module-loopback
sudo sh -c ' echo "load-module module-loopback" >>  /etc/pulse/default.pa '

The first line instructs PulseAudio (one of the many ways that your system talks with the underlying sound hardware) to load a module called loopback, which unsurprisingly, loops incoming audio back through your outputs. This means that you can hear everything that comes into your line-in port in real time. Note that this behaviour does not extend to your microphone input by design. The second line simply tells PulseAudio to load this module whenever the system starts.

Now if you’ll excuse me, I have jerks to run over in GTA…




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

Big distributions, little RAM 2

July 5th, 2010 5 comments

As a follow up to my previous post I have decided to re-run the tests, this time with the updated distributions (where available of course). Again I will be testing all of this within VirtualBox on ‘machines’ with the following specifications:

  • Total RAM: 512MB
  • Hard drive: 8GB
  • CPU type: x86

The tests were all done using VirtualBox 3.2.6 on Windows, and I did not install VirtualBox tools (although some distributions may have shipped with them). I also left the screen resolution at the default 800×600 and accepted the installation defaults. All tests were run on July 3rd, 2010 so your results may not be identical.

Results

As before I have provide state of the art graphs for your enjoyment.

First boot memory (RAM) usage

This test was measured on the first startup after finishing a fresh install.

Memory (RAM) usage after updates

This test was performed after all updates were installed and a reboot was performed.

Memory (RAM) usage change after updates

The net growth or decline in RAM usage after applying all of the updates

Install size after updates

The hard drive space used by the distribution after applying all of the updates.

Conclusion

As before I’m going to leave you to drawing your own conclusions. I will point out though that almost all of the distributions have done a good job of lowering memory usage with system updates, which is very commendable. Also it’s important to note that even though RAM and disk space increase with updates so might performance so it’s all about which metric you hold as most important.




I am currently running a variety of distributions, primarily Linux Mint 18.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Feel free to visit me at my personal website here.
Categories: Linux, Tyler B Tags: , , ,

Accessing Windows 7 Shares from Ubuntu is a Pain

June 28th, 2010 16 comments

This blog post is about my experiences. If you hit this page from a search engine looking to fix this issue click here to skip to the solution.

Recently, I’ve been reorganizing my computers based on their usage. My old HTPC, has resumed its duties as my primary desktop/server, my Mac Mini has been attached to the my desktop through Synergy, my server was given to my brother for personal use, and his old computer – a nettop – is now being used as our new HTPC.

After a painful decision making process – a topic for another time, and another post – I decided that this nettop, named Apollo after the Greek god of many things including “music, poetry, and the arts” [as close as I could get to entertainment],  should run Ubuntu 10.4 with XBMC as the media center app. After testing it’s media playback capabilities from a local file, I was rather impressed. I set out to add a SMB share from within XBMC, and was prompted to add a username and password.

I wasn’t really expecting this, because Leviathan – my desktop/sever running Windows 7 – has public sharing turned on, as well as a guest account. I entered in my credentials, and was asked yet again for a username and password. After trying multiple times, I decided to quit XBMC and see if I could get Ubuntu to connect to the share. Here too, I was prompted for a username and password, again and again.

Next I headed to the terminal to run smbclient. This didn’t work either, as I was shown a message saying smbclient failed with “SUCCESS – 0”. I guess success shouldn’t be zero, so my next move was to attempt mounting the network share using CIFS. Again, I was met with repeated defeat.

Begrudgingly I took to the internet with my problem, only to find that there were many people unable to connect to their Windows 7 from Ubuntu. The suggestions ranged from registry hacks to group policy administration, none of which worked. One repeated suggestion however, was to un-install the Windows Live Sign-in Assistant. However, as a user of the Windows Live Essentials (Wave 4) Beta that was recently released – I had no such program. I did however have a similar application called the Windows Live Messenger Companion, which I chose to uninstall – again, to no avail.

However, I soon reasoned that perhaps whatever was blocking people using the Windows Live Sign-in Assistant was now being used within the actual Windows Live Messenger client or the other Windows Live Essentials apps that I’d recently installed. I started by uninstalling everything but Windows Live Messenger – because I really, really like the beta version. Alas, this did not help. Next I uninstalled the actual Windows Live Messenger client and voila – I was able to connect with no prompting for passwords at all. Because that makes -any- sense.

As a matter of interest, I installed the regular WLM non-beta client and made sure that the Windows Live Sign-in Assistant was installed, and tried to connect again. Not surprisingly, I was no longer able to connect to my Windows 7 shares. After un-installing the Windows Live Sign-in Assistant my shares were back up and I was mostly happy. Except that I couldn’t use the new Windows Live Messenger beta.

I can’t be sure if the other tinkering I did also helped clear up my problems, but as a recap here are the steps I recommend to access your Windows 7 shares from Ubuntu:

1) If you have the Windows Live Essentials (Wave 4) beta installed, you’ll have to uninstall all of the applications that come with this. For now, you can install the current version of Windows Live Messenger and the other Windows Live Essentials.

2) If you have Windows Live Messenger installed, or ANY of the Windows Live Essentials programs installed check to see if you have the Windows Live Sign-in Assistant installed. If so, uninstall it.

3) Hopefully, now you can enjoy your Windows 7 shares in Ubuntu

Important Note:

Beta software has this nasty habit of leaving beta status sooner or later. If this issue is not resolved when the newest version of Windows Live Messenger is officially released, you may not be able to use the Window Live Messenger client if you need your Windows 7 shares from Ubuntu. I would suggest using an application like Pidgin as your instant messenger, as it can also connect to the Windows Live Messenger service. Other options include Digsby, Miranda, and Trillian.

Originally posted on my personal website here.

Fix ATI vsync & video tearing issue once and for all!

May 6th, 2010 23 comments

NOTE: ATI’s most recent drivers now include a no tearing option in the driver control panel. Enabling it there is now the preferred method.

Two of the linux machines that I use both have ATI graphics cards from the 4xxx series in them. They work well enough for what I do, very casual gaming, lots of video watching, but one thing has always bothered me to no end: video tearing. I assumed that this was due to vsync being off by default (probably for performance sake) but even after installing the proprietary drivers in the new Ubuntu 10.04 and trying to force it on I still could not get the issue to resolve itself. After some long googling I found what seems to be a solution, at least in my case. I’ll walk you through what I did.

Before you continue read this: In order to fix this issue on my computers I had to trash xorg.conf and start over. If you are afraid you are going to ruin yourself, or if you have a custom setup already, please be very careful and read before doing what I suggest or don’t continue at all. Be sure to make a backup!

1 ) Install the ATI proprietary drivers and restart so that they can take effect.

2 ) Make a backup of your xorg.conf file. Do this by opening a terminal and copying it to a backup location. For example I ran the following code:

sudo cp /etc/X11/xorg.conf /etc/X11/backup.xorg.conf

3 ) Remove your existing (original) xorg.conf file:

sudo rm /etc/X11/xorg.conf

4 ) Generate a new default xorg.conf file using aticonfig (that’s two dashes below):

sudo aticonfig –initial

5 ) Enable video syncing (again two dashes before each command):

sudo aticonfig –sync-video=on –vs=on

6 ) If possible also enable full anti-aliasing:

sudo aticonfig –fsaa=on –fsaa-samples=4

7 ) Restart now so that your computer will load the new xorg.conf file.

8 ) Open up Catalyst Control Center and under 3D -> More Settings make sure the slider under Wait for vertical refresh is set to Always On.

That should be it. Please note that this trick may not work with all media players either (I noticed Totem seemed to still have some issues). One other thing I tried in VLC was to change the video output to be OpenGL which seemed to help a lot.

Good luck!




I am currently running a variety of distributions, primarily Linux Mint 18.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Feel free to visit me at my personal website here.

Linux Media Players Suck – Part 1: Rhythmbox

May 5th, 2010 52 comments

The state of media players on Linux is a sad one indeed. If you’re a platform enthusiast, you may want to cover your ears and scream “la-la-la-la” while reading this article, because it will likely offend your sensibilities. In fact, the very idea behind this series is to shake up the freetards’ world view, and to make them realize that a decent Winamp or iTunes clone need not be the end of the story for media management and playback on Linux.

This article will concentrate on lambasting Rhythmbox, the default jukebox software of the GNOME desktop environment. Subsequent posts will give the same treatment to other players in this sphere, including Banshee, Amarok, and Songbird (if I can find a copy that will still build on Linux). If you’re a user of media players on Linux, keep your own annoyances firmly in mind, and if I don’t mention them, please share in the comments. If you’re a developer for one of these fine projects, try to keep an open mind and get inspired to do better. A media player is not a hard thing to build, and I do believe that together, we can do better.

For the remainder of this article, please keep in mind that I am currently running Rhythmbox under Kubuntu 9.10, so you’ll see it rendered with qt widgets in all of my screen shots. This doesn’t affect the overall performance of the app, but leads nicely into my first complaint:

  1. Poor Cross-Platform Support: There are basically two desktop environments that matter in the Linux world, GNOME and KDE. Under GNOME, Rhythmbox has a reasonably nice icon set that is comparable to other media players. Under KDE, the qt re-skinning replaces those icons with a horrible set of mismatched images that really make the program look second-rate:
    Isn't this shit awful?

    As you can see, these icons look terrible. Note that there isn't even an icon for 'Burn' and the icon for 'Browse' is a fucking question mark.

    This extends to the CD burning and help features too. They rely on programs like gnome-help and brasero to work, but don’t install them with the media player, so when I try to access these features under KDE, I just get error messages. Nice.

    Honestly, who packaged this thing?

    This is just plain stupid. Every package manager has the concept of dependencies, so why doesn't Rhythmbox use them?

  2. The Player Starts in the Tray: Under what circumstances would it be considered useful for a media player to automatically minimize itself to the system tray on startup? It doesn’t begin to play automatically. The first thing that I always do is click on the tray icon to maximize it so that I can select some music to start playing. Way to start the user experience off on the wrong foot.
  3. Missing Files View: This one is just plain stupid. Whenever I delete a file from my hard drive, it shows up under the ‘Missing Files’ view, even though my intent was clearly to remove the file from my library. Further, I use Rhythmbox to put music on my BlackBerry. Whenever I fill it with music, I first delete the files on it. Those files that I deleted from my mobile device? Yeah, they show up under ‘Missing Files’ too, as if they were a legitimate part of my library! So this view ends up being like a global garbage bin that I have to waste my precious time emptying on occasion, and serves no useful purpose in the mean time. Yeah, I deleted those files. What are you going to do about it?

    Seriously, why the hell are these files in here?

    As you can see, I've highlighted the fact that Rhythmbox is telling me that these files are missing from my mobile device. No shit.

  4. Shared Libraries that I can’t Play: So we’ve known for awhile now that Apple broke the ability to connect to iTunes via the DAAP protocol, and that it’s not possible to connect to a shared iTunes library from Linux. If that’s the case, why does Rhythmbox still show these libraries as available? And how come it shows my library under this node? Why would I listen to my own shared library? Finally, I’ve found that even if I’m running Rhythmbox on another machine, I still can’t connect to my shared library. This feature seems to be downright broken – so why is it still in the build?
  5. The GUI and Backend are on One Thread: I keep about half of my music collection as lossless FLAC files. When I want to rip these files to my portable media device, they need to be converted to the Mp3 format. Turns out that Rhythmbox thinks it appropriate to transcode these files on the same thread that it uses to update its GUI, so that while this process is taking place, the app becomes laggy, and at times, downright unusable. Further, the application doesn’t seem to give me any control over the bitrate that my songs are transcoded to. Fuck!
  6. Lack of Playlist Options: Smart playlists in Rhythmbox are missing a rather key feature: Randomness. When filling the aforementioned mobile device with music, I would like to select a random 4GB of music from my top rated playlist. But I can’t. I can select 4GB of music by most every criteria except randomness, which means that I get the same 600 or so songs on my device every time I fill it. This is strange, because I can shuffle the contents of a static playlist; But I cannot randomly fill a smart playlist. Great.

    If you have a device that has a small amount of memory, this feature is essential

    It's funny; I really want to like Rhythmbox, but it's shit like this that ruins the experience for me

  7. Columns: What the fuck. Who wrote this part of the application? When I choose the columns that are visible in the main window, I can’t re-order them. That’s right. So the only order that I can put my columns in is Track, Title, Genre, Artist, Album, Year, Time, Quality, Rating. Can’t reorder them at all, and I have to go into the preferences menu to choose which ones are displayed, instead of being able to right-click on the column headers to select them like I can in every other program written in the last 10 years. This is just ridiculous. I know that the GTK+ toolkit allows you to create re-order-able columns, because I’ve seen it done.

    This is just so incredibly backward. I mean, columns are a standard part of the GTK+ toolkit, and I've seen plenty of other apps that do this properly.

    Why, for the love of God, can't these be re-ordered?

  8. The Equalizer is Balls: No presets, and no preamp. So I can set the EQ, and my settings are magically saved, but I can only have one setting, because there doesn’t appear to be a way to create multiple profiles. And louder music sounds like balls, because I can’t turn down the preamp, so I get digital distortion throughout my signal. It would be better to just not have an equalizer at all.

    I mean, it works. But...

    I mean, it works. But...

  9. Context Menus Don’t Make Sense: Let’s just take a look at this context menu for a moment. There are three ways to remove a song from a playlist. You can Remove the song, which just removes it from the playlist, but not from your library or your hard drive. Alternatively, you can select Move to Trash, which does what you might expect – it removes the song from the playlist, the library, and your computer. I’ve got a problem with the naming conventions here. The purpose of Remove isn’t well explained, and confused the hell out of me at first. In addition, when browsing a mobile device that you’ve filled with music, the GUI breaks down even further. In this case, you can still hit Remove, which seems to remove the song from Rhythmbox’s listing, but leaves the file on the device. So now I have a file on my device that I can’t access. Great. The right-click menu also has the ability to copy and cut the song, even though there is no immediately obvious way to paste it. For that functionality, you’ll have to head up to the Edit menu.

    The right-click context menu

    I'm starting to run out of anger. The 10,000 papercuts that come along with this app are making me numb to it.

  10. No Command Line Tools: Now, normally, this wouldn’t bother me too much. A music library is something that’s meant to have a GUI, and doesn’t generally lend itself to working from the command line. In this case however, command line access to Rhythmbox would be really handy, because I’d like to set up a hot key on my keyboard that will skip songs or pause playback. Unfortunately, there’s no way to do that within the software, and it doesn’t have any command line arguments that I can call instead. Balls.

There you have it, 10 things that really ruin the Rhythmbox experience. While using this piece of software, I felt like the developers worked really hard to build something that was sort of comparable to Apple’s iTunes, and then stopped trying. That isn’t good enough! If we want to attract users to our platform of choice, and keep them here, we need to give them reasons to check it out, and even more to stick around. If I say to you that I want to have the best Linux media player, you tend to put the emphasis on the word Linux. Why not just make the best media player? GNOME is on at least half of all Linux desktops, if not more. Why hinder it with software that gives people a poor first impression of what Linux is capable of? Seriously guys, let’s step it up.




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

Empathy: What a Piece of Garbage

March 20th, 2010 17 comments

The Empathy instant messaging client for Gnome is not yet ready to be the default client on your favourite Gnome-based distribution. In fact, I can’t even make it work! Tyler B originally posted about this problem way back in October, but it doesn’t seem to have been fixed during the interim.

To demonstrate my point, allow me to walk you through the process of adding an MSN account, one of the officially supported protocols, to a clean install of Empathy:

  1. After launching Empathy, select Accounts from the Edit menu:

    The accounts manager for Empathy

    Hey guys, nice UI. Way to give that listbox a default width. And why the hell is this dialogue box so big, anyway?

  2. Select the MSN protocol from the dropdown menu, and hit the create button:

    The list of protocols that Empathy "supports"

    Wow, way to get icons for every protocol, guys. Either have icons, or don't, ok?

  3. Enter your MSN email address and account password, and hit the Connect button:

    Adding my account details to the new MSN account in Empathy

    Hey, see that Add button under the listbox? If I click that, I can add a new account, before even finishing with this one! Wow, recursion in a GUI! Sweet!

  4. With the new account created, hit the Close button, and watch as the authentication of your newly added MSN account fails:

    Authentication of my newly added account failed

    Wouldn't you know it, my freshly minted account failed to authenticate. I wonder what the problem is...

  5. Hit the Edit Account button, and open up the Advanced area of the Account Manager window that pops up:

    The Advanced area of the Account Manager window in Empathy

    Have you ever seen anything communicate over port 0? I haven't

  6. Open up your working copy of the trusty Pidgin instant messenging client, put the correct port number into the Port textbox in Empathy, and try to figure out how to save your changes:

    Empathy notifies me that I have unsaved changes

    Since I couldn't click apply, I hit Close. Empathy warned me that I hadn't saved my changes, and only then enabled the Apply button in the Account Manager window... Fuck me

  7. Watch as, even with the correct Server and Port information, Empathy continues to fail miserably at connecting to an MSN account:

    The contact list again

    Hey, it's still failing to connect. Imagine that.

The Bottom line? This application is buggy, untested, incompatible, falsely-advertised garbage. I want my Pidgin back. It may have some rough edges, but at least it connects. How these glaring errors and this horrible GUI design ever got past the community is beyond me. I do hope that Empathy has something even somewhat mediocre up their sleeves for their 2.29 release, but until then, I’m headed back to Pidgin.




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

Big distributions, little RAM

March 13th, 2010 19 comments

As you know I am currently running OpenSUSE 11.2 on my laptop. While I have enjoyed my time using it, I have noticed that this particular distribution tends to be on the heavy side of memory usage. This got me thinking. If OpenSUSE uses this much memory on my machine, how could it possibly run on a machine with 512MB of RAM (the lowest recommended amount)? If Ubuntu is the most popular distribution, but it is also, what I would call, a fully-fledged desktop distribution, then how does it manage given tighter memory constraints? And so the mini-experiment begins.

Points to make before I begin

  • This is not a very scientific study, but rather something I did in my spare time because I was curious.
  • I have picked the majority of the most popular desktop distributions. These distributions were chosen not because they were designed for minimal system specs but rather because they are popular and provide a full desktop experience out of the gate.
  • What do I mean by full desktop experience? The distribution should be easy enough for a novice Windows user to install, should come with all of the standard software for desktop activities, and should not require any fine tuning.
  • What you won’t find here: DSL, Arch Linux, Slackware (only because it failed at installing in VirtualBox), Gentoo, or other ‘expert’ distributions. You also will not find netbook remixes or low-resource specific distributions. This experiment is designed to see how these big distributions run on little RAM, nothing more.
  • Please do not post things like ‘you forgot to test XYZ’ or other useless comments that don’t actually help the discussion. Yes I am sorry I missed your favourite distribution, but grab a tissue, clear the tears from your eyes and let’s all move on with our lives.

How I tested them

The process was identical for all tested distributions. I set up a new virtual machine inside of VirtualBox with the following specs:

  • Total RAM: 512MB
  • Hard drive: 8GB
  • CPU type: x86

The tests were all done using VirtualBox 3.1.4 on Windows, and I did not install VirtualBox tools (although some distributions may have shipped with them) nor did I change the screen resolution from the default 800×600.

Results

I have broken the results down into a variety of categories and included fancy graphs just for you!

First boot memory (RAM) usage

For this test I installed the distribution and then on its first (post-installation) boot measured the amount of memory it used. This was to gauge the amount of resources that the stock distribution required before any updates.

Average first boot memory (RAM) usage by packaging type

This shows the average memory usage broken down by the packing type used.

Memory (RAM) usage after updates

This was a test to see whether or not system updates caused the memory usage to increase or decrease. I updated the system with all current updates and then rebooted and measured the resource usage again.


Memory (RAM) usage change after updates

This graph shows the usage difference between installation and post-updating. The formula I used was [after updates – initial installation].


Average memory (RAM) usage after updates by packaging type

Similar to above. Again this is broken down by packing type.

Filesystem layout

This is a simple graph showing the partitions that each default setup created as well as the relative size of them.

Filesystems used in partitions

This graph shows the different filesystems used for the various partitions. For example if a distribution has a value of 2 under ext4 that means that it used ext4 in two different partitions.

Occurrence of filesystem by packaging type

This graph shows the number of distributions who used a certain type of filesystem. It is broken down by packing type.

Install size after updates

This is the total OS install size after downloading and installing all of the updates. This should represent a fully updated version of the distribution.

Average install size after updates by packaging type

This shows the average install size of the distributions broken down by packing type.

Conclusion

Make you own! …well it is pretty obvious that some of these distributions would perform better than others given these low system specs. There are however other things to consider. For example which packing type you prefer, or for that matter which package manager.




I am currently running a variety of distributions, primarily Linux Mint 18.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Feel free to visit me at my personal website here.
Categories: Linux, Tyler B Tags: , , ,

Installing Apache on Ubuntu 9.10

March 9th, 2010 2 comments

Tonight I decided that I’d like to be able to do some web development from home. The basic suite is called LAMP, which stands for Linux, Apache, MySQL, and PHP; the standard web developers toolkit. After a little bit of googling, I found this great guide from Tux Tweaks that walked me through the entire process. Once installed, my system hosted any files in the /var/www/ directory, and had MySQL and phpMyAdmin installed for database access.

Enjoy!




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

My search for the best KDE Linux distribution

March 3rd, 2010 41 comments

As some of you already know, I am a big fan of the KDE desktop environment (or KDE Workspaces or whatever they’re calling it these days). In my search to reach Linux KDE perfection I have tested out a number of different distributions. First there was Fedora, which I happily ran throughout the length of the experiment. Once that was finished I attempted to install and try both Kubuntu and openSUSE. Unfortunately I was unable to do so after openSUSE decided not to play nice. However my search did not stop there, and once the community edition was ready I jumped over to Linux Mint KDE CE. Finally I decided to once again try openSUSE, this time installing from a USB drive. This somehow resolved all of my installation issues.

Now that I have tried out quite a few of the most popular distributions I figured I would write a little bit to tell you fine people my thoughts on each, and why I will be sticking with openSUSE for the near future.

Fedora 11

  • KDE Version: 4.2 – 4.3
  • Pros: very secure, not too many modifications of the KDE source, cutting edge
  • Cons: could have really used some more modifications of the base KDE packages in order to better integrate GTK+, Bluetooth problems, not always stable
  • Thoughts:

    I have written at length about my experiences with Fedora during this experiment. Without re-writing everything again here let me simply say this: Fedora is primarily a GNOME distribution and I could never shake the feeling that KDE got the left-over treatment.

Kubuntu

  • KDE Version: 4.3
  • Pros: very easy to use, nice integration of GTK+ and GNOME notifications, access to Ubuntu support
  • Cons: the hardware drivers application (jockey) simply did not work, very bad sound issues, Firefox could not handle opening file types
  • Thoughts:

    When I first installed Kubuntu I was thrilled. Ah, this must be what it’s like to use a real KDE distribution, I thought. Everything seemed smoother and far more integrated then it did in Fedora. For example: OpenOffice.org had a KDE theme and it’s file browser actually used the native KDE one. Furthermore the notification system was awesome. Now instead of a GNOME application, like Pidgin, generating GNOME notifications, it instead integrated right into the standard KDE equivalent.

    Then the problems started to show up. Oh I’ll just download this torrent file and… hmm Firefox doesn’t seem to know what to do with it. Why can’t I set the file type options inside of Firefox for torrents? Why doesn’t it use the system defaults? Then the sound issues came. YouTube stopped putting out audio all together and all of my attempts to fix it were futile. Maybe it’s just my hardware but Kubuntu just could not handle multimedia at all.

    While Kubuntu is definitely one of the better KDE experiences it is by no means problem free.

Linux Mint KDE CE

  • KDE Version: 4.3
  • Pros: excellent package manager, easy to use
  • Cons: sound issues, WiFi issues, is this actually a KDE desktop? there are so many GTK+ applications in it…
  • Thoughts:

    After hearing much praise for Linux Mint I decided to give the newly released KDE community edition a go. I must say at first I was very impressed. The package manager was far superior to KPackageKit and even included things like user ratings and comments. It also came bundled with many tools and applications designed specifically for Linux Mint. Sadly very few of these were re-written in Qt and so I was forced to deal with GTK+ skinning almost everywhere.

    Sound issues similar to those in Kubuntu (maybe it’s something in the shared source?) started to crop up almost immediately. Again YouTube just did not work no matter how much I tried to fix it. Finally the WiFi connection was very poor, often disconnected on what seemed like a  specific interval.

    While I think this distribution has a lot going for it I can only suggest the GNOME desktop for those who want to give it a try. The KDE version just does not seem polished enough to be recommended for someone looking for the ultimate KDE distribution.

openSUSE

  • KDE Version: 4.3
  • Pros: very responsive, a lot of streamlined tweaks, rock solid WiFi, excellent audio
  • Cons: slower to boot, uses quite a bit of RAM, too much green 😛
  • Thoughts:

    Installing openSUSE seemed like an awful idea. After reading all of the complaints that both Phil and Dave had written over the course of the experiment I have to admit I was a little hesitant. However, I am very happy I decided to try it anyway; openSUSE is an excellent KDE distribution.

    Everything about it, from the desktop to the little helpful wizards, all seem to be designed with one purpose in mind: make openSUSE the easiest, or at the very least most straightforward, distribution possible. YaST, often a major source of hate from my fellow Guinea Pigs, does indeed have some quirks. However I honestly think that it is a very good tool, and something that streamlines many administrative tasks. Want SAMBA network sharing? Just open up YaST and click on the wizard. Want restricted codecs? Just hop on over to openSUSE-Community and download the ymp file (think of it like a Windows exe).

    My time with openSUSE so far has been wonderful. My network card seems to actually get better range then ever before, if that’s even possible. My battery life is good and my sound just plain works without any additional effort. If I had one complaint it would be with the amount of RAM the distribution uses. After a quick reboot it takes up a very small amount, around ~350MB or so. However after a couple of hours of general use the RAM often grows to about 1-1.5GB, which is far more than I have seen with the other distributions. Thankfully I have 4GB of RAM so I’m not too worried. I wonder if it has something to do with the fact that I am running the x64 version and not the x86 version. Perhaps it assumes I have at least 4GB of RAM for choosing the newer architecture.

    Whatever the case may be I think I have finally found what I consider to be the very best KDE Linux distribution. Obviously your results may vary but I look forward to hearing what you think.

This piece was cross-posted over at my person website ‘TylerBurton.Ca‘.




I am currently running a variety of distributions, primarily Linux Mint 18.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Feel free to visit me at my personal website here.

A Practical Reference of Linux Commands

February 19th, 2010 1 comment

Just wanted to share a link to a great table that I found – the practical reference of linux commands is a handy little table of terminal commands organized by task. I’ll add it to our sidebar under the ‘Useful Sites’ heading for future reference.

Happy Linuxing!




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

The Great Linux Browser War (Well… sort of)

February 13th, 2010 1 comment

Recently while browsing the Internet I noticed that Mozilla Firefox was taking up an awfully large amount of RAM. In fact it’s RAM use continued to grow even though I had not navigated to any new website or even touched it at all! Being a KDE user I decided to try the same website within Konqueror, the KDE web browser that everyone seems to toss to the side right after install. To my amazement Konqueror seemed to render the same website, using much less RAM, and better yet it’s RAM use remained static. Perhaps there was something more to this ‘throw-away’ browser than I had first thought. And thus began my idea for a series of comparisons of four of the most popular Linux web browsers: Firefox, Chromium, Konqueror and Epiphany.

Note: The numbers you are going to see below are purely anecdotal and are based on my own personal experiences, which might not represent your own.  For the record I am running Kubuntu 9.10 with the 2.6.31-19 kernel.

Packages I had to install

This is simply a list of packages I installed for each browsers to sort of give you an idea of what to expect.

Firefox Konqueror Chromium Epiphany
firefox-3.5, firefox-3.5-branding konqueror chromium-browser ephiphany-browser, epiphany-browser-data

Startup speed (from a cold start – i.e. from reboot)

To test this I rebooted the computer and then opened the browsers for the first time.

Firefox Konqueror Chromium Epiphany
~4.5 seconds ~2.1 seconds ~3.7 seconds ~2.8 seconds

Startup speed (from a cached start – i.e. after opening and closing the program)

To test this I rebooted the computer, then opened the browsers for the first time and closed them. Then I opened them a second time and recorded the time.

Firefox Konqueror Chromium Epiphany
~1.5 seconds ~1.2 seconds ~1.1 seconds ~1.2 seconds

Memory usage (about:blank)

This is how much memory the browser took to display the web page in brackets. The memory inside of the brackets is the amount of shared memory each browser uses. This of course could change depending on your system and setup. Where there are multiple numbers separated by plus signs it means that the browser spawns multiple processes which each take up memory.

Firefox Konqueror Chromium Epiphany
24.6MiB (22.7MiB) 10.8MiB (20.8MiB) 6.6MiB (21.5MiB) + 1.6MiB (1.6MiB) + 2.3MiB (6.4MiB) + 1.5MiB (7.6MiB) 9.4MiB (20.5MiB)

Memory usage (about:blank -> kubuntu.org)

For this test I first loaded the browser to the homepage, in this case about:blank, and then navigated to the website www.kubuntu.org. In Konqueror’s case it spawned a few small KIO processes which I assume did the actually downloading of the webpage. I have averaged their values below.

Firefox Konqueror Chromium Epiphany
35.7MiB (24.1MiB) 22.6MiB (23.3MiB) + 4 x kio_http: 5.05MiB (6.2MiB) 7.9MiB (18.2MiB) + 13.6MiB (11.3MiB) + 1.6MiB (1.7MiB) + 1.5MiB (7.6MiB) 18.3MiB (23.1MiB)

Memory usage (about:blank -> kubuntu.org | google.com | bing.com)

For this test I once again started with the about:blank homepage and then opened the websites in different tabs.

Firefox Konqueror Chromium Epiphany
41.7MiB (24.6MiB) 30.5MiB (24.0MiB) + 4 x kio_http: 5.3MiB (6.4MiB) 9.0MiB (28.8MiB) + 11.4MiB (11.3MiB) + 1.6MiB (1.7MiB) + 6.8MiB (11.0MiB) + 9.3MiB (11.5MiB) + 1.5MiB (7.6MiB) 21.8MiB (24.8MiB)

Memory usage (about:blank -> kubuntu.org | google.com | bing.com) 2 minutes later

For this test I started with the above test and then waited two minutes and re-recorded the memory usage. This was to see if the browsers suffer from any sort of memory leaks, something Firefox has been famous for over the years.

Firefox Konqueror Chromium Epiphany
38.45MiB (24.6MiB) – a decline of about 0.03MiB. 30.7MiB (24.0MiB) – a growth of about 0.002MiB/second. The kio_http’s stayed the same. 13.2MiB + (20.8MiB) + 11.4MiB (11.3MiB) + 1.6MiB (1.7MiB) + 6.8MiB (11.0MiB) + 9.3MiB (11.5MiB) + 1.5MiB (7.6MiB) – a growth of about 0.04MiB/second. 21.8MiB (24.8MiB) – no change

Memory usage (about:blank -> kubuntu.org | google.com | bing.com | ubuntu.com)

Same as above but with one more tab.

Firefox Konqueror Chromium Epiphany
46.0MiB (24.9MiB) 45.1MiB (24.0MiB) + 6 x kio_http: 5.22MiB (6.4MiB) 9.3MiB (24.4MiB) + 14.2MiB (16.2MiB) + 9.3MiB (11.4MiB) + 13.5MiB (11.2MiB) + 11.6MiB (10.8MiB) + 1.6 (1.7MiB) + 1.5MiB (7.6MiB) 29.0MiB (24.9MiB)

Memory usage (about:blank -> kubuntu.org | google.com | bing.com | ubuntu.com) 2 minutes later

Same as above but with one more tab.

Firefox Konqueror Chromium Epiphany
77.8MiB (24.9MiB) – a growth of about 0.27MiB/second 50.2MiB (24.0MiB) – a growth of about 0.04MiB/second. The kio_http’s stayed the same. 9.3MiB (24.4MiB) + 23.7MiB (16.2MiB) + 9.3MiB (11.4MiB) + 13.5MiB (11.2MiB) + 11.6MiB (10.8MiB) + 1.6MiB (1.7MiB) + 1.5MiB (7.6MiB) – a growth of about 0.08MiB/second 33.2MiB (24.9MiB) – a growth of about 0.04MiB/second

Noticeable rendering glitches in pages viewed

None of the browsers had rendering glitches except for Konqueror. In Konqueror Bing’s search bar was offset from where it should be and on Kubuntu’s website the header image was overlapping text that it shouldn’t have.

Acid 2 test (http://www.webstandards.org/files/acid2/test.html)

The (now old) Acid 2 test.

Firefox Konqueror Chromium Epiphany
100% 100% 100% 100%

Acid 3 test (http://acid3.acidtests.org/)

How well did the browsers handle the Acid 3 test?

Firefox Konqueror Chromium Epiphany
93% 89% (linktest failed) 100% 100%

SunSpider JavaScript Benchmark (http://www2.webkit.org/perf/sunspider-0.9/sunspider.html)

The classic JavaScript benchmark test!

Firefox Konqueror Chromium Epiphany
Total: 2380.2ms
Click here to see full results.
Total: 2940.2ms
Click here to see full results.
Total: 445.2ms
Click here to see full results.
Total: 794.8ms
Click here to see full results.

Conclusion

Well there you have it. Rather than ‘rate’ each browser I’m going to leave it up to you to weigh the merits of each given the above information. Even though Firefox might not be the most technically superior browser it does have the advantage of being very popular and having an excellent add-on system. Then again Epiphany can use most of those add-ons as well. Chromium, based off of Google’s Chrome browser, is also a new favourite for a lot of people and is obviously the fastest in terms of JavaScript. Being a KDE user, Konqueror might be the best browser for you thanks to its infinite customization and uniform look.

Linux is all about personalization and I think everyone should extend that to the browser they use. Who knows you might just find one you like even more.




I am currently running a variety of distributions, primarily Linux Mint 18.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Feel free to visit me at my personal website here.

Enabling Video Thumbnail Previews in Dolphin

January 31st, 2010 3 comments

Just a quickie here – if you keep video of any kind on your Kubuntu 9.10 system, you may have noticed that the Dolphin file manager doesn’t show thumbnail previews of video files by default.  Turns out that it’s a very easy (if non-obvious) feature to enable. Simply follow these steps:

  1. Open up kPackageKit
  2. Search for and install the package mplayerthumbs – it has three dependencies, which include mplayer itself (I use VLC, but to each his own)
  3. Back in Dolphin, navigate to Settings > Configure Dolphin > General > Previews Tab
  4. Scroll down in the list, and you should be able to see an option called Video Files (MPlayerThumbs) – Check that box
  5. Drag the Maximum file size slider all the way to the right, and hit apply

Navigate to a folder that contains video files, and watch as they slowly begin to populate. Be patient though, it can take a few minutes if you have a lot of media. You should also note that it doesn’t work with all WMV files.

Thanks to youTube user gotbletu for the following informative video that I ripped these instructions off from:




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

Pulse Audio Nonsense

January 4th, 2010 3 comments

Just a heads up: This isn’t the kind of post that contains answers to your problems. It is, unfortunately, the kind of post that contains a lot of the steps that I took to fix a problem, without much information about the order in which I performed them, why I performed them, or what they did. All that I can tell you is that after doing some or all of these things in an arbitrary order, stuff seemed to work better than it did before.

It’s funny how these posts often seem to come about when trying to get hardware related things working. I distinctly remember writing one of these about getting hardware compositing working on Debian. This one is about getting reliable audio on Kubuntu 9.10.

You see, I have recently been experiencing some odd behaviour from my audio stack in Kubuntu. My machine almost always plays the startup/shutdown noises, Banshee usually provides audio by way of GStreamer, videos playing in VLC are sometimes accompanied by audio, and Flash videos almost never have working sound. Generally speaking, restarting the machine will change one or all of these items, and sometimes none. The system is usuable, but frustrating (although I might be forgiven for saying that having no audio in Flash prevents me from wasting so much time watching youtube videos when I ought to be working).

Tonight, after some time on the #kubuntu IRC channel and the #pulseaudio channel on freenode, I managed to fix all of that, and my system now supports full 5.1 surround audio, at all times, and from all applications. Cool, no? Basically, the fix was to install some PulseAudio apps:

sudo apt-get install pulseaudio pavucontrol padevchooser

Next, go to System Settings > Multimedia, and set PulseAudio as the preferred audio device in each of the categories on the left. Finally, restart the machine a couple of times. If you’re lucky, once you restart and run pavucontrol from the terminal, you’ll see a dialog box called Volume Control. Head over to the Configuration tab, and start choosing different profiles until you can hear some audio from your system. Also, I found that most of these profiles were muted by default – you can change that on the Output Devices tab. If one of the profiles works for  you, congratulations! If not, well, I guess you’re no worse off than you were before. I warned you that this was that kind of post.

Also, while attempting to fix my audio problems, I found some neat sites:

  • Colin Guthrie – I spoke to this guy on IRC, and he was really helpful. He also seems to write a lot of stuff for the PulseAudio/Phonon stack in KDE. His site is a wealth of information about the stack that I really don’t understand, but makes for good reading.
  • Musings on Maintaining Ubuntu – Some guy named Dan who seems to be a lead audio developer for the Ubuntu project. Also a very interesting read, and full of interesting information about audio support in Karmic.
  • A Script that Profiles your Audio Setup – This bash script compiles a readout of what your machine thinks is going on with your audio hardware, and automatically hosts it on the web so that you can share it with people trying to help you out.
  • A Handy Diagram of the Linux Audio Stack – This really explains a lot about what the hell is going on when an application tries to play audio in the Linux.
  • What the Linux Audio Stack Seems Like – This diagram reflects my level of understanding of Linux audio. It also reminds me of XKCD.
  • Ardour – The Digital Audio Workstation – In the classic tradition of running before walking, I just have to try this app out.



On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

Finally Synching my Blackberry on Linux

January 2nd, 2010 18 comments

Some readers may recall all of the attempts that I’ve made in the past to synchronize my Blackberry with Mozilla’s Thunderbird email and calendar client. During each of these tries, I had relied on the OpenSync framework, along with the Barry project for communication with my phone, and a number of different solutions to link into Thunderbird. At various times, these included the opensync-plugin-iceowl, opensync-plugin-sunbird, and bluezync packages, none of which yielded success.

While running GNOME on my Debian laptop, I had managed to successfully synchronize my phone with the Evolution mail client. Even so, I continued to work at Thunderbird synchronization because I disliked Evolution, seeing it as a Microsoft Outlook clone, which is a platform that I have had considerable problems with in the past.

With my recent installation of Kubuntu 9.10 on my PC, I have been exposed to the Kontact PIM suite, and have thus far been impressed. Kmail is a solid email client, although the way that it handles the setup of multiple email accounts is confusing to say the least, forcing the user to create a sending, receiving, and identity object for each account, and then to link them together. Likewise, Kontact is a decent application, but is sorely lacking basic GUI configuration options, something I never thought that I would say about a KDE app. Finally, Kalendar does everything that one would expect, and allows the user to display appointments in a number of useful ways. All have excellent integration, and live in a tray widget that uses the native KDE notifications system to let me know when something important has happened.

Most importantly however, I managed to get the entire Kontact suite to sync with my Blackberry after about five minutes of playing around in the terminal. Unlike during previous installation attempts, I found the latest stable Barry packages available in my repositories, so installation was a snap. I simply added the following packages to my system:

  • libopensync0 v0.22-2
  • multisync-tools v0.92
  • libbarry0 v0.14-2.1
  • opensync-plugin-kdepim v0.22-4
  • opensync-plugin-barry v0.14-2.1

From a terminal, I then used the msynctool application and the following steps to do a little bit of configuration:

  1. msynctool –listplugins if the install went well, this command should list both kdepim-sync and barry-sync as available plugins
  2. msynctool –addgroup BB create an OpenSync sync profile for my Blackberry called BB
  3. msynctool –addmember BB barry-sync add the barry-sync plugin to the BB sync group
  4. msynctool –addmember BB kdepim-sync add the kdepim-sync plugin to the BB sync group
  5. msynctool –showgroup BB this lists each of the plugins that we just added to the BB sync group, along with their member numbers. In my case, barry-sync was member number 1, and kdepim-sync was member number 2. The output also showed that while barry-sync still needed to be configured, kdepim-sync had no configuration options to be set.
  6. msynctool –configure BB 1 configures member number 1 of the sync group BB. In my case, this was barry-sync, and simply popped a config file in the nano text editor. All that had to be changed in the file was the PIN of the Blackberry that the plugin would attempt to sync with.
  7. msynctool –sync BB actually performed the synchronization process. For safety’s sake, I made sure that Kontact was fully closed before running this command.

And that’s it! In the future, I simply have to run the msynctool –sync BB command to synchronize my Blackberry with Kontact. That’s one more reason to stick with Linux – Blackberry synchronization that isn’t tied to Microsoft Outlook!




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

Filling a Zune from Linux

January 1st, 2010 2 comments

Thinking that I was up for a challenge, I decided to spend the day figuring out how to put the music in my Banshee library onto a Microsoft Zune. Since my library contains a good number of FLAC files that I’ve ripped in from my CD collection, my solution called for a caching system that converts the FLAC files to mp3s and stores them so that the playlist can be changed without having to re-convert the FLAC files into something that the Zune can play on every sync. My weapons of choice for the project were a Windows XP instance running inside of Sun Virtual Box, and 126 lines of perl script.

The Steps:

  1. Create a WinXP VM that has the Zune software installed and can see two shared folders on my Linux machine:
    1. My normal Music folder, which contain my entire collection
    2. The cache folders, which contain all of the FLAC files in my collection, but converted to mp3 so that the Zune can play them
  2. Open the Banshee database, located at ~/.config/banshee-1/banshee.db
  3. Select all of the FLAC files in the playlist that we’d like to put on the Zune
  4. For each, check if it has been converted and cached
    1. If so, simply add the path to the cached copy of the track to an m3u file in the cache folder
    2. If not, convert the track, and then add the path to the cached copy of the track to an m3u file in the cache folder
  5. Select all of the mp3 files in the playlist that we’d like to put on the Zune
  6. Put those in a separate m3u file that is located in the Music folder.
  7. Boot up the Zune software on the VM. It should autoscan it’s monitored folders, find the m3u playlists, and put them in its library
  8. Sync the playlists with the Zune by dragging and dropping them to the device icon in the lower left corner of the screen

As previously mentioned, steps 2 through 6 were accomplished by way of a perl script that I can run as often as I like:

#/usr/local/bin/perl

#Requirements: libdbd-sqlite3-perl, flac, lame

#We need database support
use DBI;

#Database path – change this to reflect your user environment
my $dbpath = “dbi:SQLite:dbname=/home/jon/.config/banshee-1/banshee.db”;

#Playlist name – change this to reflect the playlist that you want to export
my $plistname = “Favorites”;

#Cache Path – the path to the directory where you’ve been caching converted FLAC files
my $cachepath = “/home/jon/Storage/mp3Cache/”;

#Music Path – the path to the folder where your music collection is actually stored
my $musicpath = “/home/jon/Music/”;

#Connect to the database – no username/password
my $dbh = DBI->connect($dbpath,””,””,{RaiseError => 1, AutoCommit => 0});

if(!$dbh) {
print “Could not connect to database $dbpath”,”\n”,”Exiting”;
exit;
}

#Pull the list of FLAC files for conversion and caching
my $flac = $dbh->selectall_arrayref(“SELECT sme.TrackID, ct.Title, car.Name AS ‘Artist’, ca.Title AS ‘Album’, ct.Uri, ct.Duration AS ‘Length’ FROM corealbums AS ca, coreartists AS car, coresmartplaylistentries AS sme INNER JOIN coretracks AS ct ON sme.TrackID = ct.TrackID WHERE sme.SmartPlaylistID = (SELECT `SmartPlaylistID` FROM `coresmartplaylists` WHERE `Name` = ‘$plistname’) AND ca.AlbumID = ct.AlbumID AND car.ArtistID = ct.ArtistID AND ct.MimeType LIKE ‘%flac'”);

#open the m3u file to write the cached items to
open my $m3u, ‘>’, $cachepath.$plistname.’_cached.m3u’ or die “Error trying to open cache m3u playlist for overwrite. Do you have write permissions in $cachepath ?”;
print $m3u “#EXTM3U\r\n\r\n”;    #note windows \r\n here

#add /music to $cachepath so that files are in a subdirectory, away from the m3u file
$cachepath = $cachepath.”music/”;
if( ! -e $cachepath ) {
`mkdir “$cachepath”`;
}

#loop through the files and check if they need to be cached
foreach my $i (@$flac) {
my ($trackid, $title, $artist, $album, $uri, $length) = @$i;

#correct the uri by removing the file:// prefix and reverting the uri escaping
$uri = substr $uri, 7;
$uri =~ s/%([0-9A-Fa-f]{2})/chr(hex($1))/eg;

#fix time into seconds
$length = int($length/1000);

#check if the flac file has already been converted and cached at cachepath
#if not, convert it and put it at cachepath.
my $path = $cachepath . $artist . ‘/’ . $album . ‘/’ . $title . ‘.mp3′;
if( ! -e $path ) {
#file dne, convert it
print “\nTrack: $title by $artist has not yet been cached, converting…”,”\n”;

#make sure that the file actually exists before attempting to convert it
if( ! -e $uri ) {
print “WARNING: Track $title by $artist does not exist at $uri”,”\n”;
} else {

#ensure that cache album/artist directories exist
my $partpath = $cachepath.$artist;
if( ! -d $partpath ) {
`mkdir “$partpath”`;
}
$partpath = $partpath.’/’.$album;
if( ! -d $partpath ) {
`mkdir “$partpath”`;l
}

#do the conversion – we’re chaining flac and lame here, reading in the flac file from $uri, and putting the resulting mp3 at $path
`flac -cd “$uri” | lame -h – “$path”`;
}
}

#add the track to the m3u file – note that these entries are relative to the location of the m3u file in the root of $cachepath
#the paths use a backslash and a \r\n newline so that they work correctly on windows
print $m3u “#EXTINF:$length,$artist – $title\r\n”;
print $m3u ‘\\music\\’.$artist.’\\’.$album.’\\’.$title.’.mp3′,”\r\n\r\n”;
}

#close the m3u file in the cachepath directory
close $m3u;

#TODO: scan the m3u file and delete any files that aren’t in it from the cache directory

#Pull the list of MP3 files and dump them into an m3u file
my $flac = $dbh->selectall_arrayref(“SELECT sme.TrackID, ct.Title, car.Name AS ‘Artist’, ca.Title AS ‘Album’, ct.Uri, ct.Duration AS ‘Length’ FROM corealbums AS ca, coreartists AS car, coresmartplaylistentries AS sme INNER JOIN coretracks AS ct ON sme.TrackID = ct.TrackID WHERE sme.SmartPlaylistID = (SELECT `SmartPlaylistID` FROM `coresmartplaylists` WHERE `Name` = ‘$plistname’) AND ca.AlbumID = ct.AlbumID AND car.ArtistID = ct.ArtistID AND ct.MimeType LIKE ‘%mp3′”);

#open the m3u file to write the cached items to
open my $m3u, ‘>’, $musicpath.$plistname.’.m3u’ or die “Error trying to open music folder m3u playlist for overwrite. Do you have write permissions in $musicpath ?”;
print $m3u “#EXTM3U\r\n\r\n”;    #note windows \r\n here

#loop through the files and check if they need to be cached
foreach my $i (@$flac) {
my ($trackid, $title, $artist, $album, $uri, $length) = @$i;

#correct the uri to become a windows file path
$uri = substr $uri, 7;            #remove file:// prefix
$uri =~ s/%([0-9A-Fa-f]{2})/chr(hex($1))/eg;    #correct uri encoding
$uri =~ s/$musicpath//g;            #remove musicpath prefix
$uri =~ s/\//\\/g;                #change forward slashes to backslashes
$uri = ‘\\’.$uri;                #add the leading backslash

#fix time into seconds
$length = int($length/1000);

#add the track to the m3u file – note that these entries are relative to the location of the m3u file in the root of $cachepath
#the paths use a backslash and a \r\n newline so that they work correctly on windows
print $m3u “#EXTINF:$length,$artist – $title\r\n”;
print $m3u $uri,”\r\n\r\n”;
}

#close the m3u file and the database connection
close $m3u;
$dbh->disconnect;

Sorry for the horrible formatting.

The only snag that I hit during the entire process was really my fault – I have a tendency to overcomplicate things, and did so on this project by initially writing the script to output a *.zpl file instead of a *.m3u file. That didn’t work at all, and I ended up simplifying the script greatly by just outputting an *.m3u file and hoping for the best.

On the off chance that the Zune jukebox software refuses to properly update its playlists after you change the *.m3u files, first try deleting them from the application, and then restarting it. If that doesn’t work, you can write a Windows batch script with code similar to the following:

del /q “C:\Documents and Settings\Jonathan\My Documents\My Music\Zune\Playlists\*”
xcopy “\\Vboxsvr\mp3cache\Favorites_cached.m3u” “C:\Documents and Settings\Jonathan\My Documents\My Music\Zune\Playlists”
xcopy “\\Vboxsvr\music\Favorites.m3u” “C:\Documents and Settings\Jonathan\My Documents\My Music\Zune\Playlists”

This script deletes all files from the Zune playlists directory, and then copies each of the *.m3u files that we created with the above perl script directly into the Zune playlists directory. This should force the application to get it’s act together.

Overall, I’m happy with this patchwork job. It allows me to use the Zune on Linux, which is great because the Zune really is a beautiful piece of hardware. Now if only the libmtp guys could get it working natively, without a WinXP VM…

This piece was mirrored at Index out of Bounds




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.

The end of the long road

January 1st, 2010 3 comments

Well it’s official, the year is now 2010 and we still don’t have flying cars.

The End

2010 also marks the end of The Linux Experiment. I can honestly say that the last four solid months of Linux use has taught me a lot. In reflection of this I decided to look back at what I had originally wrote about my goals of this experiment and see just how many of them I had accomplished.

  • I will have learned enough of the ins and outs to be as comfortable within a Linux environment as I current am within a Windows one.
    • This one is a bit tricky to answer. I am far more familiar and comfortable with Linux now than I have ever been before. However I still do not understand a number of things. For example the Linux file system confuses me to no end. What is the difference between /bin/ and /sbin/? Or why do some things end up in /etc/ and others in /var/ or even /opt/? Clearly I have some room to improve here.
  • My bonus goal is to have a fully functional, self-created, program that runs native to Linux.
    • This one I was actually able to realize. Not only did I have a native OpenGL program running, but in recent weeks I have even created cross-platform .NET/Mono based applications. In addition Linux has proven time and again that it is the platform for web development. I can definitely see myself utilizing it as such in the future.

Fedora

Fedora has been both a terrible nightmare and an absolute pleasure. I have had more problems getting things to just work on this distribution that I care to even remember. Yet time and time again there was something about Fedora that just kept pulling me back in. Perhaps it was the challenge of trying to master a power user’s distribution of choice. Or maybe it was just pure stubbornness. The fact remains that with the exception of Fedora 12 being incompatible with my graphics hardware there was nothing I haven’t overcome.

So would I recommend Fedora to someone? Well… yes and no. Fedora has a rock solid community and lives right on the cutting edge (what? I’m already running KDE 4.3.4??) but it does not make things easy. Now that most distributions have moved up to the 2.6.31 kernel there is really less of a reason for me to recommend the cutting edge simply as a way to get decent hardware support. Obviously if your machine is even newer than mine than perhaps Fedora is still your only stable ticket to that support, but for most users I think there are far better alternatives. Don’t get me wrong, I really do enjoy Fedora and from what I have read it has come a long way in recent years, I just don’t think I will be using it again anytime soon.

The Future

Today will bring some changes to my computing setup as I plan on removing Fedora and trying out two new KDE distributions, OpenSUSE and Kubuntu, just to see which one I prefer. In addition I will be dual booting with Windows 7 for the first time. I will be sure to keep everyone up to date with my experiences as I do so.

As we here at The Linux Experiment debate where to take the experiment moving forward, be sure to check back for updates on our new experiences!




I am currently running a variety of distributions, primarily Linux Mint 18.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Feel free to visit me at my personal website here.

Setting up an LVM for Storage

December 30th, 2009 5 comments

Recently, I installed Kubuntu on my PC. Under Windows, I had used RAID1 array to create a storage volume out of two extra 500GB hard drives that I have in my system. Under Linux, I’ve decided to try creating a 1TB LVM out of the drives instead. This should be visible as a single drive, and allow me to store non-essential media files and home partition backups on a separate physical drive, the better to recover from catastrophic failures with. The only problem with this plan: documentation detailing the process of creating an LVM is sparse at best.

The Drive Situation
My machine contains the following drives, which are visible in the /dev directory:

  • sdc: root drive that contains three partitions; 1, 2, and 5, which are my boot, root, and swap partitions respectively
  • sda: 500GB SATA candidate drive that I’d like to add to the LVM
  • sdb: 500GB SATA candidate drive that I’d like to add to the LVM

First Try
Coming from a Windows background, I began by searching out a graphical tool for the job. I found one in my repositories called system-config-lvm 1.1.4.

The graphical tool that I found to create LVMs

I followed the buttons in this tool and created a 1TB LVM spanning sda and sdb, then formatted it with ext3. The result of these steps was an uninitialised LVM that refused to mount at boot. In response, I wrote the following script to activate, mount, and assign permissions to the drive at boot:

#!/bin/bash
sudo whoami
sudo lvchange -a y /dev/Storage/Storage
sudo mount /dev/Storage/Storage /home/jon/Storage
sudo chown jon /home/jon/Storage
sudo chmod 777 /home/jon/Storage

It worked about 50% of the time. Frustrated, I headed over to the #kubuntu IRC channel to find a better solution.

Second Try
On the #kubuntu channel, I got help from a fellow who walked me through the correct creation process from the command line. The steps are as follows:

  1. Create identical partitions on sda and sdb:
    1. sudo fdisk /dev/sda
    2. n to create a new partition on the disk
    3. p to make this the primary partition
    4. 1 to give the partition the number 1 as an identifier. It will then appear as sda1 under /dev
    5. Assign first and last cylinders – I simply used the default values for these options, as I want the partition to span the entire drive
    6. t toggle the type of partition to create
    7. 8e is the hex code for a Linux LVM
    8. w to write your changes to the disk. This will (obviously) overwrite any data on the disk
    9. Repeat steps 1 through 8 for /dev/sdb
    10. Both disks now have partition tables that span their entirety, but neither has been formatted (that step comes later).
  2. Make the partitions available to the LVM:
    1. sudo pvcreate /dev/sda1
    2. sudo pvcreate /dev/sdb1
    3. Notice that the two previous steps addressed the partitions sda1 and sdb1 that we created earlier
  3. Create the Volume Group that will contain our disks:
    1. sudo vgcreate storage /dev/sda1 /dev/sdb1 will create the volume group that spans the two partitions sda1 and sdb1
    2. sudo vgdisplay /dev/storage queries the newly created volume group. In particular, we want the VG Size property. In my case, it is 931.52 GB
  4. Create a Logical Volume from the Volume Group:
    1. sudo lvcreate -L $size(M or G) -n $name $path where $size is the value of the VG Size property from above (G for gigabytes, M for megabytes), $name is the name you’d like to give the new Logical Volume, and $path is the path to the Volume Group that we made in the previous step. My finished command looked like sudo lvcreate -L 931G -n storage dev/storage
    2. sudo lvdisplay /dev/storage queries our new Logical Volume. Taking a look at the LV Size property shows that the ‘storage’ is a 931GB volume.
  5. Put a file system on the Logical Volume ‘storage’:
    1. sudo mkfs.ext4 -L $name -j /dev/storage/storage will put an ext4 file system onto the Logical Volume ‘storage’ with the label $name. I used the label ‘storage’ for mine, just to keep things simple, but you can use whatever you like. Note that this process takes a minute or two, as it has to write all of the inode tables for the new file system. You can use mkfs.ext2 or mkfs.ext3 instead of this command if you want to use a different file system.
  6. Add an fstab entry for ‘storage’ so that it gets mounted on boot:
    1. sudo nano /etc/fstab to open the fstab file in nano with root permissions
    2. Add the line /dev/storage/storage    /home/jon/Storage       ext4    defaults        0       0 at the end of the file, where all of the spaces are tabs. This will cause the system to mount the Logical Volume ‘storage’ to the folder /home/jon/Storage on boot. Check out the wikipedia article on fstab for more information about specific mounting options.
    3. ctrl+x to exit nano
    4. y to write changes to disk
  7. Change the owner of ‘storage’ so that you have read/write access to the LVM
    1. sudo chown -R jon:jon /home/jon/Storage will give ownership to the disk mounted at /home/jon/Storage to the user ‘jon’

Time for a Beer
Whew, that was a lot of work! If all went well, we have managed to create a Logical Volume called storage that spans both sda and sdb, and is formatted with the ext4 file system. This volume will be mounted at boot to the folder Storage in my home directory, allowing me to dump non-essential media files like my music collection and system backups to a large disk that is physically separate from my system partitions.

The final step is to reboot the system, navigate to /home/jon/Storage (or wherever you set the boot point for the LVM in step 6), right-click, and hit properties. At the bottom of the properties dialog, beside ‘Device Usage,’ I can see that the folder in question has 869GB free of a total size of 916GB, which means that the system correctly mounted the LVM on boot. Congratulations to me!

Much thanks to the user ikonia on the #kubuntu IRC channel for all the help.

This piece has been mirrored at Index out of Bounds




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.