The Linux Experiment

We live in an old house, which has the unfortunate side-effect of lacking a wired network of any kind. All of our machines connect to a wireless network, and my desktop is no exception. I’ve got an old WUSB54GC wireless stick that was manufactured some time in 2007. In computer years, this is way old hardware. But with a bit of work, I managed to get it working with my Gentoo install.

This bitch is old... but it works

I started out by installing the NetworkManager applet with a tutorial on the Gentoo Wiki. This was a straightforward process, and after a restart, the applet icon appeared in the top right corner of my screen. If you left-click on the icon, it drops down a menu that lists your wireless interfaces. Under the Wireless Networks heading, it said that it was missing the firmware necessary to talk to my hardware.

The next step was to look around the net and figure out the firmware/kernel module combination that supports this stick. I found my answer over at the SerialMonkey project, which is run by a group that took on maintenance of older Ralink firmware after the company of the same name dropped support. According to the SerialMonkey hardware guide, my stick (or at least a very similar stick called the WUSB54GR) works with the rt73usb kernel module and related firmware.

This known, there are two methods of proceeding. Those running older kernels may need to manually compile the necessary packages using instructions similar to these, from the Arch Linux project. For more modern kernels, the Gentoo project provides a Wiki entry detailing the necessary steps.

After following the steps in the Gentoo Wiki entry, I restarted my system, and now have full wireless support. Genius!

So my base Gentoo installation came with Gnome 2.3, which while solid, lacks a lot of the prettiness of Gnome’s latest 3.2 release. I thought that I might like to enjoy some of that beauty, so I attempted to upgrade. Because Gnome 3.2 isn’t in the main portage tree yet, I found a tutorial that purported to walk me through the upgrade process using an overlay, which is kind of like a testing branch that you can merge into the main portage tree in order to get unsupported software.

Since the tutorial that I linked above is pretty self-explanatory, I won’t repeat the steps here. There’s also the little fact that the tutorial didn’t work worth a damn…

Problem 1: Masked Packages

#required by dev-libs/folks-9999,
required by gnome-base/gnome-shell-3.2.1-r1,
required by gnome-base/gdm-3.2.1.1-r1[gnome-shell],
required by gnome-base/gnome-2.32.1-r1,
required by @selected,
required by @world (argument)
>=dev-libs/libgee-0.6.2.1:0 introspection
#required by gnome-extra/sushi-0.2.1,
required by gnome-base/nautilus-3.2.1[previewer],
required by app-cdr/brasero-3.2.0-r1[nautilus],
required by media-sound/sound-juicer-2.99.0_pre20111001,
required by gnome-base/gnome-2.32.1-r1,
required by @selected,
required by @world (argument)
>=media-libs/clutter-gtk-1.0.4 introspection

This one is pretty simple to fix: you can add the lines >=dev-libs/libgee-0.6.2.1:0 introspection and >=media-libs/clutter-gtk-1.0.4 introspection to the file /etc/portage/package.accept_keywords, or you can run emerge -avuDN world –autounmask-write to get around these autounmask behaviour issues

Problem 2: Permissions

--------------------------- ACCESS VIOLATION SUMMARY ---------------------------
LOG FILE "/var/log/sandbox/sandbox-3222.log"

VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: mkdir
S: deny
P: /root/.local/share/webkit
A: /root/.local/share/webkit
R: /root/.local/share/webkit
C: ./epiphany --introspect-dump=
/var/tmp/portage/www-client/epiphany-3.0.4/temp/tmp-introspectSfeqBO/functions.txt,
/var/tmp/portage/www-client/epiphany-3.0.4/temp/tmp-introspectSfeqBO/dump.xml
--------------------------------------------------------------------------------

This one totally confused me. If I’m reading it correctly, the install script lacks the permissions necessary to write to the path /root.local/share/webkit/. The odd part of this is that the script is running as the root user, so this simple shouldn’t happen. I was able to give it the permissions that it needed by running chmod 777 /root/.local/share/webkit/, but I had to start the install process all over again, and it just failed with a similar error the first time that it attempted to write a file to that directory. What the fuck?

At 10pm at night, I couldn’t be bothered to find a fix for this… I used the tutorial’s instructions to roll back the changes, and I’ll try again later if I’m feeling motivated. In the mean time, if you know how to fix this process, I’d love to hear about it.

So I’m a big Dropbox user. I primarily use it to keep my personal info synchronized between my machines (don’t worry, I encrypt my stuff before dumping it into Dropbox, I’m not dumb), but it’s also handy for quickly sharing files with others.

Unfortunately, Dropbox doesn’t exist in the Gentoo portage tree.

To get started, head over to the Dropbox website and download the source tar.bzip file for your platform. Unzip it to your desktop, open a root terminal and cd into the resulting directory. Before you can actually install Dropbox, you’ll need to satisfy a few dependencies.

First, make sure that you’ve got python by typing emerge python into the aforementioned root terminal. Next, install docutils by typing emerge docutils in that same terminal. Now you should be able to install the dropbox stub by typing ./configure && make && make install.

At this point, Dropbox will have installed a stub of an application on your machine. You should be able to find it under Applications > Internet > Dropbox. When you launch this application, Dropbox will attempt to automatically download and install the binary portion of the application.

Optional: Verifying Binary Signatures

When dropbox downloads binary files, it verifies their legitimacy by calculating a digital signature and comparing it to a known value. In order for it to perform this task, you’ll need to have the pygpgme library installed on your system. Note that this is not the same as the python-gpgme library. They are different, and Dropbox requires the former. Like most Python libraries, pygpgme is a wrapper around a c-based library, in this case, GPGME. As such, the installation takes two steps. First, run emerge gpgme in your root terminal.

Second, you’ll need to install the pygpgme wrapper. It can be found on the project’s homepage at Launchpad. Unpack the tar.bzip, cd into the resulting directory, and run python setup.py build && python setup.py install from a root terminal. If the installation fails with an error message like

fatal error: gpgme.h: No such file or directory

then check the location of your gpgme.h file. It should have been included with the emerge gpgme command, but pygpgme expects it to live in /usr/include/. On my system, it was living in  /usr/include/gpgme/. I solved this problem by running cp /usr/include/gpgme/gpgme.h /user/include/. The only catch is that if you upgrade GPGME, you’ll need to remember that you copied the header file in order to make the python wrapper work. Once the file is copied, you should be able to run the setup script above.

Finally, run Dropbox and check to ensure that the warning message about binary signatures has gone away. You should now be good to go!

Edit: After I had figured all of this crap out, I realized that Dropbox actually is available in the Gentoo tree, but it’s called gnome-extra/nautilus-dropbox. You should be able to skip all of these steps and install Dropbox with the command emerge nautilus-dropbox, although I haven’t tried it myself.

Tonight, I finally got X11 working on my Gentoo machine. For those who are following along, on Tuesday night I managed to get my machine up to a command line. The next logical step is a graphical window manager.

I’ve chosen to give Gnome3 a spin, but before I can dive into all of it’s shiny UI-goodness, I need an X11 server installed on my machine. Because I have an nVidia graphics card in my machine, and I’ve had great luck with Ubuntu’s proprietary nVidia drivers in the past, I decided to skip over the open-source Nouveau drivers this time around. I started out the installation by following Gentoo’s nVidia guide, supplementing with info pulled from the nVidia entry on the Gentoo Wiki.

Although X is supposed to configure your system automagically, it couldn’t find my screens or devices on my first run of startx. I looked about the internet for a bit, and found out that you can force X to automatically configure itself. Simply run Xorg -configure and copy the file that it creates into your Xorg config directory (you can find it in the log file, mine is at /usr/share/X11/xorg.conf.d/).

In my case, these automagical settings still needed a bit of tweaking. I noticed in the log file (again, mine is at /var/log/Xorg.0.log, your mileage may vary) that X was failing to load GLX, which is essentially for 3D acceleration. In my case, GLX was installed, but it NVIDIA’s version wasn’t being loaded. Once again, the Gentoo Wiki came through for me, instructing me to run eselect opengl set nvidia. This worked like a charm.

Finally, I had to install twm and xterm so that I could see X working. That was a quick and painless process. Now on to Gnome!

Just a quick note tonight – I finally managed to get a bootable Gentoo system installed!

After my last post, things were looking pretty grim. Instead of continuing to perpetuate the recompile/reboot cycle, I decided to start fresh, in hopes that I had simply missed a step the first time around. With this in mind, I started back at page one of the Gentoo Handbook and worked my way through the entire thing.

When it came time to compile my kernel, I opted for a slightly less error-prone method, and started off by installing Genkernel, a tool that automates some of the kernel creation steps. When running it however, I was sure to pass the –menuconfig parameter, which gave me full control over what modules were included in the final product.

Next, I followed the kernel tutorials in the Gentoo Handbook and on the Gentoo Wiki Asus P5Q-E page. This ensured that I included every component that was necessary for my system.

Once I rebooted the machine, a login prompt came up the first time. Great success indeed!

One little gotcha that’s important to note at this step. On my first login, I didn’t have any network access. Two things that might help:

1. Open up /etc/conf.d/net in nano and add a line like config_eth0=”dhcp” for each network interface in your machine, where eth0 is the name of the interface. This tells the machine to use DHCP when initializing the device. On most home networks, this will get you an IP address.
2. Make sure that any required modules are loaded. I have two network interfaces. One uses the sky2 module, and the other uses skge. You can check to ensure that these are loaded with the command lsmod | grep sky2 where sky2 is the name of the module that you’re looking for. If it isn’t loaded, run modprobe sky2 to get it up and running. Note that you may need to recompile your kernel with support for the module in question if you missed it first time ’round.

Tomorrow, I’ll compile an X11 server, and hopefully get started on the GNOME desktop environment. Christ there’s still a lot to do…

So like Tyler, I’ve decided to run Gentoo. Hey, it seemed like a good idea at the time.

My experience thus far can be summed up with a single word: frustrating. I spent my first day working through the (excellent) Gentoo Handbook. Like Jake, I found it handy to have run lshw on my system prior to installing Gentoo. This provided me with a list of my hardware that I could refer back to during the installation process, and saved me a few headaches.

At first, my live-cd environment lacked a network connection. My machine has two network interfaces in it. One uses the sky2 kernel module, while the other uses skge. I ran:

modprobe skge
net-setup eth1
[follow on-screen instructions]
ping google.com

and was successful.

On that first day of dicking about, I managed to get all the way to Chapter 10: Configuring the Bootloader. It was at this point, in subchapter 10.d, that I was instructed to reboot the system, as though it would be a relaxing, daisy-scented walk in the park. Not so.

Apparently, the kernel that I’ve managed to compile does not recognize the SATA interface on my motherboard. When I attempt to boot, GRUB hands control off to the kernel, which goes looking for my root partition on /dev/sda3. It then dies with a message like

Kernel panic – not syncing: VFS: Unable to mount root fs on unknown-block(8,3)

This error message is the bane of my existence.

After a great deal of head-vs-desk action, approximately 37 kernel compilations, and a great deal of googling, I managed to find a Gentoo wiki entry that instructs users of my chipset on how to compile their very own working kernel. Tonight, I intend to follow it, in hopes that I can get the system to boot some time soon.

At this rate, I’ll be lucky to have a working desktop by the end of the experiment.

If you’ve been living under a rock for the past week, you may not be aware that Steve Jobs, co-founder and legendary CEO of Apple Inc., has recently died after a long and protracted battle with pancreatic cancer. After the announcement of his death, many news outlets (tech-oriented and otherwise) ran lengthy tributes to a man who has forever (and often disruptively) altered more industries than any other in recent memory.

The day after Jobs’ death, Free Software visionary and GNU Project founder Richard M. Stallman had this to say about the man:

Steve Jobs, the pioneer of the computer as a jail made cool, designed to sever fools from their freedom, has died.

As Chicago Mayor Harold Washington said of the corrupt former Mayor Daley, “I’m not glad he’s dead, but I’m glad he’s gone.” Nobody deserves to have to die – not Jobs, not Mr. Bill, not even people guilty of bigger evils than theirs. But we all deserve the end of Jobs’ malign influence on people’s computing.

Unfortunately, that influence continues despite his absence. We can only hope his successors, as they attempt to carry on his legacy, will be less effective.

Upon finding this post via Twitter, my immediate reaction was a deep loss of respect for Stallman, a man whose contributions to the Free Software movement cannot be understated. The way that I see it, Stallman and Jobs are one in the same. Both are (or were, in the case of the latter) visionaries, both contributed immeasurably to an industry that employs, informs, and entertains me on a daily basis, and both are/were zealots when it came to their personal opinions about software.

Now I’m not an Apple guy. Far from it, in fact. I don’t own a single Apple product, I use Linux whenever and wherever possible, and I only break from the four essential freedoms when obtaining and enjoying media that cannot be accessed otherwise. But regardless of your thoughts on Steve Jobs, the man deserves your respect.

While Stallman qualified his statement by noting that nobody deserves to die, he also focused his personal fanaticism when it comes to the perceived threat of non-free software directly on the shoulders of one man in a world of many.

There’s something about Freedom that Stallman doesn’t seem to (or want to, as all accounts paint him as a pretty smart dude) understand. It’s a simple point, and one that needs to be reiterated often: Freedom is the right to choose. In politics, in products, and in computing, freedom is the right to choose what is best for you.

Steve Jobs put his ideas and his products into the free market, and paying customers often chose them above those of Stallman. Perhaps those customers got shafted, but when faced with a choice between the freedom to edit configuration files and the beautiful design of an Apple product, they unsurprisingly chose the latter.

That’s freedom, whether you like it or not. Fuck Richard Stallman.

Further Reading:

• Gawker: What Everyone is Too Polite to Say about Steve Jobs
• The Most Excellent and Lamentable Tragedy of Richard Stallman
• Russel McOrmond: Spectrum from software/computing freedom to imprisonment 
• Wikipedia: Richard Stallman

So I decided to take a go at Ubuntu 11.04 in a virtual machine before taking the leap and installing it for real. As I understand it, the new Unity desktop is a pretty major departure from the Gnome 2.x desktop that I’m used to, and I want to see if it’s as bad as it looks in the screenshots.

Unfortunately, I’ve yet to make it to the desktop, as Ubuntu has decided that it will take 42 minutes to download some language packs that I neither want or need.

Didn’t I tell it what language I speak as the first step of the install process? Surely this can be done later.

Recently, Tyler linked me to Linux Multimedia Studio, a Fruityloops-type application for Linux. Since I’m big into music recording and production, he figured that I’d be interested in trying it out, and he was right. Unfortunately, the developers of same were not as interested.

To start off, I installed the application from a PPA with the following terminal commands:

sudo apt-add-repository ppa:dns/sound
sudo aptitude update
sudo aptitude install lmms

After the install process finished, I tried to launch the application from the command line, only to see a bunch of nasty error messages:

jonf@THE-LINUX-EXPERIMENT:~$ sudo lmms
bt_audio_service_open: connect() failed: Connection refused (111)
bt_audio_service_open: connect() failed: Connection refused (111)
bt_audio_service_open: connect() failed: Connection refused (111)
bt_audio_service_open: connect() failed: Connection refused (111)
bt_audio_service_open: connect() failed: Connection refused (111)
bt_audio_service_open: connect() failed: Connection refused (111)
bt_audio_service_open: connect() failed: Connection refused (111)
bt_audio_service_open: connect() failed: Connection refused (111)
Segmentation fault

I dumped the errors into Google, and found a helpful thread on the Ubuntu forums that suggested that I uninstall Bluetooth Audio Services from my machine. Since I don’t use bluetooth audio in any capacity, I happily obliged. When finished, my list of installed items with Bluetooth in the name looked like this:

Unfortunately, I didn't think ahead enough to note down the names of the packages that I uninstalled.

After ridding myself of Bluetooth audio support, I tried to launch the application again. Unfortunately, I got another Segmentation fault error:

jonf@THE-LINUX-EXPERIMENT:~$ sudo lmms
Segmentation fault

Reading on in the thread, I saw somebody suggest that I check the dmesg tail for messages pertaining to the crash:

jonf@THE-LINUX-EXPERIMENT:~$ dmesg | tail
[  233.302221] JFS: nTxBlock = 8192, nTxLock = 65536
[  233.314247] NTFS driver 2.1.29 [Flags: R/O MODULE].
[  233.343361] QNX4 filesystem 0.2.3 registered.
[  233.367738] Btrfs loaded
[ 2233.118020] __ratelimit: 33 callbacks suppressed
[ 2233.118026] lmms[10706]: segfault at 7f241c7fdd80 ip 00007f241c7fdd80 sp 00007f24187f8a38 error 14 in zm1_1428.so[7f241ca01000+1000]
[ 2523.015245] lmms[10808]: segfault at 7fd80e9bcd80 ip 00007fd80e9bcd80 sp 00007fd80a9b7a38 error 14 in zm1_1428.so[7fd80ebc0000+1000]
[ 2671.323363] lmms[10845]: segfault at 7fbe39a77d80 ip 00007fbe39a77d80 sp 00007fbe35a72a38 error 14 in zm1_1428.so[7fbe39c7b000+1000]
[ 2836.885480] lmms[11246]: segfault at 7f885b71ed80 ip 00007f885b71ed80 sp 00007f8857719a38 error 14 in zm1_1428.so[7f885b922000+1000]
[ 3039.773287] lmms[11413]: segfault at 7ff83056ed80 ip 00007ff83056ed80 sp 00007ff82c569a38 error 14 in zm1_1428.so[7ff830772000+1000]

On the last few lines, you can see that the error was thrown in a module called zml_1428.so. A bit of Googling turned up the fact that this module is a part of the LADSPA (Linux Audio Developers Simple Plugin API) stack, which provides developers with a standard, cross-platform API for dealing with audio filters and effects.

Scrolling down in the aforementioned thread, I found a post that suggested that I kill all PulseAudio activities on my system before attempting to run the application. PulseAudio is another part of the Linux audio layer that allows user-land applications to talk to your sound hardware via a simple API. It also provides some effects plugins and mixdown capabilities. I went ahead and killed the PulseAudio server on my machine with the following command:

jonf@THE-LINUX-EXPERIMENT:~$ killall pulseaudio

After executing this command, I still got a Segmentation fault when starting LMMS under my user account, but did actually get to a Settings panel when running it with Sudo:

jonf@THE-LINUX-EXPERIMENT:~$ sudo lmms
Home directory /home/jfritz not ours.
ALSA lib pcm_dmix.c:1010:(snd_pcm_dmix_open) unable to open slave
Playback open error: Device or resource busy
Expression 'snd_pcm_hw_params_set_buffer_size_near( self->pcm, hwParams, &bufSz )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1331
Expression 'PaAlsaStreamComponent_FinishConfigure( &self->playback, hwParamsPlayback, outParams, self->primeBuffers, realSr, outputLatency )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1889
Expression 'PaAlsaStream_Configure( stream, inputParameters, outputParameters, sampleRate, framesPerBuffer, &inputLatency, &outputLatency, &hostBufferSizeMode )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1994
Couldn't open PortAudio: Unanticipated host error
Home directory /home/jfritz not ours.
Home directory /home/jfritz not ours.

Although the output appeared to be riddled with audio layer errors, and the Audio Settings tab of the Setup panel gave me a clue as to why:

Notice how the Audio Interface setting in that image says “Pulse Audio (bad latency!)”. I would hazard a guess that the latency issues with PulseAudio have something to do with the fact that I killed it just prior to getting this damned thing to launch. When I hit the OK button, I was able to see the application, but there was no sound.

Figuring that sound was a necessary component of an audio production application, I booted back to the Setup menu, and told the app to funnel its audio through JACK instead of PulseAudio. The JACK Audio Connection Kit is another sound subsystem, kind of like PulseAudio, that provides an API that developers can use to interface with a machine’s audio hardware. Because of its low latency performance, JACK is often considered to be the standard API for high-quality audio recording and production apps. Unfortunately, it doesn’t work worth a damn in LMMS:

jonf@THE-LINUX-EXPERIMENT:~$ sudo lmms
jackd 0.118.0
Copyright 2001-2009 Paul Davis, Stephane Letz, Jack O'Quinn, Torben Hohn and others.
jackd comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions; see the file COPYING for details

no message buffer overruns
JACK compiled with System V SHM support.
loading driver ..
SSE2 detected
creating alsa driver ... hw:0|hw:0|1024|2|48000|0|0|nomon|swmeter|-|32bit
control device hw:0
SSE2 detected
all 32 bit float mono audio port buffers in use!
cannot assign buffer for port
cannot deliver port registration request
no more JACK-ports available!
No audio-driver working - falling back to dummy-audio-driver
You can render your songs and listen to the output files...
Home directory /home/jfritz not ours.
Home directory /home/jfritz not ours.
the playback device "hw:0" is already in use. Please stop the application using it and run JACK again
cannot load driver module alsa
Home directory /home/jfritz not ours.

Having dealt with JACK on a previous install, I had one more trick up my sleeve in my effort to get this bastard application to make a sound. I installed the JACK Control Panel from the Ubuntu Software Centre. It’s a QT app that interfaces with the JACK server and allows you to modify settings and stuff.

With it installed, I pressed the big green (or is it red – I’m colour blind, and hate when developers use these two colours for important status messages) Start button, only to encounter some nasty errors:

That might be a problem. I hit the messages button and found a message advising me to make a change to the /etc/security/limits.conf file so that JACK would be allowed to use realtime scheduling:

JACK is running in realtime mode, but you are not allowed to use realtime scheduling.
Please check your /etc/security/limits.conf for the following lines
and correct/add them:
@audio - rtprio 100
@audio - nice -10
After applying these changes, please re-login in order for them to take effect.
You don't appear to have a sane system configuration. It is very likely that you
encounter xruns. Please apply all the above mentioned changes and start jack again!

I figured that it was worth a shot, considering how far I’ve already gone just to try out a piece of software that I don’t really even need. I made the requested changes in the config file, restarted my machine and tried again… only to be greeted by the same damned error message.

At this point, I decided to give up on LMMS. It’s too damned complicated, and ultimately not worth my time. Perhaps when they release a version that I can install and start using without an hour of troubleshooting, I’ll come back and give it another shot. In the mean time, if you’re looking for a decent drum machine with more than a few tricks up its sleeve, check out Hydrogen Drum Machine. It works very well, and I’ve created some neat stuff in it.

A couple of nights ago, I left Banshee running overnight, filling my phone with music to listen to the next day at work. Sometime during that process, the media player crashed in a big way, and my entire system ground to a halt. When I woke up the next morning, my computer was frozen solid, with no keyboard or mouse input accepted, and the system clock hadn’t advanced since 11:16pm the previous night. I did a hard reboot, hoping that all would be well when it came back up.

Boy was I wrong.

The Problem

It seems that when Banshee went down, it took my library database file with it. Now, whenever I launch the application, I get a lovely dialog box that looks something like this:

Launching the application from the command line provides more details:

jonf@THE-LINUX-EXPERIMENT: ~/.config/banshee-1$ banshee-1
[Info  11:20:32.175] Running Banshee 1.9.0: [source-tarball (linux-gnu, x86_64) @ 2010-12-09 13:07:07 EST]
[Warn  11:20:32.451] Service `Banshee.Database.BansheeDbConnection’ not started: The database disk image is malformed
database disk image is malformed
[Warn  11:20:32.453] Caught an exception – Mono.Data.Sqlite.SqliteException: The database disk image is malformed

Apparently, the SQLite database that forms the backend of Banshee is severely corrupted, and the application can’t open it for reading. Based on the command line output, I figured that Banshee uses SQLite3 for its database, and after a little bit of research, I found out that it stores the database file at ~/.config/banshee-1/banshee.db. After installing the SQLite3 package from my repositories with the command sudo apt-get install sqlite3, I was able to connect to the database without issue. A little bit of reading over at sqlite.org showed me how to do an integrity check on the database, which came back with some nasty results:

jonf@THE-LINUX-EXPERIMENT:~/.config/banshee-1$ sqlite3 -interactive banshee.db
SQLite version 3.6.22
Enter “.help” for instructions
Enter SQL statements terminated with a “;”
sqlite> PRAGMA integrity_check;
*** in database main ***
Main freelist: 2291 of 2292 pages missing from overflow list starting at 18928
On tree page 28 cell 4: Child page depth differs
On tree page 28 cell 6: 2nd reference to page 15475
On tree page 28 cell 6: Child page depth differs
…
On tree page 16807 cell 25: Child page depth differs
On tree page 16807 cell 26: 2nd reference to page 18171
On tree page 16807 cell 26: Child page depth differs

Each line of this output refers to a broken or corrupted index, with a total of 99 errors reported. Yikes.

The Solution

Before attempting to fix the problem, I made a backup of the broken database, just in case some data could be resurrected from it. To do this, I made sure that Banshee wasn’t running, and then made a copy of the database file with the command cp ~/.config/banshee-1/banshee.db ~/.config/banshee-1/banshee.db.old.

Since the integrity check showed a number of broken or corrupted indices, I thought perhaps I would be able to recover the database by rebuilding all of its indices. A quick scan of the SQLite documentation turned up the REINDEX function. Again, I connected to the database through the command-line interpreter, and gave it a shot:

jonf@THE-LINUX-EXPERIMENT:~/.config/banshee-1$ sqlite3 -interactive banshee.db
SQLite version 3.6.22
Enter “.help” for instructions
Enter SQL statements terminated with a “;”
sqlite> REINDEX;
Error: database disk image is malformed

That clearly didn’t work.

My next thought was to drop and re-create all of the indices in the database, in hopes that I could rebuild them. I used the graphical tool sqliteman (available from your repositories with the command sudo apt-get install sqliteman) to dump the database schema out to a text file and then scrolled through the file looking for each index. I managed to drop all but four of them, but got the image malformed error whenever I tried to drop the remaining four or recreate any of the ones that were successfully dropped.

Out of ideas, I resorted to attempting to recover what data I could from the mangled file. Using sqliteman, I created a dump of my library data to go along with the schema dump that I had created earlier. The good thing about this dump is that it contains all of the data with none of the indices. Next, I opened up a terminal and navigated to the temporary directory where I had saved the database dump. Using the sqlite3 command line interpreter, I built an image of the old database from the dump:

jonf@THE-LINUX-EXPERIMENT:~/Desktop/banshee$ sqlite3 -interactive banshee.db
SQLite version 3.6.22
Enter “.help” for instructions
Enter SQL statements terminated with a “;”
sqlite> .read dump.sql

Finally, I copied the newly created database into the banshee data directory with the command cp ~/Desktop/banshee/banshee.db  ~/.config/banshee-1/banshee.db.

The next time I started the application, all of my data was restored.

Prevention

I’ve put a lot of work into collecting and maintaining all of the music in my Banshee library. In order to avoid losing all of that work, I’ve decided to write a short script that takes a backup image of the library database every so often, and puts it in a safe place on my hard drive. Once I get that put together, I’ll throw it up on here for so that everybody can use it.