Archive

Posts Tagged ‘programming’

How to backup and restore a Trac project

July 4th, 2013 No comments

If you use Trac as your bug and progress tracking tool then you too may one day need to take a backup of it or move it to a new server like I had to the other day. Thankfully, as I discovered, it is a relatively straight forward process. Here are the steps to backup and restore a Trac project.

Take a hot backup of your existing install. This is essentially a backup from a fixed point that you can take while still using your Trac at the same time (great for having no downtime).

trac-admin [/path/to/projenv] hotcopy [/path/to/backupdir]

For example:

trac-admin /var/www/trac/projectx hotcopy /home/awesomeadmin/trac_backup/projectx

In order to restore it on another server you just need to create the project from scratch (i.e. using initenv) like this

trac-admin [targetdir] initenv

and then simply replace the install directory contents with the backed up contents. Strictly speaking I’m not even sure if you need to initenv but that’s how I did it and it worked.

Hopefully this works for you as well. Happy… err… Trac-ing?




I am currently running a variety of distributions, primarily Ubuntu 14.04.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Check out my profile for more information.
Categories: Tyler B Tags: , ,

How to backup and restore an SVN repository with full commit history

July 2nd, 2013 No comments

Sometimes you need to move an SVN repository from one server to another but maintain the full commit history (i.e. comments and changes). Here is a very simple way to do so.

1. Dump (and compress) the source SVN in one line:

svnadmin dump [path to source SVNrepository] | gzip -9 > [path to destination gzipped dump file]

For example:

svnadmin dump /var/svn/projectx | gzip -9 > /home/awesomeadmin/svn_backup/projectx.dump.gz

2. Transfer gzipped dump to new server

3. Decompress dump

gunzip projectx.dump.gz

4. Restore dump to new SVN repository

svnadmin load [path to new SVN repository] < [path to dump file]

For example:

svnadmin load /var/svn/projecty < /home/awesomeadmin/svn_backup/projectx.dump

That’s it. Pretty simple, no?




I am currently running a variety of distributions, primarily Ubuntu 14.04.
Previously I was running KDE 4.3.3 on top of Fedora 11 (for the first experiment) and KDE 4.6.5 on top of Gentoo (for the second experiment).
Check out my profile for more information.
Categories: Tyler B Tags: , , ,

Python Development on Linux and Why You Should Too

September 1st, 2012 25 comments

If you’re a programmer and you use Linux but you haven’t yet entered the amazing world that is Python development, you’re really missing out on something special. For years, I dismissed Python as just another script kiddie language, eschewing it for more “serious” languages like Java and C#. What I was missing out on were the heady days of rapid development that I so enjoyed while hacking away on Visual Basic .NET in my early years of university.

There was a time when nary a day would go by without me slinging together some code into a crappy Windows Forms application that I wrote to play with a new idea or to automate an annoying task. By and large, these small projects ceased when I moved over to Linux, partially out of laziness, and partially because I missed the rapid prototyping environment that Visual Basic .NET provided. Let’s face it – Java and C# are great languages, but getting a basic forms app set up in either of them takes a significant amount of time and effort.

Enter Python, git/github, pip, and virtualenv. This basic tool chain has got me writing code in my spare time again, and the feeling is great. So without further ado, let me present (yet another) quick tutorial on how to set up a bad-ass Python development environment of your own:

Step 1: Python

If there’s one thing that I really love about Python, it’s the wide availability of libraries for most any task that one can imagine. An important part of the rapid prototyping frame of mind is to not get bogged down on writing low-level libraries. If you have to spend an hour or two writing a custom database interface layer, you’re going to lose the drive that got you started in on the project in the first place. Unless of course, the purpose of the project was to re-invent the database interface layer. In that case, all power to you. In my experience, this is never a problem with Python, as its magical import statement will unlock a world of possibilities that is occupied by literally thousands of libraries to do most anything that you can imagine.

Install this bad boy with the simple command sudo apt-get install python and then find yourself a good python tutorial with which to learn the basics. Alternatively, you can just start hacking away and use StackOverflow to fill in any of the gaps in your knowledge.

Step 2: Git/Github

In my professional life, I live and die by source control. It’s an excellent way to keep track of the status of your project, try out new features or ideas without jeopardizing the bits of your application that already work, and perhaps most importantly, it’s a life saver when you can’t figure out why in the hell you decided to do something that seemed like a good idea at the time but now seems like a truly retarded move. If you work with other developers, it’s also a great way to find out who to blame when the build is broken.

So why Git? Well, if time is on your side, go watch this 1 hour presentation by Linus Torvalds; I guarantee that if you know the first thing about source control, he will convince you to switch. If you don’t have that kind of time on your hands (and really, who does?) suffice it to say that Git plays really well with Github, and Github is like programming + social media + crack. Basically, it’s a website that stores your public (or private) repositories, showing off your code for all the world to see and fork and hack on top of. It also allows you to find and follow other interesting projects and libraries, and to receive updates when they make a change that you might be interested in.

Need a library to do fuzzy string matching? Search Git and find fuzzywuzzy. Install it into your working environment, and start playing with it. If it doesn’t do quite what you need, fork it, check out the source, and start hacking on it until it does! Github is an amazing way to expand your ability to rapid prototype and explore ideas that would take way too long to implement from scratch.

Get started by installing git with the command sudo apt-get install git-core. You should probably also skim through the git tutorial, as it will help you start off on the right foot.

Next, mosey on over to Github and sign up for an account. Seriously, it’s awesome, stop procrastinating and do it.

Step 3: Pip

I’ve already raved about the third-party libraries for Python, but what I haven’t told you yet is that there’s an insanely easy way to get those libraries into your working environment. Pip is like a repository just for Python libraries. If you’re already familiar with Linux, then you know what I’m talking about. Remember the earlier example of needing a fuzzy string matching library in your project? Well with pip, getting one is as easy as typing pip install fuzzywuzzy. This will install the fuzzywuzzy library on your system, and make it available to your application in one easy step.

But I’m getting ahead of myself here: You need to install pip before you can start using it. For that, you’ll need to run sudo apt-get install python-setuptools python-dev build-essential && sudo easy_install -U pip

The other cool thing about pip? When you’re ready to share your project with others (or just want to set up a development environment on another machine that has all of the necessary prerequisites to run it) you can run the command pip freeze > requirements.txt to create a file that describes all of the libraries that are necessary for your app to run correctly. In order to use that list, just run pip install -r requirements.txt on the target machine, and pip will automatically fetch all of your projects prerequisites. I swear, it’s fucking magical.

Step 4: Virtualenv

As I’ve already mentioned, one of my favourite things about Python is the availability of third-party libraries that enable your code to do just about anything with simple import statement. One of the problems with Python is that trying to keep all of the dependencies for all of your projects straight can be a real pain in the ass. Enter virtualenv.

This is an application that allows you to create virtual working environments, complete with their own Python versions and libraries. You can start a new project, use pip to install a whole bunch of libraries, then switch over to another project and work with a whole bunch of other libraries, all without different versions of the same library ever interfering with one another. This technique also keeps the pip requirements files that I mentioned above nice and clean so that each of your projects can state the exact dependency set that it needs to run without introducing cruft into your development environment.

Another tool that I’d like to introduce you to at this time is virtualenvwrapper. Just like the name says, it’s a wrapper for virtualenv that allows you to easily manage the many virtual environments that you will soon have floating around your machine.

Install both with the command pip install virtualenv virtualenvwrapper

Once the installation has completed, you’ll may need to modify your .bashrc profile to initialize virtualenvwrapper whenever you log into your user account. To do so, open up the .bashrc file in your home directory using your favourite text editor, or execute the following command: sudo nano ~/.bashrc 

Now paste the following chunk of code into the bottom of that file, save it, and exit:

# initialize virtualenv wrapper if present
if [ -f /usr/local/bin/virtualenvwrapper.sh ] ; then
. /usr/local/bin/virtualenvwrapper.sh
fi

Please note that this step didn’t seem to be necessary on Ubuntu 12.04, so it may only be essential for those running older versions of the operating system. I would suggest trying to use virtualenvwrapper with the instructions below before bothering to modify the .bashrc file.

Now you can make a new virtual environment with the command mkvirtualenv <project name>, and activate it with the command workon <project name>. When you create a new virtual environment, it’s like wiping your Python slate clean. Use pip to add some libraries to your virtual environment, write some code, and when you’re done, use the deactivate command to go back to your main system. Don’t forget to use pip freeze inside of your virtual environment to obtain a list of all of the packages that your application depends on.

Step 5: Starting a New Project

Ok, so how do we actually use all of the tools that I’ve raved about here? Follow the steps below to start your very own Python project:

  1. Decide on a name for your project. This is likely the hardest part. It probably shouldn’t have spaces in it, because Linux really doesn’t like spaces.
  2. Create a virtual environment for your project with the command mkvirtualenv <project name>
  3. Activate the virtual environment for your project with the command workon <project name>
  4. Sign into Github and click on the New Repository button in the lower right hand corner of the home page
  5. Give your new repository the same name as your project. If you were a creative and individual snowflake, the name won’t already be taken. If not, consider starting back at step 1, or just tacking your birth year onto the end of the bastard like we used to do with hotmail addresses back in the day.
  6. On the new repository page, make sure that you check the box that says Initialize this repository with a README and that you select Python from the Add .gitignore drop down box. The latter step will make sure that git ignores files types that need not be checked into your repository when you commit your code.
  7. Click theCreate Repository button
  8. Back on your local machine, Clone your repository with the command git clone https://github.com/<github user name>/<project name> this will create a directory for your project that you can do all of your work in.
  9. Write some amazing fucking code that blows everybody’s minds. If you need some libraries (and really, who doesn’t?) make sure to use the pip install <library name> command.
  10. Commit early, commit often with the git commit -am “your commit message goes here” command
  11. When you’re ready to make your work public, post it to github with the command git push https://github.com/<github user name>/<project name>
  12. Don’t forget to script out your project dependencies with the pip freeze > requirements.txt command
  13. Finally, when you’re finished working for the day, use the deactivate command to return to your normal working environment.

In Conclusion:

This post is way longer than I had originally intended. Suck it. I hope your eyes are sore. I also hope that by now, you’ve been convinced of how awesome a Python development environment can be. So get out there and write some amazing code. Oh, and don’t forget to check out my projects on github.




On my Laptop, I am running Linux Mint 12.
On my home media server, I am running Ubuntu 12.04
Check out my profile for more information.
Categories: Jon F, Programming Tags: , ,

lua + nginx + FastCGI on Debian

June 22nd, 2010 2 comments

(This was originally posted here on my personal blog.)

I’ve recently been doing some testing in lua, and have been comparing the results to the EdgeLink Consulting CMS that we’ve designed in PHP. So far this solution is able to serve substantially more requests per second than our current CMS. However, we haven’t really spent much time optimizing the CMS. The goal is to have a working copy first before any optimizations are done. We’ve also been working on some eCommerce modules for the platform.

With all that being said, I’d like to post a quick tutorial on how I got this setup. It was quite the task. Although there was a tutorial I found to do the same task, it was a little bit confusing. My tutorial will have a lot of the same steps, with some minor adjustments. This tutorial is written at an intermediate level. Some trivial steps have been omitted.

NOTE: This has been tested with Debian 5.0.4 (Stable)

  1. Install nginx

    apt-get install nginx

    We’ll have to do some modifications later on to add the FastCGI handler. For simplicity we will keep the web path to “/var/www/nginx-default” and listen on port 8081 in case you have another webserver running on port 80.

  2. Install lua 5.1 (and WSAPI libraries)

    apt-get install lua5.1 liblua5.1-wsapi-fcgi-0 liblua5.1-coxpcall0 liblua5.1-filesystem0

    apt-get install liblua5.1-wsapi-doc

    Can’t do much testing without this. Note: The second line is not necessary if you are running Debian testing, and get the liblua5.1-wsapi-fcgi-1instead.

    EDIT: You’ll notice that I added in liblua5.1-filesystem0. Steve pointed out that there is a bug in liblua5.1-wsapi-fcgi-0. It doesn’t include it as a dependency. He reported this as a bug here, and it was fixed in liblua5.1-wsapi-fcgi-1.

  3. Install spawn-fcgiIf you’re running Debian testing you may be able to get spawn-fcgi through the distribution, however, I just downloaded it and compiled from source.

    wget http://www.lighttpd.net/download/spawn-fcgi-1.6.3.tar.gz
    tar -xzvf spawn-fcgi-1.6.3.tar.gz
    cd spawn-fcgi-1.6.3.tar.gz
    ./configure
    make
    make install

  4. Create a FastCGI Socket

    spawn-fcgi -F 4 -u www-data -s /var/run/nginx-fcgi.sock -P /var/run/nginx-fcgi.pid — /usr/bin/wsapi.fcgi

    For the sake of simplicity, we will just spawn it manually for now. If you’re feeling crafty you can add the above line to the start condition in/etc/init.d/nginx, and the line below to the stop condition. You can add both of them to restart.

    cat /var/run/nginx-fcgi.pid | xargs -n 1 kill

  5. Create a lua file in /var/www/nginx-default/In this tutorial, use hello.lua. You can change this to whatever, you want but just make sure you make the modification in the nginx configuration below as well.
  6. Edit /etc/nginx/sites-available/defaultNow let’s add the code that will point nginx to the correct file. For simplicity, we will simply point it to hello.lua. You can change this to anything, or simply modify the code to accept any *.lua file, as seen in the tutorial listed above. Here is the top of my default file:

    listen   8081 default;
    server_name  localhost;
    access_log  /var/log/nginx/localhost.access.log;

    location / {
    fastcgi_pass    unix:/var/run/nginx-fcgi.sock;
    fastcgi_param   SCRIPT_FILENAME “/var/www/nginx-default/hello.lua”;
    fastcgi_param   PATH_INFO       $request_uri;
    fastcgi_param   QUERY_STRING    $query_string;
    fastcgi_param   REQUEST_METHOD  $request_method;
    fastcgi_param   CONTENT_TYPE    $content_type;
    fastcgi_param   CONTENT_LENGTH  $content_length;
    }

  7. Restart nginx

    /etc/init.d/nginx restart

  8. Visit http://localhost:8081/Congratulations! You should now see hello.lua.

If you have any problems, post in the comments. Stay tuned for more related posts.

Programming on Linux

September 27th, 2009 No comments

Now that school as resumed I am getting to spend a lot of time with my Linux install doing day to day productive tasks. The most recent thing that I have had to deal with is programming on Linux. As part of my Computer Graphics class the professor recommended that we install Dev-C++ and GLUT (with related libraries) so that we can code some OpenGL goodness. Well seeing as Dev-C++ is a Windows only IDE that just won’t do.

Instead I opted to install the C and C++ development tools for Eclipse. This works perfectly and within minutes I had a simple “Hello, world!” program up and running. In the past I had only ever used Eclipse for Java programming, however that may be changing permanently in the future.

Next up I had to install GLUT. After a quick search in my Fedora repositories I only had the option to install freeglut listed. So I figured ‘what the heck’ and gave it a try anyway. To my surprise this worked perfectly, even when I still referenced #<GL/glut.h>. This means I can use all of this great open source software to develop the same C++ code that I can then submit to my professor to mark on his Window’s machine.

The only issue I have found is I cannot for the life of me get MinGW to compile the code to a Windows exe. Yet even barring this I must say that all in all I am very impressed!