If you’re a programmer and you use Linux but you haven’t yet entered the amazing world that is Python development, you’re really missing out on something special. For years, I dismissed Python as just another script kiddie language, eschewing it for more “serious” languages like Java and C#. What I was missing out on were the heady days of rapid development that I so enjoyed while hacking away on Visual Basic .NET in my early years of university.
There was a time when nary a day would go by without me slinging together some code into a crappy Windows Forms application that I wrote to play with a new idea or to automate an annoying task. By and large, these small projects ceased when I moved over to Linux, partially out of laziness, and partially because I missed the rapid prototyping environment that Visual Basic .NET provided. Let’s face it – Java and C# are great languages, but getting a basic forms app set up in either of them takes a significant amount of time and effort.
Enter Python, git/github, pip, and virtualenv. This basic tool chain has got me writing code in my spare time again, and the feeling is great. So without further ado, let me present (yet another) quick tutorial on how to set up a bad-ass Python development environment of your own:
Step 1: Python
If there’s one thing that I really love about Python, it’s the wide availability of libraries for most any task that one can imagine. An important part of the rapid prototyping frame of mind is to not get bogged down on writing low-level libraries. If you have to spend an hour or two writing a custom database interface layer, you’re going to lose the drive that got you started in on the project in the first place. Unless of course, the purpose of the project was to re-invent the database interface layer. In that case, all power to you. In my experience, this is never a problem with Python, as its magical import statement will unlock a world of possibilities that is occupied by literally thousands of libraries to do most anything that you can imagine.
Install this bad boy with the simple command sudo apt-get install python and then find yourself a good python tutorial with which to learn the basics. Alternatively, you can just start hacking away and use StackOverflow to fill in any of the gaps in your knowledge.
Step 2: Git/Github
In my professional life, I live and die by source control. It’s an excellent way to keep track of the status of your project, try out new features or ideas without jeopardizing the bits of your application that already work, and perhaps most importantly, it’s a life saver when you can’t figure out why in the hell you decided to do something that seemed like a good idea at the time but now seems like a truly retarded move. If you work with other developers, it’s also a great way to find out who to blame when the build is broken.
So why Git? Well, if time is on your side, go watch this 1 hour presentation by Linus Torvalds; I guarantee that if you know the first thing about source control, he will convince you to switch. If you don’t have that kind of time on your hands (and really, who does?) suffice it to say that Git plays really well with Github, and Github is like programming + social media + crack. Basically, it’s a website that stores your public (or private) repositories, showing off your code for all the world to see and fork and hack on top of. It also allows you to find and follow other interesting projects and libraries, and to receive updates when they make a change that you might be interested in.
Need a library to do fuzzy string matching? Search Git and find fuzzywuzzy. Install it into your working environment, and start playing with it. If it doesn’t do quite what you need, fork it, check out the source, and start hacking on it until it does! Github is an amazing way to expand your ability to rapid prototype and explore ideas that would take way too long to implement from scratch.
Get started by installing git with the command sudo apt-get install git-core. You should probably also skim through the git tutorial, as it will help you start off on the right foot.
Next, mosey on over to Github and sign up for an account. Seriously, it’s awesome, stop procrastinating and do it.
Step 3: Pip
I’ve already raved about the third-party libraries for Python, but what I haven’t told you yet is that there’s an insanely easy way to get those libraries into your working environment. Pip is like a repository just for Python libraries. If you’re already familiar with Linux, then you know what I’m talking about. Remember the earlier example of needing a fuzzy string matching library in your project? Well with pip, getting one is as easy as typing pip install fuzzywuzzy. This will install the fuzzywuzzy library on your system, and make it available to your application in one easy step.
But I’m getting ahead of myself here: You need to install pip before you can start using it. For that, you’ll need to run sudo apt-get install python-setuptools python-dev build-essential && sudo easy_install -U pip
The other cool thing about pip? When you’re ready to share your project with others (or just want to set up a development environment on another machine that has all of the necessary prerequisites to run it) you can run the command pip freeze > requirements.txt to create a file that describes all of the libraries that are necessary for your app to run correctly. In order to use that list, just run pip install -r requirements.txt on the target machine, and pip will automatically fetch all of your projects prerequisites. I swear, it’s fucking magical.
Step 4: Virtualenv
As I’ve already mentioned, one of my favourite things about Python is the availability of third-party libraries that enable your code to do just about anything with simple import statement. One of the problems with Python is that trying to keep all of the dependencies for all of your projects straight can be a real pain in the ass. Enter virtualenv.
This is an application that allows you to create virtual working environments, complete with their own Python versions and libraries. You can start a new project, use pip to install a whole bunch of libraries, then switch over to another project and work with a whole bunch of other libraries, all without different versions of the same library ever interfering with one another. This technique also keeps the pip requirements files that I mentioned above nice and clean so that each of your projects can state the exact dependency set that it needs to run without introducing cruft into your development environment.
Another tool that I’d like to introduce you to at this time is virtualenvwrapper. Just like the name says, it’s a wrapper for virtualenv that allows you to easily manage the many virtual environments that you will soon have floating around your machine.
Install both with the command pip install virtualenv virtualenvwrapper
Once the installation has completed, you’ll may need to modify your .bashrc profile to initialize virtualenvwrapper whenever you log into your user account. To do so, open up the .bashrc file in your home directory using your favourite text editor, or execute the following command: sudo nano ~/.bashrcÂ
Now paste the following chunk of code into the bottom of that file, save it, and exit:
# initialize virtualenv wrapper if present
if [ -f /usr/local/bin/virtualenvwrapper.sh ] ; then
. /usr/local/bin/virtualenvwrapper.sh
fi
Please note that this step didn’t seem to be necessary on Ubuntu 12.04, so it may only be essential for those running older versions of the operating system. I would suggest trying to use virtualenvwrapper with the instructions below before bothering to modify the .bashrc file.
Now you can make a new virtual environment with the command mkvirtualenv <project name>, and activate it with the command workon <project name>. When you create a new virtual environment, it’s like wiping your Python slate clean. Use pip to add some libraries to your virtual environment, write some code, and when you’re done, use the deactivate command to go back to your main system. Don’t forget to use pip freeze inside of your virtual environment to obtain a list of all of the packages that your application depends on.
Step 5: Starting a New Project
Ok, so how do we actually use all of the tools that I’ve raved about here? Follow the steps below to start your very own Python project:
- Decide on a name for your project. This is likely the hardest part. It probably shouldn’t have spaces in it, because Linux really doesn’t like spaces.
- Create a virtual environment for your project with the command mkvirtualenv <project name>
- Activate the virtual environment for your project with the command workon <project name>
- Sign into Github and click on the New Repository button in the lower right hand corner of the home page
- Give your new repository the same name as your project. If you were a creative and individual snowflake, the name won’t already be taken. If not, consider starting back at step 1, or just tacking your birth year onto the end of the bastard like we used to do with hotmail addresses back in the day.
- On the new repository page, make sure that you check the box that says Initialize this repository with a README and that you select Python from the Add .gitignore drop down box. The latter step will make sure that git ignores files types that need not be checked into your repository when you commit your code.
- Click theCreate Repository button
- Back on your local machine, Clone your repository with the command git clone https://github.com/<github user name>/<project name> this will create a directory for your project that you can do all of your work in.
- Write some amazing fucking code that blows everybody’s minds. If you need some libraries (and really, who doesn’t?) make sure to use the pip install <library name> command.
- Commit early, commit often with the git commit -am “your commit message goes here” command
- When you’re ready to make your work public, post it to github with the command git push https://github.com/<github user name>/<project name>
- Don’t forget to script out your project dependencies with the pip freeze > requirements.txt command
- Finally, when you’re finished working for the day, use the deactivate command to return to your normal working environment.
In Conclusion:
This post is way longer than I had originally intended. Suck it. I hope your eyes are sore. I also hope that by now, you’ve been convinced of how awesome a Python development environment can be. So get out there and write some amazing code. Oh, and don’t forget to check out my projects on github.
Awesome post, Jon. Reminded me to get my Github repositories set up at home and move some code from SourceForge over so that a less lazy developer can take over if they’re interested. Here are some of my individual tips, tricks and usage with Git if you or anyone else finds them useful:
OS X: I still heavily rely on OS X for both work and personal development and scripting. One application that I use when I’m not interested in wrangling with the command line is SourceTree by Atlassian, the same folks behind JIRA. It’s a slick OS X UI that makes committing new changes and pushing to origin repositories quite easy, even for people who are just using Git as a Subversion replacement. SourceTree natively supports GitHub, as well as…
BitBucket: Atlassian also has a hosted Git and Mercurial solution called BitBucket. The major advantage over GitHub is that their free account offers up to five users and unlimited private repositories, so if you do need to have a more limited-access project without exposing your terrible code to the world, it can still be done for free.
Python: Yes, even a sloppy sysadmin like myself can write Python. The language really lends itself to common maintenance and management tasks, and rather than messing with Bash syntax, Python always gives you the option to extend your scripts. I have a few neat examples available on my GitHub repository. You can use it to fix WordPress permissions on dedicated or VPS environments and let users auto-update their installations; quickly generate SSL certificate requests for NameCheap, or convert a bunch of values into an SQL ‘IN’ statement.
Just as a side note, finding a Git GUI for Linux has been infuriating. My solution has been to use SmartGit – there is an Ubuntu PPA available that installs the app and all the necessary Java dependencies:
sudo add-apt-repository ppa:eugenesan/ppa
sudo apt-get update
sudo apt-get install smartgit
Don’t forget “modern python project”.
Upon finishing this write up I very quickly bookmarked it with Diigo, shared on Google+, and sent to Facebook, App.net and Twitter (oh and I subscribed to the RSS feed of the site). If you couldn’t tell, I thoroughly enjoyed it and look forward to putting these steps into action.
Currently 2 out of 4 of my computers are Linux machines and the only book in my hands right now is Learning Python. This could not have come at a more appropriate time. Thank you very much!
Just to finish this with oneliner…
nobody sane uses is in production enviroment on a big project and if you still think python is the way to go, you obviously haven’t done anything useful that has more than 10k lines of code.
@juzer:
You’re aware that Reddit is written in Python, right? They switched to Python as a platform in 2005, and haven’t looked back. They also serve roughly 50k uniques per hour. If that’s not performance, I don’t know what is. I would never argue that Python is as fast as C, Java, or even C#, but it can hold its own in a production environment, and to suggest otherwise is to show a strong bias that is not backed by any real data.
Sources:
http://www.codinghorror.com/blog/2007/04/reddit-language-vs-platform.html
http://www.digitaltrends.com/web/how-big-is-reddit-very-very-big-indeed/
I know this is a Linux blog first and foremost, but many of us also run Windows machines for various tasks (I’m typing this on one right now) and if one of those tasks happens to be coding, then you may be interested in Github for Windows. It’s a very pretty GUI that allows you to do any common source control task, so long as your code is hosted on Github. Check it out at http://windows.github.com/
Been impressed (versus other programming languages) with local user group activities (not just the U.S.), especially at Meetup – http://python.meetup.com/ as well as job openings for Python jobs at Dice.com.
Also, have you ever installed a ‘base’ Linux distro and seen how many scripts in /usr/bin, /bin, and /sbin are Python?
@Jon F
And they have used lisp until then. Everything is used, ergo everything is great?
We have 35+k users (IT/Telecommunication company) and we are optimizing our (somebody else’s) applications that are written in python since they aren’t fast enough in critical areas. I just discard an app, and then develop it in PHP.
Reddit is fast? Tell them to switch to PHP. At least PHP doesn’t break compatibility with their own language. Ergo, somebody knew what he was doing.
I like Python, used it ever since i moved to Linux. Used and liked it so much that I never got the hang of bash scripting. I find it works reasonably well with the Kdevelop IDE but I suppose everyone has his/hers own.
I’ve heard comments before that it’s not for a production environment or shouldn’t be used for large projects but I’ve no idea where that comes from. Ok, Python doesn’t use curly braces but apart from that it’s a programming language like every other.
The success or failure of a project rarely depends on the chosen language but more about the people and how they work. I appreciate a post like this which doesn’t just advocate the use of Python but also tools to create a professional workflow.
@juzer
It sounds to me like you’re the type of fellow who has already made up his mind about the differences between the two languages and simply isn’t willing to look outside of his comfort zone for new inspiration.
Both are scripting languages, neither hold a candle to a “real” language like C in terms of performance, and I can dig up performance tests that show that each is better than the other by some metric or in some suite of tests.
As far as breaking compatibility with their own language goes, I’ll grant that the jump between Python 2 and 3 is significant, but there have been similar changes to the structures of both Java and the .NET framework over their lifecycles as well. No big deal.
At the end of the day, Caesar has it right:
A good programmer can write good code in any language – it’s all about choosing the thing that best suits the job at hand, and for me, when rapid prototyping at home, Python is the way to go.
@Jake
What is wrong with gitk? 1) it is included in standard git; and 2) it is functionally the best of all the linux gui clients. And at least it does not look dog terrible now using TK8.5+.
@Caesar Tjalbo
Thanks for the comment about tools and workflow. I write C# all day at work, and Visual Studio is a big reason for my love of that language. It’s a great tool that gets out of my way and allows me to quickly write code, but also provides a good deal of support.
The toolchain that I advocate for Python does a similar task. When combined with a good text editor like Sublime Text 2, the situation gets even better still.
I do wish that somebody would pick up support for Netbeans’ Python plugin, because I use Netbeans for Java and really like it. Unfortunately, the plugin isn’t officially supportive, and I’ve found that it causes a lot of unecessary crashes.
@Jon F
Why don’t you try Eclipse/Aptana? It packs full IDE features, works great and allows for GUI version control/diff with Mercurial with its MercurialEclipse plugin. Comes with a really good debugger, too.
>sudo apt-get install python
Ubuntu Linux. This isn’t how you install python on “Linux”, this is how you install it on Ubuntu. You really only needed to mention the package names and told the reader to use their distro’s package manager to install it. If they don’t know how to install a package on Linux, I doubt they’re ready/willing to plunge head-first into python software development.
On another note, is git (and any other VCS) overkill for a single person writing code? With all that checking in, checking out, obscure commands – it’s like learning a micro language itself. It seems most hobby programmers would be fine with commenting in/out sections of code or just saving files as program1, program2, program3, etc. At best, and IDE needs a way on save to compute the diff of the current and previous version and allow the user to include comments with the save about what was changed and why. The whole VCS concept is quite a lot of overhead for local development by one person on their own code. I miss the old VAX VMS versioning file system that automatically saved multiple copies of a file; you could just go back and pull up an earlier version if you decided you’d made a serious mistake with your code (or anything else).
In your new project setup, if you want to create a new repository in github without actually going to github you can use their API and do a single curl call.
$ curl -u ‘my-github-username’ https://api.github.com/user/repos -d ‘{“name”:”my_repo”}’
Here is the slide giving the whole flow for a new git project http://image.slidesharecdn.com/deploy-120824012755-phpapp01/95/slide-10-728.jpg?1345791466 from a presentation I did on getting up and going with Python, Boto and Amazon http://engineerwithoutacause.com/presentation-64klab-with-64theihills-aug-23rd.html
@juzer PHP vs. Python? Srsly?
The instruction
sudo apt-get install python-setuptools python-dev build-essential && easy_install -U pip
should be
sudo apt-get install python-setuptools python-dev build-essential && sudo easy_install -U pip
Add sudo in front of easy_install.
Very good article with a very simple and clear roadmap to code in python.
A good and free python IDE I would recommend is Komodo edit.
http://www.activestate.com/komodo-edit
If you think Python is great, just wait until you try some higher-level languages! 🙂
A missing installation instruction is to add
# initialize virtualenv wrapper if present
if [ -f /usr/local/bin/virtualenvwrapper.sh ] ; then
. /usr/local/bin/virtualenvwrapper.sh
fi
in your .bashrc after having run the pip install virtualenv virtualenvwrapper command.
On Ubuntu 10.04LTS I had to run it with sudo to complete. Otherwise I get a permission error.
What do you think about Django? The framework is based on Python.
@John
I couldn’t agree more. The user communities around Python on both Meetup.com and GitHub are really very impressive.
@Joseph
You’re right, apt-get is really only applicable to debian-based systems like Ubuntu and Linux Mint. It happened to be what I was using while writing the tutorial, and it also happens to be what the majority (citation needed) of casual Linux users run at home, so it made sense for me to target the post at them.
With regards to your comment about source control systems: I honestly think that everybody should be using a source control system of some kind for all of their projects for the following reasons:
@Esdras Salazar
I’m a huge fan of Django. It totally changed my attitude toward web development, and I have completed one successful project with it. I’m currently working on a new project that is based around CherryPy, which is equally cool, but more about a stripped down framework and less about an all-encompassing platform.
@Chmike
Thanks for the corrections. The post has been modified to include them.