Six useful things I have learnt at Software Carpentry boot camp

Monday and Tuesday this week, I attended a Software Carpentry boot camp at the University of Southampton. Topics covered included bash, shell scripting, Git and writing/testing reproducible code. The event was well-attended and whilst making an event like this relevant to everyone is obviously a challenge, they did a great job. I certainly learnt some useful tips even for the topics that I knew a fair bit about.

As well as the SWC presentations themselves, it was good to get tips from fellow programmers as well as "UNIX and Perl to the Rescue!: A Field Guide for the Life Sciences (and Other Data-rich Pursuits)", which I brought along for some browsing. Here is half a dozen random things that I picked up over the two days...

1. Useful grep flags. I've used grep to pull out matches from text files a fair amount and was familiar with a few of the flags that I use a lot:

-i = case-insensitive search
-v = inverse search
-A n = return n lines after match
-B n = return n lines before match

I learnt a few more useful ones, though:

-w = match whole words only
-n = show line numbers
--color = highlight matches in colour
-r = search subdirectories (recursive)

2. Useful ls flags. ls is another must-have part of the UNIX users toolkit. There were still a few useful flags with which I was unfamiliar and am likely to use in future, though:

-G = colour-code directory contents
-F = appends / to directories
-R = recursive ls including subdirectory contents
-1 = one entry per line
-a = show all files including hidden files

Along with my old favourites, of course:

-l = "long" mode (more info)
-r = reverse sort
-t = sort by time
-S = sort by size
-h = user-friendly file sizes

3. Catching the standard error and standard output. I've seen this a few times but for some reason this was the first time it really sunk in. Most UNIX users will be familiar with redirecting the standard output from a command into a file using > file (or >> to append). Catching the standard error is less obvious/common. This can be done using 2> to catch the stderr alone, or &> to catch both stdout and stderr at the same time. If you want to redirect them both into different files, do something like this:

[cmd] 2> error.txt > stdout.txt

The stdout will slip by the first redirect and then get caught by the second. Of course, you can have those the other way round if you wish!

4. Navigating the terminal. This one actually came from "UNIX and Perl to the Rescue" but I discovered it at the boot camp:

Ctrl+a = move to start of line
Ctrl+e = move to end of line
Ctrl+w = delete previous word
Ctrl+l = clear screen (clear works too)
Ctrl+r = search through previous commands one letter at a time (this was an SWC revelation)

Using Ctrl+a and Ctrl+e seems to work in a number of other Mac editors too - including this blogger HTML window! If you are a Mac user, just make sure that you don't use cmd+w by mistake - this will close the current terminal window with much wailing and gnashing of teeth!

5. Reversing Python strings. In the past, I have reversed a python string by converting into a list, using the list.reverse() method and then string.join() to convert it back. But there is a better way!

string[::-1]

I've used string slicing loads before but never realised that you could add a step to this syntax as with the range() method! (string[::-1] translates as string[start=end:end=start:step=-1].) Now I am wondering where else in my code I can use this knowledge!

6. Changing the command prompt to $. Sometimes, the command prompt is too long (or the terminal window too narrow) so that almost every command wraps around in an annoying fashion. To replace with a simple $ character, just type:

PS1="\s"
PS1="\$ "


I must admit that I do not (yet) really understand exactly what this does and how it works but it does! (And if you now get lost as to where you are, just remember that pwd gives you the full path to the working directory.

There were more but I think that's enough for now! If this is the sort of thing that floats your boat, you can find useful stuff like this plus a whole bunch of lessons at the Software Carpentry website:

If you get the chance to attend a boot camp yourself, my advice is: do!