Some free code editors for Macs (that work in a UC Davis computer lab)

Every year I help teach a course[1] to grad students that hopefully leaves them with an understanding of how to use Unix and Perl to solve bioinformatics-related problems. We use a Mac-based computer lab because all Macs come with Unix and Perl installed. Many of our students are new to programming and many are new to Macs. Because of this, and because they need to use a code editor to write their Perl scripts, we have previously pointed them towards Fraise. Despite its age [2], this relatively lightweight editor has proved fine for the last few years that we have taught this course.

This year, however, Fraise proved problematic. The computer lab has now upgraded to OS X 10.8 which provides extra safeguards about what apps can be run. This Gatekeeper technology has been set up to only allow ‘signed’ apps[3]. The version of Fraise that we were using required administrator privileges for it to be opened (not possible in this computer lab).

My first thought was to direct students to download and install TextWrangler. This is an excellent, powerful, and well maintained code editor for Macs. Most importantly, it is free and also a signed app. However, it does try to install a helper app which caused a persistent dialog window to keep popping up during the installation. Clicking ‘cancel’ worked…but only after about 20 attempts[4]. I like TextWrangler as an app, but prefer the cleaner look of Fraise. So today I set out to find code editors for Macs that:

  • were free
  • could be run on the Macs in our computer lab (i.e. had to be signed apps)
  • were relatively simple to use and/or were easy on the eye

Here is what I came up with. These are all apps that seem to be under current development (updated at some point in 2013):

AppSize in MB Free? Notes

Komodo Edit301.1YesBig because it is part of a larger IDE tool which is not free[5]

Sublime Text 227.3sort of[6]Gaining in popularity (a version 3 beta is also available)

TextMate 230.3YesWhile this is technically an ‘alpha’[7] release, it seems very stable.

TextWrangler19.2YesVery robust and venerable app. Free since 2005

Tincta 25.6YesSmall app, similar to Fraise in appearance

 

If I had to suggest one, it would probably be Sublime Text 2 (though I will encourage students to buy this if they like it). TextMate 2 is a good second choice, particularly because it is also a very clean looking app. Of course, at some point we need to tell students about the joys of real text editors such as vi, vim, and emacs…but of course this might lead to hostilities![8]

  1. This course material is available for free and became the basis for our much more expansive book on the same topic  ↩

  2. Fraise is itself a fork of Smultron which stopped development in 2009 but which reappeared as a paid app in the Mac App Store in 2011.  ↩

  3. Those apps that are approved by Apple, even if they are not in the Mac App Store.  ↩

  4. Seriously, it takes a lot of clicks to make this dialog box go away. It then produces more pop-up dialogs asking whether you want to register, or install command-line tools.  ↩

  5. Currently $332 for a single license  ↩

  6. This is a paid app, but can be used in trial mode indefinitely with occasional reminders.  ↩

  7. TextMate 2 has been in alpha status since 2009  ↩

  8. Editor wars should be avoided if possible  ↩

Thinking of writing a FASTA parser? 12 scenarios that might break your code

In today's Bits and Bites Coding class, we talked about the basics of FASTA and GFF file formats. Specifically, we were discussing the problems that can arise when you write scripts to parse these files, and the types of problems that might be present in the file which may break (or confuse) your script.

Even though you can find code to parse FASTA files from projects such as BioPerl, it can be instructive to try to do this yourself when you are learning a language. Many of the problems that occur when trying to write code to parse FASTA files will fall into the 'I wasn't expecting the file to look like that' category. I recently wrote about how the simplicity of the FASTA format is a double-edged sword. Because almost anything is allowed, it means that someone will — accidentally or otherwise — produce a FASTA file at some point that contains one of the following 12 scenarios. These are all things that a good FASTA parser should be able to deal with and, if necessary, warn the user:

> space_at_start_of_header_line
ACGTACGTACGTACGT

>Extra_>_in_FASTA_header
ACGTACGTACGTACGT

>Spaces_in_sequence
ACGTACGT ACGTACGT

>Spaces_in_sequence_and_between_lines
A C 

G T 

A C

A G 

A T

>Redundant_sequence_in_header_ACGTACGTACGT
ACGTACGTACGTACGT

><- missing definition line
ACGTACGTACGTACGT

>mixed_case
ACGTACGTACGTACGTgtagaggacgcaccagACGTACGTACGTACGT

>missing_sequence

>rare, but valid, IUPAC characters 
ACGTRYSWKMBDHVN

>errant sequence
Maybe I accidentally copied and pasted something
Maybe I used Microsoft Word to edit my FASTA sequence

>duplicate_FASTA_header
ACGTACGTACGTACGT
>duplicate_FASTA_header
ACGTACGTACGTACGT

>line_ending_problem^MACGTACGTACGTACGT^MACGTACGTACGTACGT^M>another_sequence_goes_here^MACGTACGTACGTACGT^MACGTACGTACGTACGT

The FASTA file format: a showcase for the best and worst of bioinformatics coding practices

The FASTA file format is one of the oldest recognized formats in bioinformatics and has become the lingua franca when trying to store sequence information in a plain text format. It is probably true to say that many people are much more likely to know of FASTA (the file format) than FASTA (the program). The FASTA program, a tool for comparing DNA or protein sequences, was first described by Bill Pearson and David Lipman back in 1988. However, FASTA (the program) was itself a successor to the original FASTP program that was described a few years earlier by the same authors in 1985.

The FASTA tool is still a going concern and the latest version (v36.3.5e) was released in May 2011. Thinking of FASTA as a single tool is a little misleading; it's really a package with over a dozen different programs. Part of the success of the FASTA software is perhaps due to the simplicity of the file format (which merits its own Wikipedia page). Lets look to see what the FASTA manual says about the format:

FASTA format files consist of a description line, beginning with a ’>’ character, followed by the sequence itself
...
All of the characters of the description line are read, and special characters can be used to indicate additional information about the sequence. In general, non-amino-acid/non-nucleotide sequences in the sequence lines are ignored.

You will note the absence of the word 'unique' in this description and this brings us to the issue that I really want to talk about. Today, Vince Buffalo sparked an interesting conversation thread on twitter:

It's not controversial if I say every FASTA ID should always be treated as a primary, unique foreign key, ya?— Vince Buffalo (@vsbuffalo) June 24, 2013

The FASTA format does not specify whether the description line (sometimes known as the 'header line' or 'definition line') should be unique or not. In some cases, it may not matter. In other cases, it clearly does. I find this issue — which probably doesn't sound very complex — gets to the heart of what is good and bad about bioinformatics. This is because it reveals a lot about the programmers who write code to work with FASTA-formatted files, and about how they deal with error handling.

Let's consider the following hypothetical FASTA file (which contains some overly simplistic DNA sequences):

>AT1G01140.1 (version 1)
AACCGGTT
>AT1G01140.1 (version 2)
TTGGCCAA
>AT1G01140.2 (version 1)
CCGGAATT
>AT1G01140.2 (version 1)
CCGGAATT

The 3rd and 4th entries are clearly duplicated (both FASTA header and sequence), and the other entries have similar. but unique, headers. Of course, the person who is using this sequence file may not be aware of this (a FASTA file could contain millions of sequences). So question number 1 is:

Who's job is it to check for unique FASTA headers? The person who generated the data file, or the person who wrote the software tool that has to process it?

We frequently generate FASTA tools directly from websites and databases so I think that we — bioinformatics users — have increasingly put too much faith in assuming that someone else (the person who wrote the FASTA-exporting code) has already tackled the problem for us.

This problem starts to get more complex when I tell you that I have used various bioinformatics tools in the past that have done one of two things with FASTA headers:

  1. Ignore any characters beyond the nth character
  2. Ignore any characters after whitepsace

In both of these situations, a program might consider the above file to contain only two unique entries (ignores after whitespace), or even one unique entry (ignores after 8 characters). This is clearly dangerous, and if the program in question needs to use a unique FASTA header as a key in hash, then duplicate headers may cause problems. Hopefully, a good programmer won't impose arbitrary limits like this. However, this brings me to question number 2:

How should a program deal with duplicate FASTA headers?

There are many options here:

  1. Refuse to run without telling you what the problem is (I've seen this before)
  2. Refuse to run and tell you that there are duplicate headers
  3. Refuse to run and tell you which headers are duplicated
  4. Run and simply ignore any entries with duplicate headers
  5. Run and simply ignore any entries with duplicate headers, and report the duplicates
  6. Check whether entries with duplicate headers also have duplicate sequences. This could potentially reveal whether it is a genuine duplication of the entire entry, or whether different genes/sequences just happen to share the same name/identifier (this is possible, if not unlikely, with short gene names from different species).
  7. Check for duplicate headers, report on duplicates, but don't stop the program running and instead add an extra character to duplicate headers to make them unique (e.g. append .1, .2, .3 etc)

The last option is the most work for the programmer but will catch most errors and problems. But how many of us would ever take that much time to catch what might be user error, but which might be a glitch in the FASTA export routine of a different program?

This whole scenario gets messier still when you consider that some institutions (I'm looking at you NCBI) have imposed their own guidelines on FASTA description lines. In the original twitter thread, Matt MacManes suggested that FASTA headers be no longer than 100 characters. Sounds good in practice, but then you run into entries from databases like FlyBase such as this one:

>FBgn0046814 type=gene; loc=2L:complement(9950437..9950455); ID=FBgn0046814; name=mir-87; dbxref=FlyBase:FBgn0046814,FlyBase:FBan0032981,FlyBase_Annotation_IDs:CR32981,GB:AE003626,MIR:MI0000382,flight:FBgn0046814,flymine:FBgn0046814,hdri:FBgn0046814,ihop:1478786; derived_computed_cyto=30F1-30F1%3B Limits computationally determined from genome sequence between @P{EP}CG5899<up>EP701</up>@ and @P{EP}CG4747<up>EP594</up>@%26@P{lacW}l(2)k13305<up>k13305</up>@; gbunit=AE014134; MD5=10a65ff8961e4823bad6c34e37813302; length=19; release=r5.21; species=Dmel;
TGAGCAAAATTTCAGGTGT

That's 555 characters of FASTA description for 20 characters of sequence. You would hope that this FASTA header line is unique!