Showing posts with label work. Show all posts
Showing posts with label work. Show all posts

Monday, 29 February 2016

Thank YOU, PLOS ONE!

There are many flaws with the peer review system but it remains the best system we have for ensuring a certain degree of quality control prior to publication. One of the ways that the system could be improved is better recognition - and therefore motivation - for reviewers.

Ideally, there would be some form of payment, but I find it hard to see this happening any time soon. (It is difficult enough to get funds to publish papers - getting funds to get papers reviewed when they might well end up getting rejected is going to be way harder.)

The next best thing is some kind of reward or recognition. Some journals give discounted publication fees to reviewers, which is a great idea. Another great idea has just been put into action by PLOS ONE*: public recognition for reviewers:

On behalf of PLOS and the PLOS ONE editorial team, I would like to thank you for participating in the peer review process this past year at PLOS ONE.

We know there are many claims on your time and expertise and we very much appreciate your valuable input in 2015. With your help, we have continued to publish an influential, lively and highly accessed Open Access journal. Simply put, we could not do it without you and the thousands of other volunteers for PLOS ONE and the other PLOS journals who graciously contributed time reviewing manuscripts.

A public “Thank You” to our 2015 reviewers – including you – was published earlier this week.

(2016) PLOS ONE 2015 Reviewer Thank You. PLoS ONE 11(2): e0150341. doi:10.1371/journal.pone.0150341

Your name is listed in the Supporting Information file associated with the article. I hope that you will be able to use this letter, along with the article citation, to claim the credit and recognition you deserve within your institution for supporting PLOS ONE and Open Access publishing.

The article itself is short but sweet:

PLOS and the PLOS ONE editorial team would like to express our gratitude to all those individuals who participated in the peer review process of submissions to PLOS ONE over this past year. During 2015 PLOS ONE published over 28,000 research articles. This would not have been possible without the contribution of more than 76,000 reviewers from around the world and a wide range of disciplines.

The names of our 2015 PLOS ONE reviewers are listed in S1–S5 Reviewer Lists. Thank you to all our reviewers for generously sharing your time, insight and expertise with PLOS ONE authors in the evaluation of their work. Your efforts are a key reason for PLOS ONE’s success as an innovative and influential publication.

It’s nice to be appreciated. One more reason to be a fan of PLOS ONE. (Which I am, despite those who look down their noses at the journal because of its “scientifically rigorous research, regardless of novelty” policy.)

*The other PLOS journals did it to but I did not review anything for them this year.

Sunday, 1 November 2015

A tale of two conferences - #ABACBS2015 and #AGTA15

Two weeks ago, I attended the awesome double-header of ABACBS2015 and AGTA15. In some ways they were chalk and cheese - ABACBS was cheap and cheerful, where as AGTA was expensive and (therefore) exclusive - but both were great and highlighted some of the very best features of successful academic conferences.

As part of the ABACBS2015 organising committee, I needed to collect my thoughts for a debrief, so I thought I’d got down some thoughts here. (Thanks to grant writing, the post itself got a little delayed!) As with biology itself, most decisions are trade offs and the following comparison is not to criticise - I just find it interesting. As with the awesome ABiC14 conference last year, I mainly want to document good practice for future reference. (Some of these will no doubt be repeats of my ABiC14 thoughts!)

Size. Both conferences were, for me, the perfect size - around 180-190 people. This is enough to give the conference a good buzz and ensure that there are sufficient interesting people to listen to and posters to visit. Critically, though, it was small enough that you could find and speak to the people you wanted to. (Even if I didn’t fully get the chance at ABACBS2015 because I was busy, and managed to miss some people at AGTA15 because time ran out!)

Venue. ABACBS2015 was run on a budget and we were very fortunate to have the Garvan Institute provide a free venue for the conference. The Garvan lecture theatre is lovely, with a great AV system and comfy seats. The only negative for a conference of that size - and we got close to our 200 person limit - is that the space outside the theatre for breaks and posters does get a bit cramped. In contrast, AGTA15 was held in the Crowne Plaza at Hunter Valley, which is a pretty luxury hotel in the wine region of New South Wales. It took me a while to warm to the main conference setup, with seats around tables in a large, wide room with a screen each side of (and a long way from) the podium. However, the “trade exhibition” space where the breaks, lunch and posters were, was excellent. I am big fan of having the posters up all the time and in the same place as the breaks.

Location. ABACBS was held in the city centre at a research institute. This kept the costs down and made it easy to get to. (We did not organise accommodation.) It also made it easy to escape from, which might have affected evening social numbers. AGTA was out in the sticks, which made it harder to get to and may have put some people off attending - it essentially added another day to the length of the conference. The plus side of this was that everything was on one site and it was not so easy to disappear or go home, and so most people were around most of the time, I think. (Although there was the temptation of wine tasting on the doorstep!)

Goodie bags. I’m pleased to say that both conferences went down the reusable shopping bag route (pick above). Being budget, we got UNSW to sponsor us with some canvas bags for ABACBS. (Complete with the scary statistic that plastic bags take 15-1000 years to break down.) AGTA was (a) a bit more upmarket and (b) in wine country, so they provided wine coolbags!

Trade exhibition. We binfies are a cheap lot - and I don’t mean tacky or miserly, I mean that we don’t need a lot of money to do great things. As a result, it is hard to attract trade sponsors: we just don’t spend (or often even have!) any money! Genomics is clearly a different matter, as you literally cannot do it without lots of expensive kit. The AGTA trade exhibit was therefore pretty big and bustling. Again, putting it with the food and posters made for some great…

…Breaks. Breaks really do make or break a conference. One of the few criticisms of ABACBS2015 was that the breaks were a bit too short - we were somewhat hemmed in for time because we needed to leave people enough time to get to AGTA in the afternoon of the second day. We also tended to over-run a little, because people would be enjoying the breaks and take a bit of time to filter back into the theatre for the talks. My advice for conference planners: (1) make all breaks 5-10 minutes longer than you think they should be (e.g. 40 minutes for coffee and over an hour for lunch); (2) build in some dummy time into the program to soak up delays; (3) Make sure you have a bell or something to signal that the next session is starting! Being a more leisurely multi-day conference, AGTA had nice long breaks, including a session off for posters etc. after lunch. Lots of opportunities for mingling and looking at posters.

Invited speakers. Both conferences had outstanding invited speakers that gave really interesting talks. I know some of the binfie crowd would have enjoyed a bit more about the methodology in ABACBS2015 - and we should perhaps brief our speakers a little better in future - but personally I was blown away by the quality of the science presented. Both fields are rapidly changing as technology opens up opportunities and there is some really cool stuff going on out there! AGTA in particular seemed to have a lot of invited speakers, talking about really cool stuff - possibly part of the reason it's so expensive. (It must be said: the quality of the selected abstracts was good too!)

Gender Equality. Both conferences did pretty well on the gender front, I think. ABACBS2015 in particular nailed it, with a 50% split of invited speakers and over 50% female speakers overall! (The latter wasn’t deliberate as such - there are just loads of good female bioinformaticians in Australia who submitted interesting abstracts!)

Twitter. (And #confBingo.) I have mixed feeling about live tweeting at conferences. This year, I decided to throw myself in a bit more and, on balance, I feel that I got more out of it than I missed. It is true that sometimes I missed something a speaker was saying because of a tangential Twitter conversation (such as #JediKelpie) - but I also learnt stuff and picked up on things that I had missed thanks to the Twitter feed. The #confBingo thread was also quite entertaining a fun, and helped networking and building community spirit, which is what a lot of a good conference is ultimately about.

Coffee. Unfortunately, we were unable to attract the barista sponsor from ABiC14 to sponsor ABACBS2015. AGTA did have a sponsored barista, though. Roche (I think) gave out three tickets with registration for the trade exhibit barista. Not quite as good as unlimited but it hit the spot. My other top conference coffee tip: bring a reusable cup! It' so much easier/safer if you want to take coffee into the talks and generally cuts down on spills - and refills! (Next year, I am hoping for ABACBS Keep Cups in the goodie bags!)

Overall, I had a lot of fun helping with the conference organisation - and would recommend it - and even more fun attending. Australia has some great science going on and a really strong/vibrant bioinformatics community, and I am proud to be part of it.

Sunday, 12 July 2015

Developments in high throughput sequencing (June 2015 Edition)

This is nearly a month old now but Keith Bradnam’s ACGT blog a while back drew my attention to the June 2015 edition of Lex Nederbragt’s Developments in high throughput sequencing in which he plots Gigabases* per run against (log) read length (*the human genome is about 3Gb):

I’m particularly excited by the two technologies on the right of this graph, which represent the latest single molecule “long read” sequencing technologies, both of which we now have access to through the Ramaciotti Centre for Genomics. In fact, we got our first data from the PacBio RS II (right) and it’s looking good! (More on that later.)

Despite being a bioinformatician with a background in genetics, I have been keeping my distance a bit from “next generation sequencing” as the technical challenges of dealing with short read data far eclipse the scientific interest. (For me, that is - the kinds of things that I am most interested in do not suit short read data.) The new long read technologies are a real game changer, and I see a lot more genomics in my (and this blog’s) future.

Friday, 5 December 2014

Edwards Lab: On the hunt for molecular mimicry in viral pathogens

Yesterday’s excellent Australian Pathogen Bioinformatics Symposium 2014 (#APBS2014) reminded me that this post was a bit overdue: originally posted at the UNSW Science Wavelength blog on October 30, 2014. Following APBS2014, we are also looking into how we can identify which other pathogens might be using a similar strategy.

Read more at the Edwards Lab blog…

Wednesday, 10 July 2013

Science blogging is hard!

My last post was number 400, which has caused me have a reflective moment about my blogging. (And the 87 drafts in various states of incompleteness!) One of my New Year’s aspirations was to blog more original science papers. Half a year on, I’ve not really done that well, to be honest. (Nor on my other aspirations, to be honest: Mystic Mog and the Exploding Tortoise has been on hold since Jaunary!)

Part of the reason is the impending move to Australia and all the organisation and house-selling that goes with it. A bigger part of the reason, though, is that (for me, at least) science blogging is hard! I have a lot of respect for those who can do it well, like Jacquelyn Gill and her ecology & climate change blog, The Contemplative Mammoth. (Her most recent post, The many scales of climate change, part 1: Tectonic timescales is well worth a read and I am looking forward to the parts to come.)

I consider science communication to be an important part of my job and I really want to get better at it. Happily, it is not too late. We are about half-way through 2013 and so there are 25(ish) weeks left. The only way to improve is to practice and rather than risk upsetting anyone by getting their science wrong, I have therefore decided to aim to blog one of my own papers each week to see how that goes. It may not happen but by making a public declaration of sorts, I am hoping I will get the extra motivation boost needed to make it happen. (Embracing Markdown should help too!)

Sunday, 16 June 2013

The Ehux genome is here and it looks epic!

In addition to my main research on Short Linear Motifs, I have a number of collaborations with environmental scientists looking at how various organisms, such as corals, might be affected by rising CO2 levels.

One of the more fun collaborations has been with Bethan Jones (now at Oregon State University) and Debora Iglesias-rodriguez (now at UC Santa Barbara), working on the marine phytoplankton Emiliania huxleyi. Along with Paul Skipp at Southampton, I was involved with with proteomic analysis of this small but important organism. This was challenging because, at the time, their was no annotated genome available - hence the need for some bioinformatics help. (The second paper from this came out in April but my planned blog post was postponed after the Boston marathon bombing and is now sitting on my To Do list.)

Happily, the genome is now out! As reported in the KQED blog post, Opening the Gene Box of a Key Ocean Species:

“This week in the journal Nature, a worldwide team of 75 scientists revealed the genetic blueprint of the one-celled alga Emiliania huxleyi, which may be the most important species you’ve never heard of. The genomes of the domestic dog and cat are interesting, but the E. huxleyi genome is a much bigger story. Some day this organism may become another of our partner species, as vital to us as yeast.”

Even more happily, Bethan and Debora are both authors on the paper (as part of the Emiliania huxleyi Annotation Consortium), Pan genome of the phytoplankton Emiliania underpins its global distribution, which reports “sequences from 13 additional isolates” in addition to the genome of the reference strain CCMP1516. Many years in the making, it’s a bit of an epic, and I look forward to learning some more of Ehux’s secrets.

h/t: Bethan. [Picture from KQED] Citation: Read et al. (2013). Nature doi:10.1038/nature12221.

Wednesday, 29 May 2013

The best exam (and coursework) tip ever!

I'm heavily in the middle of exam and coursework marking, hence the distinct paucity of posts, but I felt compelled to revisit a similar post from last year with the number one best exam tip of all time:

Answer the Question!

There is an equivalent for coursework too:

Follow the Instructions!

As examiners, we're not out to trip you up but we can only give credit for relevant material. (More exam tips here.)

PS. If someone knows why all my post titles have gone wonky, I'd love to know! Blogger, what have you done?!

Wednesday, 1 May 2013

QSLiMFinder conference posters now up on F1000Posters

Although I have aspirations, I'm not very good at blogging about my work. I think this is partly because of my British squeamishness at self-promotion and partly because I can't decide where to start.

Having made a few posts about my recent(ish) visit to Cold Spring Harbor (and its geeky architecture/sculpture), I thought I should make at least one post related to the conference itself (the 2013 CSHL "Systems Biology: Networks" meeting. The two posters my lab presented have just gone live on F1000 Posters, which seemed as good an excuse as any:

  • Palopoli N & Edwards RJ. Improved computational prediction of Short Linear Motifs using specific protein-protein interaction data.
  • Edwards RJ & Palopoli N. Computational prediction of short linear motifs mediating host-pathogen protein-protein interactions.
  • I'll try to write a more informative post soon but, in the meantime, you can read the abstracts at my SeqSuite blog (or even click on the links to the posters) if you want to know more!

    Tuesday, 16 April 2013

    Something to cure the Monday Blues

    I'm not sure why I find nature's blues so appealing. I suspect it is because blue creatures are clearly there to stand out - it is unlikely to be camouflage - and therefore usually also somewhat flamboyant. This is combined with the fact that my mild red-green colour deficiency reduces the equivalent wow-factor of reds. Anyway, whatever the reason, here are a couple of great blue animals. The first is Homarus americanus, the American lobster, which is featuring in an undergrad practical I am running next week - or, at least, some DNA sequences are. (Picture from World Register of Marine Species.)

    The second is even more spectacular: a close-up of a blue damselfly, which is part of a set of amazing photos in an item at The Telegraph website, Bug-eyed: macro photographs of insects by Ireneusz Irass Waledzik. Isn't it adorable‽ It's well worth checking out the complete set.

    Friday, 8 June 2012

    Who does Microsoft Academic Research think you are?

    A while ago, I posted some thoughts about Microsoft Academic Research. Today, I was sent a link by a colleague about one of the features on the site - the Co-Author "Academic Degrees of Separation" path. She sent me the path between us - we're five steps apart. The only problem was, it wasn't me! The picture was me and the name was me but all the other details were wrong!

    Here's the entry:

    The worrying thing for me, is that I did not create this entry and yet they have somehow affiliated both my picture and one of my genuine URLs - my Southampton Computational Modelling Group homepage - with a tiny handful of wrong publications and an institution where I have never worked or visited. Perhaps more worrying, I cannot work out how or why. I can only guess that they have some kind of data-mining algorithm that has gone badly wrong. Very badly wrong!

    Here's the real me:

    As you can see, I have a totally different set of metrics, keywords and interests to the spurious me. My colleague is not really to blame, though. Search my name, and up pops the wrong entry complete with my actual photo before the real me:

    The sooner they roll out ORCID, the better! For now, I guess I need to work out how to get the wrong Richard Edwards removed and/or merged with the right one.

    For those interested in the Co-Author path, here it is for the real me to the fictitious me:

    Of course, any one of these links could be through a totally incorrect entry. This whole scenario just reinforces my views of Microsoft Academic Research: they have some nice tools but until they can sort out the underlying data, it remains essentially useless and possibly worse than useless.

    If you have a remotely common name, even if you have no interest in a profile on Microsoft Academic Research, go and look yourself up now! You never know what someone else might be finding out about "you".

    UPDATE. I was able to log on and remove my picture and URL from that entry, although it looks like it will take a week to take effect. Hopefully, it will work...

    Postdoc position available: one week to go before deadline closes

    This is just a quick unashamed plug for the postdoc position currently available in my lab. (Who wouldn't want to work with me?) Follow the link for details. Closing date is June 15th.

    Thursday, 17 May 2012

    The importance of tailoring job applications

    I am recruiting again (a two year bioinformatics postdoc in motif detection) and so my mind has once again turned to the stormy seas of job applications.

    Happily, I have already had a few interested parties. (The ad only went live a couple of days ago.) Among them, however, I already have a couple of the "generic" applications that make my heart sink. "Dear Sir/Professor..." (you're emailing me, you should know my name) "...I would like to apply for a position in [insert unrelated research area or, at best, a list of broad research areas that by chance overlap the project area..." (you are applying for a specific project, you should be interested - or, at least, feign interest - in the specific area) "...please see my attached CV/resume and unrelated publication" (presumably in the hope that I will spot the relevant connections that they have failed to).

    Getting a job is hard because you never know who you are up against. Writing a decent job application is not so hard, though. At least, not if you are actually suitable for the job. There is a wishlist of criteria, so point out how you meet them and how you are really willing/interested/able to learn the skills that you don't yet have. (Like most wishlists, desired criteria are often up for negotiation in reality.) Mention a keyword or two from the project - the name of the project, at least! - to show that you know what job you are applying for. And, if contacting a specific person, show that you know who you are contacting and what they do.

    In an ideal world, you will not have to customise your CV much, if at all, because you are genuinely a good fit for the post. If you are not, though, you will have to do some work. If, for example, your background is in a different field, highlight your relevant experience and compress the pages of irrelevant stuff. If the job really interests you, it is worth the effort. If not, don't bother to waste your time and the poor soul short-listing - you won't get short-listed anyway.

    Even if your CV doesn't need customising - and ideally it won't because you will be applying to a job that suits your experience - you still need to put a little effort into your covering letter/email. At the very least, I feel that I am worth the time to look up my name, research interests and the think of a sentence or two about what interests you about my work. If you don't feel that I am worth the time, am I really going to want to hire you? If there isn't enough information, then ask. Show you care. It's not hard. If you don't give me the impression that you care about my work/project/job, I am not going to care about your application. It really is as simple as that.

    On the other hand, if you show that you (a) know what you are applying for, (b) really want to do it and (c) have the experience to hit some of my wishlist and track record to learn the rest, I will really want to short-list you, even if someone else looks more experienced/qualified on paper. So, if motifs are (or might be) your thing, you know how to program and you're interested in molecular evolution... drop me a line!

    Wednesday, 9 May 2012

    Education is the key to impact

    This month is a busy one and two activities that were going on side-by-side last week were (1) writing an outline grant proposal to further develop SLiMSuite (servers and programs) and SLiMdb, and (2) dig up possible "impact" for the upcoming REF. Part of this has featured digging into Google Analytics data for both the bioware servers and my homepage.

    Sadly, I stupidly neglected to monitor the documentation and download pages for SLiMSuite (now rectified) but I could get visit data from the webservers and the results made fairly happy reading, especially when compared to the citation metrics associated with the relevant papers.

    The bioware.ucd.ie front page currently receives around 350 visits a month from across the world, with 4,174 visits in the period 1 May '11 to 30 Apr '12. Ireland is currently the main user, reflecting the fact that the current host of these resources is University College Dublin, although the balance seems to be shifting towards America now. (Given the amount of science being done there, you would expect the US to be number 1.) Germany and the UK complete the top 4 (again not surprising in terms of both science and presence of participating/collaborating labs) but the usual suspects (India, Canada, Israel, China) are all in there too, which is good to see. The servers themselves receive around 500-1000 page views a month, with the top five servers visited in the period 1/5/11-30/4/12 being: 1. SLiMFinder (2,651 views); 2. SLiMPred (1,812 views); 3. SLiMSearch 2.0 (1,721 views); 4. SLiMSearch 1.0 (805 views); 5. CompariMotif (653 views).

    This is all well and good - these are published servers - and hopefully will continue to increase over time, particularly if we can get more funding for development. The thing that surprised me, though, was when I looked at the usage stats for the UPGMA walkthrough I made for a Year 2 practical that I run with a colleague. This website got 5,721 visits in the same period (1 May '11 to 30 Apr '12) - 37% more hits than the bioware front page. This number is also on the rise with nearly 1000 hits last month, due in part to the fact that it now appears on the UPGMA wikipedia page.



    This perhaps should not be surprising but when you consider that UPGMA is a largely obsolete method used predominantly for teaching (and quick and dirty clustering) and I knocked up the website in an evening or two, while SLiMFinder is a cutting-edge bioinformatics tool that represents years of hard work, it is still a little depressing. I guess I should make more educational web pages...

    Sunday, 29 April 2012

    SeqSuite: another blog is born

    Although this is my blog, I feel a bit funny about explicitly blogging about work stuff in a blatant "look at what I've done, please read/use/cite it!" fashion. I have therefore created a new blog explicitly for that side of things. Actually, it has existed for a while but not really had much content until today's post about SLiMMaker (my second solo excursion into python CGI). The blog is currently titled SeqSuite: open-source bioinformatics in Python and the central theme is the application and development of the various tools that I have cobbled together over the years, although I hope to diversify into other related stuff should time allow.

    The main focus is actually SLiMSuite, which is the collection of short linear motif (SLiM) tools that I have been involved in developing. (The term "SLiM" will probably be my longest lasting contribution to science. It's weird seeing it start to appear in textbooks! (Although I must point out that I did not invent the concept. I'll save SLiMs for another post, I think.)) SLiMSuite is both the main pillar of my research but also, I think, the more original and unique of my software. (Some of the other stuff I have made because I have been too lazy to hunt down something that does what I want.)

    Despite the ascendancy of SLiMSuite, I've stuck with "SeqSuite" as the name, though, as (a) other "Slim Suites" seem to exist and I want to head off any legal objections in advance, and (b) I have some other tools that are nothing to do with SLiMs and, along with SLiMSuite, form the larger SeqSuite package.

    I'm still not entirely sure how the two blogs will work - finding the time to write one blog is hard enough - but I think I will continue to post the more "human" aspect of work-related matters here, and the more technical aspects on the SeqSuite blog. (Hopefully, I can convince some of my collaborators to contribute there too.) I think, like a lot of academics, I realise the importance of trying to communicate what we do, but haven't quite worked out yet the best way to go about it.

    Monday, 16 April 2012

    Short-cited VII: does Open Access publishing encourage bad science?

    The argument about Open Access publishing is one that is going to run for some time, I am sure. (See yesterday's piece in The Economist, for example. One spectre that raise its ugly head from time to time, though, is the notion that Open Access is bad because it encourages bad science. In essence, the argument goes: Open Access is cheap for the publishers, therefore they are willing to publish anything. In support of this, I have seen people cite the high acceptance rates at PLoS and BioMedCentral of (apparently) 50-70% as an indicator that they will publish pretty much anything. I've even seen this used in favour of removing pre-publication peer-review altogether in favour of post-publication peer-review, as in the case of WebmedCentral - the implication being that, in the Open Access world, pre-publication peer-review is useless.

    I think this is all rather unfair on Open Acess journals, especially the likes of PLoS and BMC. The latter have just done away with the need for the science to be deemed "interesting" as long as the science is sound. (And this for only some of their titles.) This is a far cry from publishing anything and peer-review being worthless.

    The peer-review process involves highlighting things that need to be changed to meet the "scientific soundness" criterion. Most authors then go ahead and make the required changes, usually improving the paper as a result. I would argue that this is the single biggest utility of peer-review; as an author, I want to go through this process myself, even if it is sometimes irritating. The fact that PLoS and BMC publish 50-70% of papers just indicates that scientists are capable of producing scientifically sound work (following revisions) 50-70% of the time. The higher rejection rate at Science and Nature is more to do with what science is perceived as being of general interest, rather than the soundness or quality of the work. (I also suspect the acceptance rate for the flagship PLoS and BMC journals that do stipulate a need for impact is considerably lower than across the whole series.)

    The peer-review process and quality of reviewers is not always perfect but it is still better than none at all. I am very reluctant to ever publish in a journal that does not have pre-publication peer review and it is not just about the journal's "Impact Factor"; the quality of the papers is going to be lower without the opportunity for revisions in the face of peer-review and at least the threat of being in that 30-50% that aren't deemed scientifically sound enough to publish.

    The biggest problem in the modern scientific literature is the pressure on scientists to publish too soon and too often, not the presence of journals that are willing to publish boring science. Until we get rewarded for the quality of our science and not the quantity or impact of our papers (first is not always best), I cannot see this changing, sadly.

    Wednesday, 11 April 2012

    Short-cited V: where to publish?

    The second side of bibliometrics is the one I have not considered so much: how to use citation data to help you decide where to publish. This was more important, I suspect, when Impact Factor was the thing that everyone cared about. Happily this is no longer the case - at least, not in the UK - but it is still undeniably important and used in a general sense to establish whether you, as an author, are routinely publishing in top ranking journals.

    Call me old-fashioned but I prefer to choose my journals on the basis of their research scope and target audience. Second, the format of the articles can be important - sometimes, a long article is easier to write and might have higher impact than the short, concise papers that some of the highest-impact journals favour. Nevertheless, when the scope and style of the journals appears similar, citation metrics can be an indicator of readership and influence; I know my own opinion of journals is heavily influenced by individual papers I have read from those journals, as well as being biased by my genetics background, and so an impartial view can be informative.

    What I did not realise is the range of citation metrics and resources available for comparing journals. The main two sites (as listed on the bibliometrics course I attended) seem to be  Scimago and JCR (Web of Knowledge), which use Scopus and ISI citation data, respectively. To compare journals, however, you need to pick a category and here is where I encountered my first problem: in Scimago, I could not find any bioinformatics or computational biology category. JCR does, at least, have a mathematical and computational biology category but the overlap with my kind of bioinformatics (i.e. molecular evolution) is not that great and so the problem still remains: if your field does not nicely align with one of their selected categories, it is nigh-on impossible to get a general overview or ranking of your options (or papers).

    You can, of course, still do pairwise comparisons of specific journals. Then the question becomes, which metric?  The traditional metric is "Impact Factor", which is the mean number of times a paper is cited in the two year period following its publication. As explored in my last post on bibliometrics, the suitability of this period is obviously very field-dependent and also genre-dependent. Methods articles, for example, are likely to have a greater lag before people start publishing papers using them than discovery articles. More worrying, though, is the way these metrics can be blatantly abused. Self-citation (such as editorials citing papers in that issue) has been used for years to boost Impact Factor a little but it seems that this at least is something that JCR etc. monitor and respond to. More sinister is something that I only discovered today: the citation cartel. I recommend reading the linked Scholarly Kitchen article but, in a nutshell, this is when a group of editors boost their journal's Impact Factor by writing a review in another journal that almost exclusively cites papers from their own journal that were published within the last two years! How they can write a coherent article by doing this without plaigiarising heavily, I am not sure, as they must have to ignore masses of relevant literature, but the examples in the article are pretty horrific.

    If you are suspicious of the standard two-year Impact Factor due to these abuses, you can always opt for a 5 year Impact Factor, which is probably a little less prone to abuse and, arguably, a better indicator of impact. (I know that I still make use of a lot of papers dating back to 2007!) If not sure which duration is more appropriate, you can check the cited half-life (median age of articles cited by the journal) and citing half-life (median age of articles from that journal when cited). There are also slightly more esoteric metrics out there, including the "Article Influence Score", which is Impact Factor weighted (how?!) by the journal prestige/ranking of citing articles, and the rather mysterious "Eigenfactor" score, which was described to us as the amount of time spent in a given journal when taking a random walk through article space. Hmmm. At least these ones are harder to fake, I guess.

    I think it's quite interesting to look at these things, partly because I am just a geek who likes data and graphs and things, but I think the general conclusion of short-cited IV still holds: don't use citation metrics alone to make decisions. Any decisions. Ever!

    Friday, 30 March 2012

    Short-cited IV: bibliometric comparisons

    As part of my on-going attempt to get to grips with citation data, I attended a short lunchtime session on bibliometrics at the library today. Unfortunately, I did not really learn much more than I had from exploring my own ResearcherID and Google Scholar pages, which I think reflects the reality of the murkiness of bibliometrics more than my own thoroughness.

    I had hoped to learn a bit more about Scopus, as this is apparently the bibliometrics provider that will be used as part of REF but, due to its expense, we do not have access at Southampton. (This begs the question as to how useful Scopus really is - if the average person cannot access it, what's the point?)

    The introduction reiterated some key points that are worth repeating. The first is that no one source is (currently) ideal and so it can be sensible to use more than one. Moreover, the biases in Web of Science and Google Scholar are very discipline-specific. The Computer Science REF panel, for example, apparently uses Google Scholar instead of Scopus, as this better reflects their important citations. (Conference proceedings are much more important in Computer Science than biology, which is actually a bit of a barrier for computational biology bringing together computer scientists and bioinformaticians.) 

    Beyond the citation data sources, there are also differences in citation profiles for different disciplines, depending on how active a field is (crudely, more researchers = more citations) and how rapidly ideas are taken up and/or can be applied (Adler R et al (2008) Citation Statistics).

    This is clearly true for both the age of papers cited (the citation curve, top) and the number of publications cited. There is an issue within an issue here, as exemplified by the bottom chart: how do you define a field? When does biological science become life science? This is a particular issue for me, as bioinformatics often falls down the cracks: neither computer science nor experimental biology are appropriate comparators.

    The upshot of this is that you should only ever compare like-with-like, i.e. the same metric using the same dataset and the same subject area. With this in mind, it does not really matter too much what you use; although the absolute numbers differ, the relative rankings of authors has been shown to be fairly consistent between resources. (I need to dig out the source for this!) This is, of course, only true if it is the citation stats that you are after. Personally, I link to my citation pages predominantly as a way to look at the citations, so I want completeness. I'm still not convinced that bibliometric scores are really useful for any kind of comparison beyond a bit of ego massaging, which brings me onto the final point of the introduction that is worth repeating: don't use metrics alone to make decisions. Any decisions. Ever!

    The session did not go into the different citation metrics themselves but did highlight a few resources for calculating different statistics, although only using Google Scholar (with all its pitfalls) as a source. In addition to Publish or Perish, which I mentioned before, they drew attention to QuadSearch, the Mozilla Firefox Google Scholar add-in, and Scholarometer. As with the fancy Microsoft Academic Research stats and comparisons, however, "garbage in, garbage out". In particular, I am not sure that any of these tools give you the option to edit your Google Scholar data by adding or subtracting citations. (The cited work included, yes, but not the citations.) Publish or Perish certainly doesn't seem to and this is the one that the presenter thought was the best.

    My take-home message: if there is a genuine desire by governments, 
    league tables and employers to make use of citation data, they need to invest in making robust and accurate tools for collecting these data and policing them. (If I was disingenuous, I could easily "claim" some other R Edwards publications in my ResearcherID, Google or Microsoft profile and I suspect that not many people would notice.) Until that time, I am not sure that there is much we can do apart from learn how to best present our own citations on the one hand, while resisting their use on the other. (Never use metrics alone to make decisions!)

    Wednesday, 21 March 2012

    Short-cited III: affirmative action

    In a previous post about citation metrics, I criticised Web of Science for under-estimating citations due to (a) only including stuff that's indexed in ISI, and (b) mistakes in their database that cause citations to be lost.

    Although I stand by those criticisms - and I am not the first to make them - it is worth pointing out the something can be done about the second one, at least. In another previous post, I mentioned a paper about Bobtail squid that had cited one of my papers on E. hux proteomics. This was not listed as a citation in ISI due to an error in the reference list for that paper. Fortunately, Thomson Reuters Technical Support allows you to report errors and, I can happily report in this case, they will fix them.

    If your Google Scholar count for given paper is higher than for Web of Science, it might therefore be having a quick look at the difference and checking the reference list of those missing papers.

    Monday, 12 March 2012

    Short-cited II: Microsoft Academic Research

    Since my last post, my Microsoft Academic Research page has been updated in light of my editing. It seems that my initial views of over-inflation were wrong, though. Whereas Google has my h-index at +2 versus Web of Science, Microsoft has me at -2. This seems to be, at least in part, due to a lack of recent publications. My 2012 papers are missing, for example, which makes me wonder how many other papers (and therefore citations) are missing.

    The site does seem to have some nice features but, because the underlying data seems to be unreliable, I'm not convinced that they're (currently) that useful. The co-authors listing, for example, has one of my regular collaborators Norman Davey twice, presumably because he moved institutions. Unfortunately, the Citation Graph does not work on an iPad, so I'm not sure if this makes up for any of the mess.

    Similarly, the organisation comparison function seems interesting but does it really mean anything if you cannot trust it? I guess that kind of sums up bibliometrics all round, though.

    Monday, 5 March 2012

    A short-cited look at bibliometrics with Web of Knowledge

    A few things have got me thinking more about citations recently. First, the dreaded "Research Excellence Framework" is looming, for which each academic in the UK has to submit their four "best" papers. "Best", of course, is a rather subjective issue. Although journal Impact Factors and citation data are not meant to be taken into account for REF, the number of times that a paper has been cited is one indicator that can help determine how your papers are received. The second thing was that I was recently looking through a bunch of CVs as part of a recruitment process and the importance of maximising your perceived impact was clear.

    I've had one eye on these issues for a while, so I have had both my ResearcherID and Google Scholar publication metrics linked from my website (and this blog) but never really thought too hard about either. My assumption was that the ResearcherID metrics, provided through Thomson Reuters Web of Knowledge, would be the better metric provider, as it's an "official" supplier of citations and is manually curated. Google Scholar, on the other hand, is more automated and has a tendency to over-inflate citations by including stuff that might be weeded out by more careful citation monitors.

    Looking into things a bit more, though, the ResearcherID metrics do not appear to be as trustworthy as I had assumed. The problem with Web of Knowledge is that (a) they only include stuff that's indexed in ISI, and (b) for a manually curated citation index there seem to be a lot of mistakes that cause citations to be lost. As a result, although libraries seem to prefer Web of Knowledge, the citation metric calculator "Publish or Perish" uses Google Scholar. The publications of mine that I've looked at certainly back this up. One, for example, has ZERO citations on Web of Knowledge but Google Scholar lists three perfectly acceptable (in my mind) peer-reviewed citations.

    It's not all positive for Google Scholar, though, as the criticisms levelled at it are also valid. Although Google are the kings of automated searches and returning relevant data, they are not flawless and I have noticed the odd duplication here and there. I have not checked yet (as the numbers I've checked are bigger) but I would not be surprised if there were also some citations missing; I know this has been an issue for some colleagues. The other problem is that, unlike ISI, there is little or no filtering of the types of citations returned. Going to the other end of the spectrum, and looking at my most highly cited paper, Google Scholar had added 15 citations, including the PDF manual of one my software packages. (There's a lesson in citation-inflation there, I think!)

    This issue of over- and under-reporting of citations is not new and has been reported but I had never realised the extent of the under-reporting before. Furthermore, a couple of the other extras returned by Google are less cut-and-dried with respect to their "inflation" status. One was a doctoral thesis, which is a genuine peer-reviewed publication and I would consider a real citation. Indeed, including theses could be one of the biggest assets of Google Scholar, for it is normally hard (or impossible) to find out about theses that cite your work unless it is followed up by a paper. As a result, not only is the citation a useful discovery but potentially the thesis itself. Another two were foreign language (i.e. not English) publications, which again might well be perfectly valid. (Not speaking Polish, I cannot tell in this instance.)

    Then, there is Microsoft Academic Research, although this seems to inflate things even more as it includes extra publications belonging to other people - at the moment, anyway. I've created a LiveID account and done a bit of cleaning up of my publication list, so it will be interesting to see what it says after that. (I now have a couple of publication missing but I am not sure which ones.)

    So, which to use?! Currently, my feeling is that none of them are perfect. For me, ResearcherID is a definite underestimate but, at the same time, the extra 96(!) citations from Google Scholar are not all valid. It makes a difference - my h-index goes up by 2 with Google versus ISI - but it would be just as bad to be perceived as inflating my citation metrics as it would be to under-sell myself. The only real solution at the moment is to provide both metrics (and maybe Microsoft too, if that is different again) and keep an eye for one that allows editing of both publications and citations. (I don't think any of them currently have this function.) In the long run, though, I have a horrible feeling that I'll have to compile the genuine citations from the different sources myself. If nothing else, it will settle the question of which is best - for me, at least.