SHORT STEPS ON HOW TO READ A PAPER. PART 7: RETROSPECTIVE METHODS

This next short step is on interpreting the methods section of a paper. We need to understand the method because this is a  crucial part of a paper.  Yet, we tend to rush this because we want to see what the authors found.

I am going to do the methods for retrospective studies, and Padhraig will look at randomised controlled trials.

The methods section.

The best way to look at the method is to think of it as a recipe.  It needs to be so clear that it can be repeated and followed by other investigators.  If we use a  recipe is not straightforward to follow, there is more scope for our meal not turning out well.  This is the same for a method of a scientific study.

As is the case with clinical trials, there are accepted guidelines illustrating how best to report these studies. You can find these here: https://www.equator-network.org/

The retrospective study

I will start with a caveat. We recognise that retrospective studies are not as high a level of evidence as trials or prospective cohorts. Nevertheless, we should not totally ignore them. This is because an excellent retrospective study may provide us with some information. Furthermore, investigators may use them for question generation and planning other studies. Bearing this in mind, these are the points that I look for in a retrospective study.

• Importantly, we need to identify selection bias. This is because the investigators have got the sample of records from a store (real or virtual).  As a result, we must consider why the records were compiled. In effect, we evaluate if there is any bias. For example,  did the investigators collect the records because they were nicely finished treatments.
• Look to see if the authors state that the cases were consecutively started or if the inclusion criteria state that they ensured that all the possible records were collected.
• If the authors state that patient sample was restricted to patients whose treatment was complete and all records were available, this generally means that the study sample is biased. I usually stop reading a retrospective study if this is the case.
• Also, look for statements that suggest some selection of records took place. The best example of this I have found was a recent study on the Carriere appliance and treatment duration. There was a clear bias in this study because the authors rejected patients whose treatment had taken too long!
• Find the inclusion and exclusion criteria for the sample. You need to assess if this is clear, appropriate, representative and repeatable.
• It is also essential to check to see if the patients are similar to those in your practice. This means that the sample has generality.

What’s next?

These are the most critical points in looking at a retrospective paper. We will look at methods for trials next week. Then we will describe the choice of outcome measures.