How to read a paper on a randomised trial

How to read a paper on a randomised trial.

Since I started this blog several readers have contacted me and asked if I could write a post on what I look for when I read a paper on a randomised trial.  As my next few posts are on clinical trials I thought that it would be a good time to do a post on what I look for.

I hope that I have made this fairly straightforward and I would like to point out that none of the points are my own, I have simply distilled several useful sources.

The introduction

I know it is tempting to skip past the introduction to a paper, but I always carefully read this so that I am clear on what the investigators are trying to discover. This may seem obvious but it is surprising how often the title or the abstract of the paper do not have  a relationship to the study. The introduction also helps me update my knowledge on the subject of the study. At the end of the introduction I look very carefully at the hypothesis that they are trying to test. I then generally write this down and refer back to it as I read the paper.

Method

This is a most important section and it should be written carefully and  clearly. I look for the following main points;

The patients and setting of the study

Is the sample of participants drawn from a population that is relevant to my clinical practice?  This is important because if the findings of the study are going to influence practice, then the patients and treatment setting should be similar to our practice setting. The study should have generalisability. If we consider various orthodontic studies; the levels of generalisability from high to low, to clinical practice, could be;

1. 10 to 16-year-olds in a “High Street” practice/office
2. 10 to 16-year-olds being treated in a University/Hospital setting
3. adults being treated in a “High Street” practice/office
4. adults being treated in a University setting by residents.

In short, you need to identify whether the group of patients being treated are relevant to your clinical setting. If you feel that they are so far from the sphere in which you work, then you could argue that the findings of the study are not relevant to you.

What was the control group?

It is very important that the control group is matched to the intervention group and this can be checked by looking at the baseline data, which should be included in a table.  It is also important that if an intervention involves a method of pain reduction that it is compared against either another pain control intervention or a placebo.  It should not be compared against nothing. This is because it is unlikely that an operator would recommend no intervention to their patients. This will also take into account any placebo effect.

Did the authors  carry out a sample size calculation?

This is very critical because this assures us that the study had sufficient power to detect a difference between the interventions. This should be based on previous literature and this reference stated.

Was the intervention or treatment clearly described?

The authors should clearly state what their treatment involves and how it could potentially  ‘ work’. I use this information to evaluate the studies relevance to  the treatment that I am prepared to carry out.

Issues of randomisation

The main reason to carry out a clinical trial is to find out  about treatment using a form of research that minimises bias. As a result, the following details of randomisation are an essential requirement of reporting a trial. The most important concept is that the trial needs to have processes in place so that any bias caused by the preferences of the operators, or researchers, for certain treatment is minimised. As a result, the authors need to outline the following in some detail;

How was the randomisation done?

For example, was it done by computer-generated randomisation remotely from the site of the study (low risk of bias) to drawing lots for treatment out of a hat on the clinic (high risk of bias)

How was concealment achieved?

Concealment ensures that, when a person is being enrolled in a study, the operator  has no idea what the treatment allocation for the patient is going to be. This is important because if they are aware of this treatment allocation, they may not enrol a patient into the study because of their potential bias about a particular type of treatment. The authors should also state who generated the allocation sequence and who enrolled patients.

The ideal way to ensure adequate randomisation, allocation and concealment is to generate a random allocation of treatment using a computer which is held on a site away from the clinic. The person who is enrolling the patient into the trial then contacts the centre and provides details of the patient. Once this data is recorded the operator is then given details of the treatment allocation. There are many trials units that will carry this out for people running studies.

Blinding

Blinding means that the participants, the operators and those recording the data do not know the treatment that the participant had received. This is very important because it ensures that any personal bias in providing the treatment, recording and interpreting data is minimised. Ideally, a study should be double-blind, but this is not possible from any orthodontic studies because it is impossible to conceal the treatment allocation from the patient and the operator. However, a realistic degree of blinding can be achieved by keeping the treatment allocation from the person who is recording the data for example cephalometric analysis, study cast recording etc.

Data

The authors should provide a flow diagram of the flow of participants through the study and this shows important features such as recruitment issues, total dropouts and any differences in dropouts between the intervention groups.

The data presentation should be clear and  the means and 95% confidence intervals should be presented. This allows a reader to interpret any uncertainty in the data. I have covered this in this post.

Finally, when I read the discussion I look closely to see if the authors have justified their results, discussed the generalisability of their conclusions to clinical practice and I look very closely to see if the conclusions are supported by the data!  This is not always the case…

CONSORT guidelines

The evaluation of a trial is made much easier if the journal adopts a set of reporting guidelines called CONSORT. These can be found here. If these are followed by both the authors of the paper and the editor of the journal, then all of my points are covered. The following orthodontic journals have endorsed the consort guidelines, American Journal of Orthodontics, Journal of Orthodontics, and the European Journal of Orthodontics. I was surprised that the Angle Orthodontist was not included on the website and I wonder if this is an omission?

I hope that these pointers are useful. My next three posts will be based on some newly of  published trials, so remember this posting if you read them!