Although randomized trials have been conducted for several decades now, some aspects
of their analysis remain contentious. Two such issues are what to do about trial participants
who do not adhere to the protocol (for example, if they do not receive the intended
treatment) and how to deal with those for whom outcome assessments are missing (for
example, because they are lost to follow-up). Both of these issues are relevant to
the adoption of so-called “intention to treat” (ITT) analysis – a topic that, not
surprisingly, also causes debate.
ITT analysis is widely recommended as the preferred approach to analyzing the outcomes
of randomized trials.1,2 In an ITT analysis, all randomized patients are included
in the analysis in their assigned groups regardless of all considerations, including
whether they in fact received the designated intervention. ITT analysis should therefore
compare outcomes in groups that correspond exactly to the randomization scheme. Any
deviation from that principle may introduce bias.
An immediate problem is that some data are missing from almost all randomized trials.3
Clearly, just a few missing outcomes will not be a concern, but one review found that,
in about half of randomized controlled trials (RCTs), outcomes are missing for more
than 10% of participants.4 A major concern is that being lost to follow-up could be
related to a patient’s response to the treatment; indeed, we should assume that this
will be so. That concern can be compounded if the reasons for, or frequency of, dropout
differs between the treatment groups.
No analysis option is ideal here; there is, in effect, a choice between omitting participants
without final outcome data or estimating (imputing) the missing outcome data. What
should researchers do? A “complete case” (or “available case”) analysis simply omits
those for whom data are incomplete. This commonly used approach loses power, and bias
may well be introduced, given that the incompleteness of data will not be random.
Further, excluding some patients is not compatible with the ITT principle. Imputation
of the missing data allows the analysis to conform to ITT analysis but requires strong
assumptions that may be hard to justify.5,6 However, some concerns about “making up
the data” are misplaced.7
Methods for the imputation of missing values have been the topic of much methodological
and empirical research in recent years. The generally preferred imputation methods
are quite complex,5-8 and some simple approaches that have been around a long time
are much more popular. One of the simplest and most commonly used of these in the
analysis of continuous outcomes is “last observation carried forward” (LOCF) analysis,
in which missing final values of the outcome variable are replaced by the last known
value before the participant was lost to follow-up. LOCF analysis appeals through
its simplicity and ease of application, but there are strong grounds for not using
it. Specifically, the method may introduce bias in the results, and this bias can,
according to circumstance, be in either direction.9 Also, because in LOCF analysis
no allowance is made for the uncertainty of imputation, the resulting confidence intervals
are too narrow.10
Even if missing outcomes are random across trial participants, LOCF analysis assumes
that the missing final values would be the same as the last recorded values. That
assumption is often implausible (even as an average), because dropping out is likely
to be associated with response to treatment; obvious examples are failure to respond
to treatment and adverse effects. In practice, missing data are very likely to be
related to response to treatment and prognosis.
Molnar and colleagues have discussed these issues in the specific context of trials
of dementia therapies.9,11 They found that 34 of 57 RCTs used LOCF as the only form
of ITT analysis. Not surprisingly, that is a much higher proportion than the 19% observed
in a review of trials across various medical specialties published in 4 general medical
journals.4
As Molnar and colleagues note,11 their study cannot quantify the magnitude of the
effect of the use of LOCF analysis on trial results, but it does highlight the high
prevalence of conditions for which this method of analysis could promote bias in favour
of more toxic therapies and against less toxic alternatives.
Their study focused on a limited number of trials concerning a single medical condition,
but many of the issues it raises apply to all trials. It is known that most trials
include some patients whose outcomes are ultimately not known, and for which an ITT
analysis is not possible without some type of imputation (although authors commonly
mislabel available case analyses as ITT).12,13
Whichever imputation approach is used, it is desirable to report analyses with and
without imputation. Also, it may be valuable to explore different imputation approaches.
As Molnar and colleagues suggest, “the onus is on the investigators who publish these
trials to disprove the possibility that these analyses have introduced bias by performing
ITT sensitivity analyses … . This is particularly true for those studies demonstrating
higher dropout rates in treatment groups.”11
A further important issue is that this information should be included in the report
of a trial. The CONSORT (Consolidated Standards for Reporting of Trials) Statement
recommends that authors specify the methods used for all statistical analyses reported.14
The statement’s accompanying explanatory paper included various comments on ITT analysis,
but there was no specific mention of imputation.15 Noting that omission, Shapiro wrote:
“A variety of options are available to handle missing data from participants who drop
out of the trial and it is important for readers to know what strategy the investigators
adopted. Without such information, the fact that an intent-to-treat analysis was carried
out is only partially informative.”16 That omission will be remedied in the forthcoming
2009 update of the CONSORT Statement.
Transparency of reporting facilitates reliable appraisal of the quality and relevance
of health research.17 Successful transparency may be judged by whether others are
able to reproduce all the methods used. The International Committee of Medical Journal
Editors makes the sensible recommendation to “[d]escribe statistical methods with
enough detail to enable a knowledgeable reader with access to the original data to
verify the reported results” (http://www.icmje.org/). That is very sound advice for
all research articles, not just RCTs; indeed, I see no reason why it should not apply
to all study methods.
Molnar and colleagues have gone further than Shapiro by suggesting that the CONSORT
group should give guidance on methods for analysing trials when data are missing.11
However, because the purpose of the CONSORT group is to give guidance on the reporting
of what was done, and not to advise on what is good or bad methodology, such guidance
will not come from that source. But, given that so many others have warned about the
dangers of LOCF analysis, it seems clear that its use as the sole form of analysis
should be discontinued.8,9,16,18,19
Finally, as Liu and colleagues observed, it is important in the design and conduct
of studies to try to prevent losses to follow-up and to try to minimize the bias caused
by the inevitable missing data.20 To that end, they provide helpful guidance for both
the conduct and reporting of RCTs where missing outcomes are likely:
discuss at the planning stage methods and procedures that maximize the chance of retaining
patients (e.g., short course of rescue medicine),
continue to collect data post-withdrawal to preserve the ITT population,
document the reasons for missing data,
anticipate and investigate the types of missing data,
pre-specify primary as well as sensitivity statistical analyses,
fully report the extent and pattern of missing data,
support conclusions based on results from the planned analyses with proper sensitivity
assessment.20
Adherence to this advice will lead to better design, analysis, reporting and interpretation
of future trials in all medical specialties.