Abstract
Two critical questions one must answer as one applies the results of a clinical trial to clinical practice are: (1) Regardless of whether the trial result is likely to be replicated or reproduced in a second large-scale trial, are the results likely to be reproduced in one's practice? (2) Regardless of whether the experimental treatment was better than the alternative on average for a population of patients, are the results clinically important for a given patient in one's practice? To determine if a study result is likely to be reproduced in one's clinical practice, it may be helpful to answer 5 questions: (1) Have steps been taken to minimize bias? (2) Is the result likely due to the treatment? (3) Is the result unlikely due to chance? (4) Is the study population representative of one's patients? (5) Is the totality of evidence consistent? If the answer to all 5 questions is “yes,” then we posit that the trial result is likely to be reproduced in one's practice. If not, the likelihood of reproducibility is low. If the answer is yes to all questions except the last, then reproducibility in one's practice is not clear and depends on the strength of the prior versus the current evidence. If the prior evidence is strong, such as multiple pivotal randomized clinical trials, and if the current trial result is not consistent with the previous studies, then the current result may not be reproduced in one's practice. To determine if a study result is clinically important, a 3-step approach is suggested. Step 1. Decide, a priori, what a clinically meaningful difference between 2 treatments would be. This choice defines regions of beneficial, harmful, and trivial outcomes. Step 2. Identify the confidence intervals (CIs). Determine whether the 95% CI mostly includes the range of clinically beneficial outcomes and lies outside the range of clinically harmful outcomes. If these conditions are met, the result is probably clinically important, but the result may or may not be statistically significant. Put the CIs and the regions of benefit/harm together to make a decision about clinically important effects. Step 3. Assess the proportion of eyes with clinically meaningful changes in vision. The proportion of “responders” among patients receiving a given treatment reflects the likelihood of one's patient having a clinically meaningful response to the treatment. In summary, not all statistically significant results are reproduced, even those of carefully designed clinical trials. Determining if a study result is likely to be reproduced in one's practice is even more problematic. The 5-question test may help in this regard. The 5-question test attempts to assess whether steps have been taken to: minimize bias; avoid confounding; ensure adequate statistical power to support precision in the estimates of population parameters; insure external validity of the trial result; and determine whether there is a convergence of evidence consistent with the trial's major findings. To determine if a statistically significant result is likely to be clinically important, a 3-step approach may be useful, focusing on CIs and the proportion of eyes with clinically meaningful changes in vision. Application of clinical trial results to clinical practice requires critical analysis of the extant literature and good clinical judgment.