Suggested Reading List: Methods of Comparative Effectiveness Research

The following is a list of recommended articles on methods that are used to conduct comparative effectiveness research. Papers are identified by the type of method (clinical trials; observational studies; or syntheses). A list of key papers on policy issues related to comparative effectiveness research has also been identified.

This list of suggested reading will grow as the development of new methods in comparative effectiveness research expands. Suggestions for other articles or texts are welcome and may be sent to [email protected] . Suggested articles will be reviewed and considered for inclusion in this list by the AcademyHealth Methods Council.

Articles on the field of HSR are available in the 'About' section. Please also note that in some cases, readings are listed in multiple sections. These cases are denoted by [Repeat] at the beginning of each citation.

* Overview of Comparative Effectiveness Methods

Strom, B.L. “Methodologic Challenges to Studying Patient Safety and Comparative Effectiveness,” Medical Care, Vol. 45, No. 2, 2007, pp. S13–S15. Also see [Abstract].

I. Clinical Trials (Head to Head; Cluster Randomized; Adaptive Designs; Practical/Pragmatic Trials)

Head to Head Trials

[suggestions welcome]

Cluster Randomized Trials

Mazor, K.M. et al. “Cluster Randomized Trials: Opportunities and Barriers Identified by Leaders of Eight Health Plans,” Medical Care, Vol. 25, No. 2, 2007, pp. S29–S37. Also see [Abstract].

Glynn, R.J. et al. “Design of Cluster-Randomized Trials of Quality Improvement Interventions Aimed at Medical Care Providers,” Medical Care, Vol. 45, No. 2, 2007, pp. S38–S43. Also see [Abstract].

Adaptive Designs

Berry, D. “Bayesian Clinical Trials,” Nature Reviews Drug Discovery, Vol. 5, 2006, pp.27-36. Also see [Abstract].

Practical/Pragmatic Trials

Higgins, JPT, et. al. "The Cochrane Collaboration’s Tool for Assessing Risk of Bias in Randomised Trials," BMJ, October 18, 2011. [Free Full Text]

Luce, B. et al. “Can Managed Care Organizations Partner with Manufacturers for Comparative Effectiveness Research?” American Journal of Managed Care, Vol. 14, No. 3, 2008, pp. 149-156. Also see [Free Full Text].

Maclure, M. et al. “Designed Delays Versus Rigorous Pragmatic Trials: Lower Carat Gold Standards Can Produce Relevant Drug Evaluations,” Medical Care, Vol. 45, No. 2, 2007, pp. S44–S49. Also see [Abstract].

Schwartz, D. and J. Lellouch. “Explanatory and Pragmatic Attitudes in Trials,” Journal of Chronic Diseases, 1967.

Sackett, D. section on Pragmatic Trials in Haynes R.B., Sackett DL, Guyatt GH, Tugwell P. Clinical Epidemiology: How to Do Clinical Practice Research, 3rd edition. 2005. Philadelphia: Lippincott, Williams and Wilkins. Also see [Book Information].

Schneeweiss S, Gagne JJ et. al. "Assessing the Comparative Effectiveness of Newly Marketed Medications: Methodological Challenges and Implications for Drug Development," Clinical Pharmacology and Therapeutics, November 2, 2011. [Free Full Text]

Tunis, S. et al. “Practical Clinical Trials: Increasing the Value of Clinical Research for Decision Making in Clinical and Health Policy,” Journal of the American Medical Association, Vol. 290, No. 12, September 24, 2003, pp. 1624-32. Also see [Abstract].

II. Observational Studies (Natural Experiments; Registries; Database Studies)

Natural Experiments

[REPEAT] Maclure, M. et al. “Designed Delays Versus Rigorous Pragmatic Trials: Lower Carat Gold Standards Can Produce Relevant Drug Evaluations,” Medical Care, Vol. 45, No. 2, 2007, pp. S44–S49. Also see [Abstract].

Registries

Gliklich RE, Dreyer NA, eds. Registries for Evaluating Patient Outcomes: A User’s Guide. (Prepared by Outcome DEcIDE Center [Outcome Sciences, Inc. dba Outcome] under Contract No. HHSA29020050035ITO1.) AHRQ Publication No. 07-EHC001-1. Rockville, MD: Agency for Healthcare Research and Quality. April 2007. Also see [Full Free PDF].

Lieu, T.A. et al, for the Vaccine Safety Datalink Rapid Cycle Analysis Team. “Real-Time Vaccine Safety Surveillance for the Early Detection of Adverse Events,” Medical Care, Vol. 45, No. 2, 2007, pp. S89–S95. Also see [Abstract].

Jhung, M.A. et al. “Evaluation and Overview of the National Electronic Injury Surveillance System- Cooperative Adverse Drug Event Surveillance Project (NEISS-CADES),” Medical Care, Vol. 45, No. 2, 2007, pp. S96–S102. Also see [Abstract].

Database Studies

Horn, S. D. and J. Gassaway. “Practice-Based Evidence Study Design for Comparative Effectiveness Research,” Medical Care, Vol. 45, No. 2, 2007, pp. S50–S57. Also see [Abstract].

Crystal, S. et al. “Studying Prescription Drug Use and Outcomes with Medicaid Claims Data: Strengths, Limitations, and Strategies,” Medical Care, Vol. 45, No. 2, 2007, pp. S58–S65. Also see [Abstract].

Grijalya, C.G. et al. “Assessment of Adherence to and Persistence on Disease-Modifying Anti-Rheumatic Drugs (DMARDs) in Patients with Rheumatoid Arthritis,” Medical Care, Vol. 45, No. 2, 2007, pp. S66–S76. Also see [Abstract].

Stürmer, T. et al. “Adjustments for Unmeasured Confounders in Pharmacoepidemiologic Database Studies Using External Information,” Medical Care, Vol. 45, No. 2, 2007, pp. S158–S165. Also see [Abstract].

Curtis, L.H. et al. “Using Inverse Probability-Weighted Estimators in Comparative Effectiveness. Analyses with Observational Databases,” Medical Care, Vol. 45, No. 2, 2007, pp. S103–S107. Also see [Abstract].

Instrumental Variables

Brookhart, M.A. et al. “Evaluating the Validity of an Instrumental Variable Study of Neuroleptics. Can Between-Physician Differences in Prescribing Patterns Be Used to Estimate Treatment Effects?” Medical Care, Vol. 45, No. 2, 2007, pp. S116–S122. Also see [Abstract].

Brooks, J.M. and E.A. Chrischilles. “Heterogeneity and Treatment Effect Estimates from Risk-Adjustment and Instrumental Variable Methods: Surgery for Early-Stage Breast Cancer,” Medical Care, Vol. 45, No. 2, 2007, pp. S123–S130. Also see [Abstract].

Angrist, J.D. et al. “Identification of Causal Effects Using Instrumental Variables,” Journal of the American Statistical Association, Vol. 81, 1996, pp. 444–455.

Brookhart, M.A. et al. “Evaluating Short-term Drug Effects Using a Physician-Specific Prescribing Preference as an Instrumental Variable,” Epidemiology, Vol. 17, 2006, pp. 268–75. Also see [Abstract].

Glynn. R.J. “Commentary: Genes as Instruments for Evaluation of Markers and Causes,” International Journal of Epidemiology, Vol. 35, 2006, pp. 932–934. Also see [Full Free PDF].

Propensity Score Matching

Seeger, J.D. et al. “Use of Propensity Score Technique to Account for Exposure-Related Covariates: An Example and Lesson,” Medical Care, Vol. 45, No. 2, 2007, pp. S143–S148. Also see [Abstract].

Segal, J.B. et al. “Using Propensity Scores Subclassification to Estimate Effects of Longitudinal Treatments: An Example Using a New Diabetes Medication,” Medical Care, Vol. 45, No. 2, 2007, pp. S149–S157. Also see [Abstract].

Stürmer, T. et al. “Comparison of Performance of Propensity Score Calibration (PSC) and Multiple Imputation (MI) to Control for Unmeasured Confounding Using an Internal Validation Study,” [abstract], Pharmacoepidemiology and Drug Safety, Vol. 15, pp. S39.

Glynn, R.J. et al. “Indications for Propensity Scores and Review of their use in Pharmacoepidemiology,” Basic and Clinical Pharmacology and Toxicology, Vol. 98, 2006, pp. 253–9. Also see [Abstract].

Buzas, J.S. and L.A. Stefanski. “Instrumental Variable Estimation in Generalized Linear Measurement Error Models,” Journal of the American Statistical Association, Vol. 91, 1996, pp. 999–1006.

Stürmer, T. et al. “Performance of Propensity Score Calibration – a Simulation Study,” American Journal of Epidemiology, Vol. 165, No. 10, 2007, pp. 1110-8. Also see [Full Free PDF].

*Comparison of RCT & Database studies:

Schneeweiss, S. et al. “Increasing Levels of Restriction in Pharmacoepidemiologic Database Studies of Elderly and Comparison with Randomized Trial Results,” Medical Care, Vol. 45, No. 2, 2007, pp. S131–S142. Also see [Abstract].

III. Syntheses (Synthesis & Meta-analysis; Simulation & Modeling)

Synthesis & Meta-Analysis

Agency for Healthcare Research and Quality. Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews, Version 1.0 [Draft posted Oct. 2007]. Rockville, MD. Also see [Full Free PDF].

Berlin JA, Cepeda MS. "Some Methodological Points to Consider When Performing Systematic Reviews in Comparative Effectiveness Research," Clinical Trials, November 2, 2011. [Free Full Text]

Pignone, M. et al. “Challenges in Systematic Reviews of Economic Analyses,” Annals of Internal Medicine, Vol. 142, No. 12, Pt. 2, June 21, 2005, pp. 1073-9. Also see [Abstract].

Vandermeer, B.W. et al. “Comparison of Meta-Analytic Results of Indirect, Direct, and Combined Comparisons of Drugs for Chronic Insomnia in Adults: A Case Study,” Medical Care, Vol. 45, No. 2, 2007, pp. S166–S172. Also see [Abstract].

Viswanathan M, Ansari MT, Berkman ND, Chang S, Hartling L, McPheeters LM, Santaguida PL, Shamliyan T, Singh K, Tsertsvadze A, Treadwell JR. Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Agency for Healthcare Research and Quality Methods Guide for Comparative Effectiveness Reviews. March 2012. AHRQ Publication No. 12-EHC047-EF. Available at: www.effectivehealthcare.ahrq.gov/

US Cochrane Center.“Understanding Evidence-based Healthcare: A Foundation for Action.”

Simulation & Modeling

Berry, D.A. et al. “Effect of Screening and Adjuvant Therapy on Mortality from Breast Cancer,” New England Journal of Medicine, Vol. 353, 2005, pp. 1784–92. Also see [Abstract/ Full Free PDF].

Feuer, E.J. “Modeling the Impact of Adjuvant Therapy and Screening Mammography on U.S. Breast Cancer Mortality between 1975 and 2000: Introduction to the Problem,” Journal of the National Cancer Institute Monographs, Vol. 36, 2006, pp. 2-6. Also see [Full Free PDF].

Joffe, M. and J. Mindell. "Complex Causal Process Diagrams for Analyzing the Health Impacts of Policy Interventions," American Journal of Public Health, Vol. 96, No. 3, March 2006, pp. 473-9. Also see [Full Free PDF].

Samore, M.H. et al. “A Simulation-Based Evaluation of Methods to Estimate the Impact of an Adverse Event on Hospital Length of Stay,” Medical Care, Vol. 45, No. 2, 2007, pp. S108–S115. Also see [Abstract].

[REPEAT] Stürmer, T. et al. “Performance of Propensity Score Calibration – a Simulation Study,” American Journal of Epidemiology, Vol. 165, No. 10, 2007, pp. 1110-8. Also see [Full Free PDF].

Weinstein, M.C. et al. “ISPOR Task Force on Good Research Practices--Modeling Studies,” Value Health, Vol. 6, No. 1, January – February, 2003, pp. 9-17.

IV. Policy Topics and Translation

Alemayeu, D., et. al. "Examination of Data, Analytical Issues and Proposed Methods for Conducting Comparative Effectiveness Research Using “Real-World Data,” Journal of Managed Care Pharmacy. November/December 2011 Supplement. Also see [Full Free PDF]

Lohr, K.N. “Emerging Methods in Comparative Effectiveness and Safety: Symposium Overview and Summary,” Medical Care, Vol. 45, No. 2, 2007, pp. S5–S8. Also see [Abstract].

Atkins, D. “Creating and Synthesizing Evidence with Decision Makers in Mind: Integrating Evidence From Clinical Trials and Other Study Designs,” Medical Care, Vol. 45, No. 2, 2007, pp. S16–S22. Also see [Abstract].

Sedrakyan, A. and C. Shih. “Improving Depiction of Benefits and Harms: Analyses of Studies of Well-known Therapeutics and Review of High-impact Medical Journals,” Medical Care, Vol. 45, No. 2, 2007, pp. S23–S28. Also see [Abstract].

Greenwald, L.M. “Medicare Part D Data: Major Changes on the Horizon,” Medical Care, Vol. 45, No. 2, 2007, pp. S9–S12. Also see [Abstract].

Rupper, R.W. et al. “Out-of-pocket Pharmacy Expenditures for Veterans under Medicare Part D,” Medical Care, Vol. 45, No. 2, 2007, pp. S77–S80. Also see [Abstract].

Touchette, D.R. et al. “Survey of Medication Therapy Management Programs Under Medicare Part D,” Journal of the American Pharmacists Association, Vol. 46, No. 6, November – December, 2006, pp. 683-91. Also see [Full Free PDF].

DeStefano F, Whitehead N, Lux LJ, Lohr KN. Infrastructure To Monitor Utilization and Outcomes of Gene-Based Applications: An Assessment. (Prepared by RTI International DEcIDE Center under Contract No. HSA290220050036I.) AHRQ Publication No. 08-EHC012. Rockville, MD: Agency for Healthcare Research and Quality. May 2008. Also see [Full Free PDF].

Schneeweiss, S. and J. Avorn, “A Review of Uses of Health Care Utilization Databases for Epidemiologic Research on Therapeutics,” Journal of Clinical Epidemiology, Vol. 58, 2005, pp.323–37. Also see [Abstract].