Title: Using Evidence Factors in Observational Studies: Concepts, Examples and Selected Methods

Summary: Observational studies are often biased by the failure to adjust for a covariate that was not measured. A series of studies may replicate an association because the bias that produced this association has been replicated, not because a treatment effect has been demonstrated. If a limited sample size is not the major problem in an observational study, then an increase in sample size is not the solution. To be of value, a replication should remove, or reduce, or at least vary a potential source of bias that resulted in uncertainty in earlier studies.

Having defined the goal of replication in this way, we may ask: Can one observational study replicate itself? Can it provide two statistically independent tests of one hypothesis about treatment effects such that the two tests are susceptible to different unobserved biases? Can the sensitivity analyses for these two tests be combined using meta-analytic techniques as if they came from unrelated studies, despite using the same data twice? Can such a combination provide stronger evidence that an association is an effect caused by the treatment, not a bias in who was selected for treatment? When this is possible, the study is said to possess two evidence factors. A study has two evidence factors if it permits two (essentially) statistically independent analyses using the same data that are affected by different types of unmeasured biases. More specifically, the sensitivity analyses for the two factors must be capable of combination as if they came from different unrelated studies, despite using the data twice.  The talk is divided into two parts:
(i) a brief, largely conceptual discussion of replication in observational studies;
(ii) a longer discussion of concepts underlying the use of evidence factors, and example of their use.

The talk is aimed at scientists engaged in causal inference from observational studies, and it omits the underlying statistical theory.  A parallel technical talk including the statistical theory is based on an IMS Medallion Lecture given in 2020, a version of which is available on YouTube https://www.youtube.com/watch?v=ONKJf_XoEuE.  Both aspects are discussed in a 2021 book entitled Replication and Evidence Factors in Observational Studies published by Chapman&Hall/CRC.

Biography: 

Paul R. Rosenbaum is the Robert G. Putzel Professor Emeritus in the Department of Statistics and Data Science at Wharton School of the University of Pennsylvania. He has written extensively about causal inference in observational studies, sensitivity analysis, optimal matching, design sensitivity, evidence factors, and (with Donald B. Rubin) the propensity score. With various coauthors, he has also written about health outcomes, racial disparities in health outcomes, instrumental variables, psychometrics, and experimental design.

Rosenbaum is the author of several books: Observational Studies (Springer 1995, 2002), Design of Observational Studies (Springer 2010, 2020), Observation and Experiment: An Introduction to Causal Inference, (Harvard University Press 2017), and Replication and Evidence Factors in Observational Studies (Chapman & Hall/CRC, 2021).

For work in causal inference, the Committee of Presidents of Statistical Societies gave Rosenbaum the R.A. Fisher Award and Lectureship in 2019 and the George W. Snedecor Award in 2003.

Register here

All welcome