Elements of Causal Inference

1 Statistical and Causal Models 1

1.1 Probability Theory and Statistics . . . . . . . . . . . . . . . . . . 1

1.2 Learning Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Causal Modeling and Learning . . . . . . . . . . . . . . . . . . . 5

1.4 Two Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Assumptions for Causal Inference 15

2.1 The Principle of Independent Mechanisms . . . . . . . . . . . . . 16

2.2 Historical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3 Physical Structure Underlying Causal Models . . . . . . . . . . . 26

3 Cause-Effect Models 33

3.1 Structural Causal Models . . . . . . . . . . . . . . . . . . . . . . 33

3.2 Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.3 Counterfactuals . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.4 Canonical Representation of Structural Causal Models . . . . . . 37

3.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4 Learning Cause-Effect Models 43

4.1 Structure Identifiability . . . . . . . . . . . . . . . . . . . . . . . 44

4.2 Methods for Structure Identification . . . . . . . . . . . . . . . . 62

4.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

5 Connections to Machine Learning, I 71

5.1 Semi-Supervised Learning . . . . . . . . . . . . . . . . . . . . . 71

5.2 Covariate Shift . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

5.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6 Multivariate Causal Models 81

6.1 Graph Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 81

6.2 Structural Causal Models . . . . . . . . . . . . . . . . . . . . . . 83

6.3 Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

6.4 Counterfactuals . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

6.5 Markov Property, Faithfulness, and Causal Minimality . . . . . . 100

6.6 Calculating Intervention Distributions by Covariate Adjustment . 109

6.7 Do-Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

6.8 Equivalence and Falsifiability of Causal Models . . . . . . . . . . 120

6.9 Potential Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . 122

6.10 Generalized Structural Causal Models Relating Single Objects . . 126

6.11 Algorithmic Independence of Conditionals . . . . . . . . . . . . . 129

6.12 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

7 Learning Multivariate Causal Models 135

7.1 Structure Identifiability . . . . . . . . . . . . . . . . . . . . . . . 136

7.2 Methods for Structure Identification . . . . . . . . . . . . . . . . 142

7.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

8 Connections to Machine Learning, II 157

8.1 Half-Sibling Regression . . . . . . . . . . . . . . . . . . . . . . . 157

8.2 Causal Inference and Episodic Reinforcement Learning . . . . . . 159

8.3 Domain Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . 167

8.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

9 Hidden Variables 171

9.1 Interventional Sufficiency . . . . . . . . . . . . . . . . . . . . . . 171

9.2 Simpson’s Paradox . . . . . . . . . . . . . . . . . . . . . . . . . 174

9.3 Instrumental Variables . . . . . . . . . . . . . . . . . . . . . . . 175

9.4 Conditional Independences and Graphical Representations . . . . 177

9.5 Constraints beyond Conditional Independence . . . . . . . . . . . 185

9.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

10 Time Series 197

10.1 Preliminaries and Terminology . . . . . . . . . . . . . . . . . . . 197

10.2 Structural Causal Models and Interventions . . . . . . . . . . . . 199

10.3 Learning Causal Time Series Models . . . . . . . . . . . . . . . . 201

10.4 Dynamic Causal Modeling . . . . . . . . . . . . . . . . . . . . . 210

10.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Appendices

Appendix A Some Probability and Statistics 213

A.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . 213

A.2 Independence and Conditional Independence Testing . . . . . . . 216

A.3 Capacity of Function Classes . . . . . . . . . . . . . . . . . . . . 219

Appendix B Causal Orderings and Adjacency Matrices 221

Appendix C Proofs 225

C.1 Proof of Theorem 4.2 . . . . . . . . . . . . . . . . . . . . . . . . 225

C.2 Proof of Proposition 6.3 . . . . . . . . . . . . . . . . . . . . . . . 226

C.3 Proof of Remark 6.6 . . . . . . . . . . . . . . . . . . . . . . . . 226

C.4 Proof of Proposition 6.13 . . . . . . . . . . . . . . . . . . . . . . 226

C.5 Proof of Proposition 6.14 . . . . . . . . . . . . . . . . . . . . . . 228

C.6 Proof of Proposition 6.36 . . . . . . . . . . . . . . . . . . . . . . 228

C.7 Proof of Proposition 6.48 . . . . . . . . . . . . . . . . . . . . . . 228

C.8 Proof of Proposition 6.49 . . . . . . . . . . . . . . . . . . . . . . 229

C.9 Proof of Proposition 7.1 . . . . . . . . . . . . . . . . . . . . . . . 230

C.10 Proof of Proposition 7.4 . . . . . . . . . . . . . . . . . . . . . . . 230

C.11 Proof of Proposition 8.1 . . . . . . . . . . . . . . . . . . . . . . . 230

C.12 Proof of Proposition 8.2 . . . . . . . . . . . . . . . . . . . . . . . 231

C.13 Proof of Proposition 9.3 . . . . . . . . . . . . . . . . . . . . . . . 231

C.14 Proof of Theorem 10.3 . . . . . . . . . . . . . . . . . . . . . . . 232

C.15 Proof of Theorem 10.4 . . . . . . . . . . . . . . . . . . . . . . . 232