Causality

Module 2.1: Identification

Author

Affiliation

Alex Cardazzi

Old Dominion University

All materials can be found at alexcardazzi.github.io.

Terms

Let’s begin this module by defining a few terms. Be on the lookout for these terms, and more, throughout the notes bolded and underlined.

Estimand – any quantity we want to estimate
- Causal Estimand – the change in some outcome $Y$ given some treatment $D$
- Statistical Estimand – the change in the expectation of $Y$ given $D$
Estimate – an approximation of an estimand, using data
Estimator – function, or series of functions, applied to sample data to generate an estimate
- Estimation – the act of applying the estimator to data.

Second, here is a flow chart to help you visualize the relationship between these terms:

Causality

When you’re first taught econometrics, much focus is placed on estimation and, perhaps to a lesser extent, inference. Your professor, maybe even me, probably tried to hammer home “a one unit change in $X$ is associated with a $\widehat{\beta}$ units change in $Y$.” This is estimation. Then, you were probably asked to test if $\widehat{\beta} = 0$ by considering $\beta$’s standard error. This is inference.

Once you’re familiar with how to estimate and interpret $\beta$s, which admittedly is not always easy, you can start thinking more about your modeling decisions. Estimation and interpretation are the spelling and grammar of modern econometrics, and you need strong foundations as you take this next step.

The next step I’m talking about is called identification. In this module, we are going to start thinking about how to estimate a plausibly causal relationship between $X$ and $Y$ rather than a correlational relationship between $X$ and $Y$. Before jumping into identification, though, we need to lay out some more foundations.

Estimands

Economists use econometric models to examine the effects of some policy, shock, or treatment on some outcome. For example, suppose you are in charge of figuring out if a new pill successfully alleviates headaches. Perhaps the best way to test the effectiveness of the drug would be to recruit a number of participants who have headaches. Some of these participants would be given the new pill while the other half would not be allowed to take anything. Consider individual $i$ who is part of this study.

Let $D$ denote their treatment status. We can represent being given the pill with $D=1$. Therefore, $D=0$ will represent being untreated.
Let $Y$ denote the status of their headache. We can represent no headache with $Y=1$. Of course, this means that $Y=0$ represents having a headache.

Individual $i$’s outcome, when treated, can be written as $Y_i|_{D = 1}$ (or $Y_i|_{D=0}$ if untreated). However, as shorthand, you’ll see this written as either $Y_i(1)$ or $Y_{i}^1$. If $Y_i(1) = 1$ and $Y_i(0) = 0$, then we’d say that the new pill worked. On the other hand, if $Y_i(1) = 0$ and $Y_i(0) = 0$, we’d probably say that the pill did not work. Therefore, the treatment effect can be written as the difference between $Y_i(1)$ and $Y_i(0)$: $Y_i(1) - Y_i(0)$.

If the treatment effect for person/unit $i$ is $Y_i(1) - Y_i(0)$, then we can write the average treatment effect (or ATE) as $E[Y_i(1) - Y_i(0)] = E[Y_i(1)] - E[Y_i(0)]$. Remember, this is shorthand for $E[Y_i|_{D = 1}] - E[Y_i|_{D = 0}]$. This statement is our causal estimand – the quantity we want to estimate. We need to find a statistical representation of this so we can take it to data.

In statistics, the expectation of $Y$, $E[Y]$, simply represents the average of $Y$. The expectation of $Y$ given $X$, $E[Y|X]$, is the conditional average of $Y$. The connection between this and econometrics is that you can think of OLS as a way to calculate conditional expectations. This represents our statistical estimand.

If the data generating process for $Y$ is $Y = \alpha + \delta X + \epsilon$, then $E[Y] = E[\alpha + \delta X + \epsilon]$. This can be simplified to $\widehat{\alpha} + \widehat{\delta}E[X]$. Further, consider $E[Y|X]$. $E[Y | X] = \widehat{\alpha} + \widehat{\delta}E[X|X]$. This can be simplified further to $\widehat{\alpha} + \widehat{\delta}X$ since $E[X|X] = X$.

Back to the ATE – in words, this is equal to the average difference in the expected outcome of $Y$ minus the average outcome of $Y$ for untreated units.

Okay, cool, but there’s a big issue with all of this… We are talking about $Y_i(1)$ and $Y_i(0)$. How can we observe both $Y_i(1)$ and $Y_i(0)$? Either individual $i$ is treated or they’re not!

In other words, there are two possible states of the world. In one, individual $i$ takes the pill, and their outcome is $Y_i(1)$. In the other version of the world, where they don’t take the pill, their outcome is $Y_i(0)$. To find the treatment effect like I talked about above only works if we can simultaneously observe both of these versions. It’s not even enough for us to believe in parallel universes – we would need to observe each outcome in each universe. This is the fundamental problem of causal inference, and this issue is going to motivate the rest of this class.

So, how can we plausibly estimate treatment effects? Since, at the time of writing, we’re limited to only a single universe, we usually have to rely on observational data. Like in the new pill example, there might be a group of treated individuals and a group of untreated individuals. If the only difference between the groups is their treatment status, then we can attribute any differences in outcomes to the difference in treatment status.¹ In other words, we can use $E[Y|D=1]$ to approximate $E[Y(1)]$. Think of $E[Y|D=1]$ as the average of $Y$ for the subset of individuals who are treated. This is subtly different from $E[Y(1)]$, which is is the average of $Y$ for all individuals in the specific universe they are treated. If the treated subset of individuals are a random sample of all individuals, then $E[Y|D=1] \approx E[Y(1)]$. This is why randomized control trials (RCTs) are considered the gold standard in causal inference. With randomization, we can be sure that the treated subset is representative of the rest of the population.

However, it’s usually impossible to run RCTs in real life. For example, if I want to know the effect of speed limits on crash rates, I cannot go around randomly changing the speed limits of different highways. Rather, I have to rely on natural experiments that occur out in the wild. Usually, these natural experiments are comprised of one group that experiences some change and another very similar group that does not. For example, maybe a few states change their speed limits but others do not, or the law change affects interstates by not state routes. Natural experiments can also occur when some institutional quirk allows us to isolate and leverage random variation in treatment. As an example, lotteries of all shapes and sizes (e.g. military drafts, health insurance lotteries, housing voucher lotteries, etc.) can mimic the randomness of an RCT. Using data in a way that allows us to pick out the variation we care about is called identification. We’ll discuss different identification strategies to estimate causal treatment effects in observational data as we progress throughout this course.

Identification

Before working with natural experiments, let’s first consider some toy models to demonstrate what identification really means. Suppose I have the following hypothesis:

Sleeping with shoes on leads to headaches in the morning.

How could I test this hypothesis?

The first option would be to run an RCT. In this RCT, I would sneak into homes at random as some Nike-Santa and put shoes onto people who are asleep. Then, I would record information on whether the participants had headaches the next morning. Of course, this is next to impossible, but let’s think through this. If I were to estimate a treatment effect, I would be able to interpret it as causal since the only thing differing between the groups is their treatment status. In this setting, the randomization allows me to ensure that I am identifying the treatment effect and not some other factor.

In the absence of an RCT or natural experiment, I need to rely on observational data. In this case, I would have to find some people who slept with their shoes on and some people who did not. Suppose the data I collect are as follows.

Table of Treatment Status and Observed Outcomes
Individual $(i)$	Treatment $(D)$	Outcome $(Y)$
1	1	1
2	1	1
3	1	0
4	1	1
5	0	0
6	0	1
7	0	0
8	0	0

Reminder: $D=1$ for individuals who slept with their shoes one, and $Y = 1$ for individuals who woke up with a headache.

Remember, we can only observe a single universe, so this table is really a condensed version of the following:

Table of Treatment Status and Potential Outcomes
Individual $(i)$	Treatment $(D)$	Outcome $(Y(0))$	Outcome $(Y(1))$
1	1	?	1
2	1	?	1
3	1	?	0
4	1	?	1
5	0	0	?
6	0	1	?
7	0	0	?
8	0	0	?

Here we observe most of the treated group have an outcome of 1 while most of the control group have an outcome of 0. Using these data, we can calculate $E[Y|D = 0]$ and $E[Y|D = 1]$ (the statistical estimands) with the parts of the data that we can observe as stand-ins for $E[Y(0)]$ and $E[Y(1)]$ (the causal estimands). Since $E[Y | D = 1] = 0.75$ and $E[Y | D = 0] = 0.25$, it seems that wearing shoes when you sleep does lead to more headaches! If we do this, our strategy to identify the treatment effect of shoes on headaches is simply throwing our hands up and assuming (hoping?) that treatment was randomly assigned.

Before we run to tell people not to wear shoes when they sleep, let’s think about this a bit deeper. Is assuming random assignment of treatment a good assumption? Put differently, might there be something causing both sleeping with shoes on and waking up with a headache? Is there something about the treated individuals that makes them different from the untreated individuals? Since this is observational, there could be many reasons!

One possible reason would be drinking the night before. Drunk people are much more likely to fall asleep with their shoes on and wake up with a headache, or so I’ve been told. This is an example of a confounding variable: something that causes both the outcome and the treatment. When we have confounding variables, we need to control for them. In other words, we want to account for the factor that differs between the treatment and control groups. This is what we call our identification strategy. When we had our RCT, that randomization was our identifications strategy. In this case, our identifications strategy is controlling for this confounding variable.

How do we implement identification strategies? In short: econometrics. Using OLS, we can account for, or control for, confounding variables by including them in our regressions. For example, we can write the initial regression as the following:

\[\text{Headache}_i = \alpha + \delta\text{Shoes}_i + \epsilon_i\]

Here, $\widehat{\delta}$ will be biased. In other econometrics courses, you might have heard of this as omitted variable bias. To address omitted variable bias, we need to control for the omitted, or confounding, variable. We can modify the regression like the following:

Note: drinking is positively correlated with both wearing shoes to bed and waking up with a headache. Therefore, $\widehat{\delta}$ would be exhibit positive biased. In other words, we would probably fine $\widehat{\delta} > 0$ because wearing shoes to bed is proxying for drinking the night before.

\[\text{Headache}_i = \alpha + \delta\text{Shoes}_i + \beta\text{Drink}_i + \epsilon_i\]

In this regression, we will get a plausibly causal estimate of $\delta$ since we’ve controlled for the influence of drinking. In reality, it’s very likely that $\delta \approx 0$, but now we can think of this as causal rather than correlational.²

Directed Acyclic Graphs

In the examples above, it was fairly easy to pick out confounding variables and think through the logic of how to control for them. However, as models gets more complicated, we need a systematic way to think through them. Many economists have begun to use Directed Acyclic Graphs, which are more commonly known as DAGs, to illustrate or graphically represent causality. Let’s break down the meaning of DAGs:

Directed: One thing causes one (or many) other thing(s).
Acyclic: No feedback in causality!
Graph: a visual.

Every DAG is be made up of nodes, which are causal factors or variables. Each node will be connected to at least one other node by an edge. Edges establish the flow of causality between nodes via arrows.

To do this in R, we are going to install and load the dagitty package.

Code

library(dagitty)

To start, let’s re-imagine our RCT where we randomly put shoes on people as they sleep. We can illustrate this setting with a DAG. First, let’s label our variables. Our treatment, wearing shoes, will be denoted by $D$. Our outcome, headaches, will be labeled $Y$, and drinking the night before as $X$. Second, we are going to use the dagitty() function. In this function, we need to do the following:

We need to tell dagitty that we are trying to make a DAG. We do this by typing dag at the very beginning of our string.
Next, we need to tell dagitty what nodes we’ll be using for our outcome and exposure (treatment).
Finally, tell dagitty how each node is connected.

Code

dag <- dagitty("dag {
  D [exposure]
  Y [outcome]
  
  D -> Y
  X -> Y
}")

plot(dag)

Here we have two factors that cause Y. Since there’s only one path from $D$ to $Y$, we are free to estimate a regression of the following form:

\[Y_i = \alpha + \delta D_i + \epsilon_i\]

In this DAG, there is not much use in controlling for $X$ since it is independent of $D$. If we were to control for $X$, though, we’d be removing something from the error term. This should increase the precision of our estimate (i.e. shrink our standard errors), but ultimately leave $\widehat{\delta}$ unchanged.

Confounders

What would the DAG look like if we did not have an RCT to assign treatment? Now, drinking would cause both shoes and headaches. We’d have to re-write the DAG as the following:

Code

dag <- dagitty("dag {
  D [exposure]
  Y [outcome]

  D -> Y
  X -> Y
  X -> D
}")

plot(dag)

Display Output

Note that there are now two paths from $D$ to $Y$:

$D \rightarrow Y$
$D \leftarrow X \rightarrow Y$

I know this is a bit weird with the way the arrows are drawn, but this is the norm. This second path, where you see two arrows pointing away from one another $(\leftarrow X \rightarrow)$, is called a backdoor path. Any time you see an arrow pointing back at the treatment variable, you have a backdoor path. For every backdoor we see, we need to control for something along that path (that isn’t the treatment or the outcome, of course).

In fact, dagitty has a function called paths() that will spit out the paths from exposure (what we call treatment) to the outcome. You’ll notice that the paths are written the same way as I’ve written them, and the output also tells us that both paths are open.

In this case, since $X$ is the only variable on this backdoor path, we need to control for it. This will block/close the backdoor path from D to Y, which is not the path we’re interested in. Once $X$ is controlled for, and assuming we’re satisfied with this DAG, we can interpret $\widehat{\delta}$ as the causal effect of wearing shoes while asleep on waking up with a headache.

Translating from DAG to regression, we would write $Y_i = \alpha + \delta D_i + \beta X_i + \epsilon_i$. Without $\beta X_i$ in the regression, our analysis would suffer from confounding or omitted variable bias.

We can also use a function from dagitty, adjustmentSets(), to help us identify which variables we need to control to identify a causal effect of the treatment on the outcome.

Code

cat("Adjustment Set for DAG:\n")
print(adjustmentSets(dag, effect = "direct"))

Adjustment Set for DAG:
{ X }

In this function call, I am specifying effect = "direct". There are two options you can use: "total" or "direct". Which one you pick simply depends on your research question, and we can discuss this at another time.

Of course, estimating this regression is only possible if we can observe and measure $X$. If $X$ was either unobservable or unmeasurable, we would not be able to control for it, and thus be unable to interpret $\delta$ as causal. Let’s modify our DAG such that $X$ is unobservable and see what the output of adjustmentSets() gives us.

Code

dag <- dagitty("dag {
  D [exposure]
  Y [outcome]
  X [unobserved]
  
  D -> Y
  D <- X -> Y
}")
plot(dag)
cat("Adjustment Set for DAG:\n")
print(adjustmentSets(dag, effect = "direct"))

Standard Output

Adjustment Set for DAG:

Display Output

When $X$ was observed, dagitty told us to control for it. However, now that $X$ is unobserved, it no longer tells us to control for it. This is not because we don’t have to, but rather dagitty won’t tell us to control for something that we physically can’t. Unfortunately, given the current DAG, there’s no way to isolate the effect of $D$ on $Y$.

However, if $X$ causes another observable thing, $W$, before causing $D$, then we could just condition on $W$ instead of $X$ because $W$ is on the same backdoor path.

Code

dag <- dagitty("dag {
  D [exposure]
  Y [outcome]
  X [unobserved]
  
  D -> Y
  D <- W <- X -> Y
}")
plot(dag)
cat("Paths in DAG:\n")
print(paths(dag))
cat("Adjustment Set for DAG:\n")
print(adjustmentSets(dag, effect = "direct"))

Standard Output

Paths in DAG:
$paths
[1] "D -> Y"           "D <- W <- X -> Y"

$open
[1] TRUE TRUE

Adjustment Set for DAG:
{ W }

Display Output

These are the basics of how to handle confounding variables and sketching DAGs. Of course, there’s more to it, but in the interest of not being too verbose, I will let Nick Huntington-Klein fill in some of those gaps with the following two (optional) videos:

Colliders

The opposite of backdoor paths are collider paths. To identify collider paths, somewhere along the path you’ll see something like this: $\rightarrow X \leftarrow$. These paths are already closed. When we have backdoor paths, we need to control for at least one node along the path to close the backdoor. If we condition on a node on a collider path, we are actually opening a new backdoor path that had been already closed to start.

This might be a bit confusing to start, but think through this example. Suppose you’re interested in the effect of someone’s IQ (intelligence³) on the IQ of their significant other. In other words, do intelligent people tend to match with people with similar intelligence? To investigate this, we could collect data on the IQ of each subject $(D)$, their significant other $(Y)$, and their first-born child $(X)$. Let’s draw the DAG.

Code

dag <- dagitty("dag{
               Y[outcome]
               D[exposure]
               
               D -> Y
               Y -> X
               D -> X
}")
plot(dag)

Display Output

In this DAG, both the subject and their significant other cause the IQ of their child. The edge connecting $D$ and $Y$ is then the relationship we’d like to test. This should make some intuitive sense as IQ is likely genetic.

Since the relationship between $D$ and $Y$ is what we’re testing, this variable will appear in our regression: $Y_i = \alpha + \delta D_i + \epsilon_i$. There are two reasons why $X$, the IQ of the child, should not appear in the regression:

Since we are trying to model the data generating process for $Y$, the right side of this equation should only contain factors that cause $Y$. Since our DAG shows $Y\rightarrow X$, we can safely omit $X$ from the equation.
Suppose in reality, there is no relationship between $D$ and $Y$. Further, suppose $X$, the child’s IQ, is an average of $D$ and $Y$. If we know that a child’s IQ is 100, then at least one of their parent’s IQs must also be high. If we know the IQ of the high-IQ parent is 120, then we know the IQ of the other parent must be 80. If the IQ of the high-IQ parent increases to 130, then we know the IQ of the other parents must be 70. Controlling for the child’s IQ is creating a negative correlation between the two parents’ IQs out of thin air!

Let’s explore a simple example simulation of this setting.

For another example, take a look at this discussion about Beauty and Talent from The Mixtape.

Headaches

For practice, let’s build a DAG to explain headaches and to study the effects of taking medicine. The variables and paths we’ll consider are as follows:

We are testing the effects of medicine on headaches.
Being sick causes headaches.
Being sick also causes taking medicine.
Rain causes sickness.
Drinking the night before causes headaches.
Being sick causes doctor visits.
Doctor visits cause taking medicine.

Drawing this setting out in a DAG would look like the following:

Code

dag <- dagitty("dag {
  Headache[outcome]
  Medicine [exposure]
  
  Medicine -> Headache
  Sick -> Headache
  Sick -> Medicine
  Rain -> Sick
  Drinking -> Headache
  Headache -> Doctor
  Sick -> Doctor
  Sick -> Medicine
}")
plot(dag)

Display Output

Let’s check on the paths of this DAG.

Code

p <- dagitty::paths(dag)
as.data.frame(p)

Standard Output

                                   paths  open
1                   Medicine -> Headache  TRUE
2 Medicine <- Sick -> Doctor <- Headache FALSE
3           Medicine <- Sick -> Headache  TRUE

The first path is the one we’re interested in.
Second, we have a backdoor path that is in fact closed, due to Doctor being a collider.
The final path is an open backdoor that we would need to close. We would do this by controlling for Sick in a regression.

An equation to identify the impact of medicine on headaches would be the following:

\[\text{Headache}_{i} = \alpha + \delta \times \text{Medicine}_{i} + \beta\times \text{Sick}_i + \epsilon_i\]

Here, our estimate of $\delta$, $\widehat{\delta}$, can be interpreted as causal.

Future Modules

In upcoming modules, we will be exposed to different research designs (like RCTs) that leverage variation in observational data to identify causal effects. Each of these designs rely on their own sets of assumptions, data, etc. Causal inference is the umbrella term for these designs, and it’s become a rapidly growing field.

In Class Exercise

In Homework 1, I asked you to “estimate a model to explain a woman’s wages,” and I had you pick the variables. Most students do the sensible-sounding thing: they included everything that seemed related to wages, and justify each variable with a story about why it matters. More controls, better model. Right?

Here is what I did not tell you at the time: “explain wages” is two different questions.

Prediction: given what we know about a woman, what is our best guess of her wage?
Causation: if something about a woman changed, what would happen to her wage?

For prediction, the kitchen sink is a fine strategy: if husband’s wage improves your guess, throw it in; you don’t care why it works. For causation, the kitchen sink can be actively harmful. The catch is that both questions are answered with the same lm() command that does not know which one you are asking.

So let’s ask a causal question: what is the effect of a woman’s education on her wage?

A DAG for wages

Using the Homework 1 variables, a plausible set of relationships:

Education affects wages (the effect we want).
Age affects education (older cohorts of women completed less schooling) and affects wages.
Living in a city affects educational opportunities and affects wages.
Education affects fertility, and kids affect wages through hours and interrupted careers.
Education affects who a woman marries: in a marriage market with assortative matching, more educated women tend to match with higher-earning men.

Code

library("dagitty")
dagitty("dag {

EDU[exposure]
WAGE[outcome]

EDU -> WAGE
EDU -> KIDS
EDU -> HWAGE
KIDS -> WAGE
AGE -> EDU
AGE -> WAGE
CITY -> EDU
CITY -> WAGE

}") -> dag_wage
plot(dag_wage)
as.data.frame(paths(dag_wage))
adjustmentSets(dag_wage, effect = "total")

Standard Output

                paths open
1 EDU -> KIDS -> WAGE TRUE
2         EDU -> WAGE TRUE
3  EDU <- AGE -> WAGE TRUE
4 EDU <- CITY -> WAGE TRUE
{ AGE, CITY }

Display Output

Reading the paths: age and city sit on open backdoors (confounders), so we have to control for them. Kids are a mediator, part of the effect we want, so we leave them alone. Husband’s wage is just a child of the treatment that leads nowhere, so it is irrelevant. Notice what this already implies about Homework 1: most of the variables in your regression should not have been there, and dagitty says a lean specification identifies the effect. So far, causality looks good!

A Woman’s Ability

This DAG is missing something: ability, a stand-in for intelligence, motivation, and family background. Ability plausibly drives schooling, drives wages directly, and (if the marriage market sorts on it) drives who a woman marries. It also appears in no dataset, which we tell dagitty with the [latent] tag. A DAG describes the world, not your spreadsheet.

Code

dagitty("dag {

EDU[exposure]
WAGE[outcome]
ABIL[latent]

ABIL -> EDU
ABIL -> WAGE
ABIL -> HWAGE
EDU -> WAGE
EDU -> KIDS
EDU -> HWAGE
KIDS -> WAGE
AGE -> EDU
AGE -> WAGE
CITY -> EDU
CITY -> WAGE

}") -> dag_abil
plot(dag_abil)
adjustmentSets(dag_abil, effect = "total")

Display Output

dagitty returns nothing, and that is not a bug. The backdoor EDU ← ABIL → WAGE runs straight through a node we cannot measure, so no set of variables in this dataset identifies the effect of education on wages. Adding controls does not converge to the causal effect; it just moves us between differently-biased numbers, and the R output for every one of them looks equally respectable.

Ability also booby-traps a variable that was harmless a minute ago. Husband’s wage is now a collider: EDU → HWAGE ← ABIL. Left alone, that path is closed. Control for hwage, as many students do, and it opens: among women married to high earners, the less educated ones must have had something else going for them, so conditioning on hwage manufactures a correlation between education and ability, and ability drives wages. One of the most defensible-sounding controls (“household resources matter!”) is now one that quietly taints the estimate.

The Employed Subsample

In Homework 1, students were also asked to run their regression on employed women, coefficients moved, and the answer to “why?” was an interpretation pivot. Now we have a broader vocabulary. Working is not random: education raises the probability of working, and so do unobserved wage-relevant traits (ability again). In DAG terms, EDU → EMPLOYED ← ABIL, which makes employment a collider, and restricting the sample to employed women is just a blunt way of conditioning on it. The subsample estimate is contaminated by the same mechanism as the hwage control. Meanwhile the full-sample regression treats non-working women as earning $0.00 per hour, mixing “what determines wages” with “what determines working at all.” Neither regression is the causal effect, and they differ for reasons that have nothing to do with one of them being right.⁴

Grading Yourself

You were asked whether the education coefficient tells us what happens if a woman gets more schooling. You now know the honest answer: not with those controls, and, given ability, not with any controls available in this dataset.
You were asked why her wages and her husband’s wages move together even if his raise does nothing to her paycheck. Whatever story you wrote (matching, shared background, similar people marrying), you were describing hwage as a common effect of things about her, which is precisely why it is a collider and precisely why it does not belong in the regression.

The variables you chose were not wrong, and neither were your rationale sentences. What was missing was the question. “Does this variable relate to wages?” is a prediction question, and everything you included passes it. “Does this variable sit on a backdoor path between education and wages?” is the causal question, and most of your controls fail it.

If controls cannot solve the ability problem, we need variation in education that has nothing to do with ability: something that assigns schooling to people rather than letting people select it. That is what the rest of this course is about, starting with the next homework, where a hockey league will do us the favor of assigning a treatment by rule.

Footnotes

Of course, differences could also be due to random chance, but this is why we use hypothesis tests and/or confidence intervals↩︎
Is drinking before bed the only confounder? Maybe. Maybe not. Ultimately, this is something you would have to argue in your writing.↩︎
In economics, this is usually called ability or something similar. I am using IQ since it’s a bit more quantifiable albeit imprecise at best and pseudo-science at worst.↩︎
Sample selection in women’s wage equations, using these very data, is what James Heckman’s famous correction was built for, and it is a large part of why he has a Nobel Prize. You stumbled into a Nobel-level problem in week one; you just didn’t have the words for it yet.↩︎

Individual \((i)\)	Treatment \((D)\)	Outcome \((Y(0))\)	Outcome \((Y(1))\)
1	1	?	1
2	1	?	1
3	1	?	0
4	1	?	1
5	0	0	?
6	0	1	?
7	0	0	?
8	0	0	?