Problem 1

Suppose we have a sample $Y_1 \ldots Y_n$ drawn with replacement from a population $y_1 \ldots y_m$ with mean $\mu = \frac{1}{m}\sum_{j=1}^m y_j$. We have it in an vector $Y$ of length $n$. We run this code.

B=1000
Y.bar = rep(NA,B)
for(b in 1:B) {
  Y.bar[b] = mean(sample(Y,size=n,replace=TRUE))
}

Part A

What does this code calculate? State it in words and mark it with an ‘a’, and an arrow or bracket if necessary for clarity, on the appropriate plot above.

mean(Y) - 1.96* sd(Y)/sqrt(n)
mean(Y) + 1.96* sd(Y)/sqrt(n)

🔒

Solution

Locked (Week 0)

Part B

How could you calculate roughly the same thing as in Part A using sd(Y.bar) and not sd(Y)?

🔒

Solution

Locked (Week 0)

Problem 2

Suppose $Y_1 \ldots Y_n$ are sampled with replacement from a population with mean $\mu=1$ and variance $\sigma^2=1$. In Midterm 1, we discussed the estimator $\tilde{Y}_k = \frac{1}{n-k} \sum_{i=1}^n Y_i$ for $k=2$. Here, we’ll talk about that case and the case $k=\sqrt{n}$.

Part A

For both estimators $\tilde{Y}_2$ and $\tilde{Y}_{\sqrt{n}}$, for each of the three sample sizes $n=10$, $n=10^2=100$, and $n=10^4=10,000$, do the following.

Calculate the estimator’s mean $\E[\tilde{Y}_k]$ and variance $\Var[\tilde{Y}_k]$.
Sketch the estimator’s sampling distribution. Label your axes clearly and include $\mu=1$ in your sketch.
On top of your sketches, illustrate how you’ve calculate the coverage probability of the interval estimate $\tilde{Y}_k \pm 1.96 \sqrt{\Var[\tilde{Y}_{k}]}$.
Give a rough approximation of this coverage probability. You don’t have to do this well—just do it by eye. Is it about 95%? Is it above 50%? That’s the kind of precision I’m looking for.

You should have 6 sketches with corresponding means, standard errors, and coverage probabilities. That’s more than we’d ask for on the real exam. If you get the idea and want to save time by skipping the $n=10^2$ case, that’s fine.

🔒

Solution

Locked (Week 0)

Problem 3

In the plot below, I’m showing some data from a (fake) survey of people in your city who’ve recently finished with school and begun working. For whatever reason, nobody who attends school in this city goes to college, so everyone finishes with between 8 and 12 years of education. There are 10,000 of these people, and we mailed out (and received back) 2000 surveys, sampling from this population of 10,000 with replacement. The x-axis shows the survey recipient’s years of education, the y-axis sjpws their income, the points with arms are within-column means and standard deviations, and the heights of the bars are proportional to the proportion of recipients with each level of education.

Part A

Consider two summaries of the incremental value of a year of high school in this sample.

\[ \begin{aligned} \hat\theta_{\text{years}} &= \frac{\sum_{x \in 9 \ldots 12} \qty{\hat\mu(x) - \hat\mu(x-1)}}{4} \\ \hat\theta_{\text{people}} &= \frac{\sum_{i:X_i \in 9 \ldots 12} \qty{\hat\mu(X_i) - \hat\mu(X_i-1)}}{\sum_{i:X_i \in 9\ldots 12} 1} \\ \end{aligned} \]

Which is larger? Explain how you know.

🔒

Solution

Locked (Week 0)

These survey recipients went to two schools: School 0 (circles) and School 1 (triangles).¹

Children in your district go to School 0. Your cousin, who lives in a district that goes to School 1, encourages you to pretend that you and your family live with them so that your child can go to School 1. This is a thing that happens, at least on the TV show ‘Friday Night Lights.’ Their argument is that School B must be better because the average income of students who have attended it is clearly higher. Check out this confidence interval for the difference in mean incomes of students who have attended the two schools, they say.

Delta.hat = mean(Y[W==1]) - mean(Y[W==0])

Delta.hat.star = replicate(1000, {
  J.star = sample(1:n, size=n, replace=TRUE)
  W.star = W[J.star]
  X.star = X[J.star]
  Y.star = Y[J.star]
  
  mean(Y.star[W.star==1]) - mean(Y.star[W.star==0])
})

interval.width = width(Delta.hat.star)
confidence.interval = c(Delta.hat - interval.width/2, 
                        Delta.hat + interval.width/2)
confidence.interval

[1]  7323 12428

They even based their calculation on one of your QTM 285 homework solutions, so it’s got to be legit. School 1 is at least $7323 better.

You, on the other hand, look at the data and claim that a year at School 0 is worth just as much as a year at School 1. And you’re right. You calculate school-specific versions of $\hat\theta_{\text{years}}$, bootstrap the difference between them, and find convincing evidence that the incremental value of a year at each school is the roughly the same.

your.summary = function(W,X,Y) {
  muhat = Vectorize(function(w,x) { mean(Y[W==w & X==x]) })
  (mean(muhat(1,9:12)-muhat(1,8:11))) - (mean(muhat(0,9:12)-muhat(0,8:11)))
}

theta.hat.years.difference = your.summary(W,X,Y)

theta.hat.years.difference.star = replicate(100, {
  J.star = sample(1:n, size=n, replace=TRUE)
  your.summary(W[J.star], X[J.star], Y[J.star])
})
 

interval.width = width(theta.hat.years.difference.star)
confidence.interval = c(theta.hat.years.difference - interval.width/2, 
                        theta.hat.years.difference + interval.width/2)
confidence.interval

[1] -1518  1302

Part B

Explain, in terms your cousin can understand, how it’s possible that the average income of students who have attended School 1 is higher than the average income of students who have attended School 0, but the average incremental value of a year at both schools is the same.

Problem 4

Consider this population of 6 people in a randomized experiment.

$j$	$y_j(1)$	$y_j(0)$	$\tau_j$
1	6	2	4
2	0	0	0
3	4	1	3
4	7	7	0
5	8	4	4
6	2	0	2

Part A

What is the average treatment effect $\bar\tau = \frac{1}{6}\sum_{j=1}^6 \tau_j$?

🔒

Solution

Locked (Week 0)

Part B

Suppose we assign treatments $W_1 \ldots W_6 = (1, 0, 0, 1, 1, 0)$. Fill in the realized outcomes $Y_j = y_j(W_j)$ in the table below.

$j$	$W_j$	$Y_j$
1	1
2	0
3	0
4	1
5	1
6	0

🔒

Solution

Locked (Week 0)

Part C

Calculate the difference-in-means estimator $\hat\tau = \bar Y_1 - \bar Y_0$ for this treatment assignment, where $\bar Y_1$ is the mean outcome among the treated and $\bar Y_0$ is the mean among the untreated.

🔒

Solution

Locked (Week 0)

Part D

Is $\hat\tau$ equal to $\bar\tau$? In one sentence, explain why they differ.

🔒

Solution

Locked (Week 0)

Now suppose we randomize by choosing 3 people uniformly at random to treat (and the other 3 are untreated). There are $\binom{6}{3} = 20$ possible treatment assignments.

Part E

For the assignment $(1,1,1,0,0,0)$—treating persons 1, 2, and 3—calculate $\hat\tau$.

🔒

Solution

Locked (Week 0)

Part F

For the assignment $(0,0,0,1,1,1)$—treating persons 4, 5, and 6—calculate $\hat\tau$.

🔒

Solution

Locked (Week 0)

Part G

If we average $\hat\tau$ over all 20 possible assignments (each equally likely), what do we get? You don’t need to enumerate all 20—just state what the answer must be and why.

🔒

Solution

Locked (Week 0)

\(j\)	\(y_j(1)\)	\(y_j(0)\)	\(\tau_j\)
1	6	2	4
2	0	0	0
3	4	1	3
4	7	7	0
5	8	4	4
6	2	0	2

\(j\)	\(W_j\)	\(Y_j\)
1	1
2	0
3	0
4	1
5	1
6	0