No R packages are permitted for use in this assignment.

  1. If \(P\) is a symmetric and idempotent matrix, show that the Pythagorean relationship holds:
    \[ \|y\|^2 = \|Py\|^2 + \|(I - P)y\|^2. \]
  2. Let \(G : \mathbb{R}^p \to \mathbb{R}\) defined by \(G(\beta) := (y - X\beta)'W(y - X\beta)\). Derive an expression for \(\nabla_\beta G(\beta)\).
  3. Suppose that \(Y_i \sim \text{Binomial}(p, n_i)\) for \(i \in \{1, \ldots, N\}\), and assume that \(Y_1, \ldots, Y_N\) are independent.
    1. Write this as a linear model.
    2. Find the BLUE of \(p\).
    3. Find the MLE of \(p\). How does the variance of the MLE compare to the variance of the BLUE?
  4. Suppose that \(Y_1, \ldots, Y_n \overset{\text{iid}}{\sim} \text{Uniform}(0, 2\theta)\), and define \(U_i := Y_i - \theta\) for \(i \in \{1, \ldots, n\}\).
    1. Find the mean and variance of \(U := (U_1, \ldots, U_n)'\).
    2. Show that \(Y := (Y_1, \ldots, Y_n)'\) is generated according to a linear model that satisfies the Gauss-Markov assumptions.
    3. Find the BLUE of \(\theta\), and denote the BLUE by \(\hat{\theta}_{\text{OLS}}\).
    4. Find \(c\) so that the estimator \(\hat{\theta} = cY_{(n)}\) is unbiased for \(\theta\), where \(Y_{(i)}\) denotes the \(i\)th order statistic, and compute the variance of \(\hat{\theta}\).
    5. Compare the variances of \(\hat{\theta}_{\text{OLS}}\) and \(\hat{\theta}\), and provide intuition for your finding.
  5. Show that if \(X\) is a \(p\)-dimensional random vector with mean \(\mu\) and covariance \(\Sigma\), \(A\) is a \(p \times p\) matrix, and \(Y = X'AX\), then \(E(Y) = \text{tr}(A\Sigma) + \mu'A\mu\).
  6. For a random vector \(Y\), with finite second moment, verify the following properties.
    1. \(E(a'Y) = a'E(Y)\), for a fixed vector \(a\).
    2. \(\text{Var}(a'Y) = a'\text{Var}(Y)a\), for a fixed vector \(a\).
    3. \(\text{Cov}(a'Y, c'Y) = a'\text{Var}(Y)c\), for fixed vectors \(a\) and \(c\).
    4. \(\text{Var}(A'Y) = A'\text{Var}(Y)A\), for a fixed matrix \(A\).