Statistics & Time Series

Overview

Statistical inference is the science of drawing conclusions about unknown parameters from data. This topic covers the foundational criteria for estimators, the two main estimation methods (MOM and MLE), confidence intervals, hypothesis testing, and linear regression. These tools appear directly in quant interviews involving data analysis, model validation, and quantitative research.

1. Estimator Properties

1.1 Bias, Variance, MSE

Definition

Let $\hat{\theta} = \hat{\theta}(X_1, \ldots, X_n)$ be an estimator of a parameter . The three fundamental quality criteria are:

Test	$H_0$	Statistic	Distribution under $H_0$
$z$ -test	$\mu = \mu_0$ ( $\sigma$ known)	$Z = \frac{\bar{X}-\mu_0}{\sigma/\sqrt{n}}$	$\mathcal{N}(0,1)$
$t$ -test (one-sample)	$\mu = \mu_0$ ( $\sigma$ unknown)	$T = \frac{\bar{X}-\mu_0}{S/\sqrt{n}}$
$t$ -test (two-sample)	$\mu_1 = \mu_2$	$T = \frac{\bar{X}_1-\bar{X}_2}{S_p\sqrt{1/n_1+1/n_2}}$
$\chi^2$ -test	$\sigma^2 = \sigma_0^2$	$χ^{2} = \frac{(n - 1) S^{2}}{σ_{}^{}}$
$F$ -test	$\sigma_1^2 = \sigma_2^2$	$F = S_1^2/S_2^2$

Criterion	Definition
Unbiased	$E[\hat{\theta}] = \theta$
MSE	$\text{Var}(\hat{\theta}) + \text{Bias}^2$
Consistent	$\hat{\theta}_n \xrightarrow{P} \theta$
Efficient	Achieves Cramér-Rao bound
Cramér-Rao bound	$\text{Var}(\hat{\theta}) \geq 1/(n\mathcal{I}(\theta))$
Asymptotic MLE	$\sqrt{n}(\hat{\theta}-\theta) \xrightarrow{d} \mathcal{N}(0, 1/\mathcal{I}(\theta))$

Parameter	CI
$\mu$ ( $\sigma$ known)	$\bar{X} \pm z_{\alpha/2}\,\sigma/\sqrt{n}$
$\mu$ ( $\sigma$ unknown)	$\bar{X} \pm t_{\alpha/2,n-1}\,S/\sqrt{n}$
Proportion $p$	$\hat{p} \pm z_{\alpha/2}\sqrt{\hat{p}(1-\hat{p})/n}$

Concept	Formula
OLS estimator	$\hat{\boldsymbol{\beta}} = (X^TX)^{-1}X^T\mathbf{y}$
Covariance of $\hat{\boldsymbol{\beta}}$	$\sigma^2(X^TX)^{-1}$
Hat matrix	$H = X(X^TX)^{-1}X^T$
Residual variance	$\hat{\sigma}^2 = \\|\hat{\boldsymbol{\varepsilon}}\\|^2/(n-p)$
$R^2$	$1 - \text{SS}_\text{res}/\text{SS}_\text{tot}$
Adjusted $R^2$	$1 - (1-R^2)(n-1)/(n-p)$

Statistics & Time Series

1. Estimator Properties

1.1 Bias, Variance, MSE

1.2 Consistency & Efficiency

1.3 Cramér-Rao Bound & Fisher Information

2. Estimation Methods

2.1 Method of Moments (MOM)

2.2 Maximum Likelihood Estimation (MLE)

2.3 Asymptotic Properties of MLE

3. Confidence Intervals

3.1 Definition & Interpretation

3.2 CI for the Mean (known $\sigma$ )

3.3 CI for the Mean (unknown $\sigma$ )

3.4 CI for a Proportion

4. Hypothesis Testing

4.1 Framework: $H_0$ , $H_1$ , Errors, $p$ -value

4.2 Common Tests

4.3 Neyman-Pearson Lemma

5. Linear Regression

5.1 Model & Assumptions

5.2 OLS Estimator

5.3 Gauss-Markov Theorem

5.4 Goodness of Fit ( $R^2$ )

6. Quick Reference

Estimator Criteria

Confidence Intervals

Regression Formulas

Statistics & Time Series

1. Estimator Properties

1.1 Bias, Variance, MSE

1.2 Consistency & Efficiency

1.3 Cramér-Rao Bound & Fisher Information

2. Estimation Methods

2.1 Method of Moments (MOM)

2.2 Maximum Likelihood Estimation (MLE)

2.3 Asymptotic Properties of MLE

3. Confidence Intervals

3.1 Definition & Interpretation

3.2 CI for the Mean (known σ\sigmaσ)

3.3 CI for the Mean (unknown σ\sigmaσ)

3.4 CI for a Proportion

4. Hypothesis Testing

4.1 Framework: H0H_0H0​, H1H_1H1​, Errors, ppp-value

4.2 Common Tests

4.3 Neyman-Pearson Lemma

5. Linear Regression

5.1 Model & Assumptions

5.2 OLS Estimator

5.3 Gauss-Markov Theorem

5.4 Goodness of Fit (R2R^2R2)

6. Quick Reference

Estimator Criteria

Confidence Intervals

Regression Formulas

3.2 CI for the Mean (known $\sigma$ )

3.3 CI for the Mean (unknown $\sigma$ )

4.1 Framework: $H_0$ , $H_1$ , Errors, $p$ -value

5.4 Goodness of Fit ( $R^2$ )