Ctrl+K

Reading 3: Classical Inference I: Maximum Likelihood Estimation

Reading 3: Classical Inference I: Maximum Likelihood Estimation#

For the class on Monday, January 22nd

Reading assignments#

  1. Read the following sections of [ICVG20] (sections based on the 2020 revised edition)

    • Sec. 4.2 “Maximum Likelihood Estimation (MLE)”, and its first 6 subsections (4.2.1–4.2.6)

  2. Read the following sections of [Mur22] (sections based on the 2023-06-21 draft PDF file)

    • Sec. 4.2.3 “Example: MLE for the Bernoulli distribution”

    • Sec. 4.7.1 “Sampling distributions” (focusing on what an “estimator” is)

Questions#

Hint

Submit your answer on Canvas. Due at noon, Monday, Jan 22nd.

  1. List any concepts that confuse you from your reading. Explain why they confuse you. If nothing confuses you, briefly summarize what you have learned from this reading assignment.

  2. You have some independent measurements \(N_1, N_2, \dots, N_i\) that you believe come from one particular Poisson distribution with an unknown mean \(\lambda\). Derive the Maximum Likelihood Estimation of \(\lambda\) given \(N_1, N_2, \dots, N_i\).

    Hint

    The pmf of a Poisson distribution with a mean of \(\lambda\) is \(P(N) = \frac{\lambda^N e^{-\lambda}}{N!}\)

Discussion Preview#

Note

We will discuss the following questions in class. They are included here so that you have a chance to think about them before class. You need not submit your answers as part of this assignment.

  1. You flipped a coin \(N\) times, and you got heads \(K\) out of \(N\) times. What is the likelihood that this coin is a fair coin (express in terms of \(N\) and \(K\))? If you flip a coin 100 times, at least how many heads do you need to see to say that the coin favors heads at a \(2\sigma\) level?

  2. Use your own words to explain what is an estimator and give some specific examples of estimators.

  3. Consider the following definitions:

    • A consistent estimator is an estimator that converges (in probability) to the true value of what the estimator is estimating as the number of data points increases.

    • An unbiased estimator is an estimator whose expected value equals to the true value of what the estimator is estimating.

    Do you know whether the examples of estimators that you give in the previous discussion question are consistent and/or unbiased? How can you find out?

Contents