Hints for Lab 5: Importance Sampling

For the class on Monday, January 29th

See also

Go back to Lab 5

B. Double the Rewards!

Hints for Part B: theoretical expected value

The theoretical expected value of your reward is \( (3/2)^{n_\text{flips}} \) dollars.

Hints for Part B3 - Code

n_flips = 1000
n_games = 100
n_sims = 10

p_baised_values = np.linspace(0, 1, 101)[1:-1]
typical_log_average_reward = []
for p_biased in p_baised_values:

    # The following inner part is identical to B2
    log_average_reward_each_sim = []
    for _ in range(n_sims):

        log_reward_each_game = []
        log_weight_each_game = []
        for _ in range(n_games):
            game_result = run_one_game(n_flips, p_biased)
            n_heads = np.count_nonzero(game_result)
            log_reward = np.log(2) * n_heads
            log_reward_each_game.append(log_reward)
            log_weight = calc_log_likelihood_ratio(n_flips, n_heads, p_biased)
            log_weight_each_game.append(log_weight)

        log_average_reward = logsumexp(np.add(log_reward_each_game, log_weight_each_game), b=1/n_games)
        log_average_reward_each_sim.append(log_average_reward)

    typical_log_average_reward.append(np.median(log_average_reward_each_sim))

plt.plot(p_baised_values, typical_log_average_reward)

## TODO: Uncomment the line below and replace ... with the theoretical expected value.
#plt.axhline(..., c="C3", lw=1)

## TODO: Uncomment the line below and replace ... with the best p_biased value.
#plt.axvline(..., c="C2", ls="--")

Hints for Part B3 - Question

Refer to the reading assignment. Look for the discussion on “zero-variance importance sampling”.