Framework

In a single response to prompt $x$, the model emits $K$ hypotheses $h_k \in \mathcal{Y}$ with weights $w_k \ge 0$ plus an Other bucket $w_0$, all summing to 1. A categorical distribution with a residual bucket — probabilistic programs are the natural generalization (explored on DDXPlus).

Listed hypotheses get their stated weight; all unlisted outcomes split $w_0$ in proportion to the base LM’s own probabilities — the LM-prior backoff. This is what gives the output full support over $\mathcal{Y}$, which the log score needs: with $w_0 = 0$ and gold $y^* \notin \mathcal{H}$, the reward is $-\infty$.

A scoring rule $S$ grades a predicted distribution against an outcome. It is strictly proper when the expected score is uniquely maximized by reporting the true distribution — so optimizing it drives the prediction toward the target.

The log score is unbounded below; Brier and spherical are bounded alternatives that trade the $-\infty$ cliff for a finite miss penalty. That cliff is why the Other bucket is load-bearing under the log score but only useful under the bounded ones.

Apply the scoring rule to the induced distribution at the gold outcome. Under the log score the reward is just $\log p_V$ of the gold; in expectation that is the negative cross-entropy $-H(p^*, p_V)$, maximized at $p_V = p^*$.

When the dataset gives the gold distribution directly — ProtoQA’s crowd counts, DDXPlus’s per-case differential — the reward is the full cross-entropy against it. A single label is the one-sample estimate, with $p^* = \delta_{y^*}$.