The Kelly Criterion from Shannon Information Theory

In 1956, John Kelly Jr. at Bell Labs published a remarkable paper connecting two apparently unrelated problems: the reliable transmission of information over noisy channels, and the optimal sizing of repeated bets. His central result was that a gambler with access to a noisy private wire to a racetrack can grow capital at an exponential rate equal to the Shannon mutual information between the source and the received signal. The optimal strategy, now called the Kelly criterion, prescribes exact fractions of wealth to wager on each outcome.

This post derives Kelly's result from first principles. We build the information-theoretic foundations (§1), formulate the gambler's optimization problem (§2), solve it via Karush-Kuhn-Tucker conditions (§3), prove the connection to mutual information (§4), specialize to binary prediction markets (§5), and discuss application to algorithmic trading (§6).

Conventions. All logarithms are base 2 unless otherwise noted, so that information is measured in bits. We write conditional probabilities as $p(r \mid s)$, $q(s \mid r)$, and use the geometric language of the probability simplex throughout. Kelly's 1956 paper uses a slash in place of the conditioning bar (e.g., $p(r/s)$ for $p(r \mid s)$); an appendix restates the key results in his original notation for readers following the paper directly.

1. Shannon's Communication Theory

Shannon's 1948 theory models communication as a probabilistic process. A source emits symbols from a fixed, finite set of possibilities: letters in a language, bits in a wire protocol, horses in a race. The restriction to finitely many symbols is not a limitation for our purposes. Every prediction market resolves to one of finitely many outcomes (typically just YES or NO), and every horse race has finitely many horses. The mathematical structure we need begins with this finite set and the space of probability distributions over it.

1.1. The probability simplex and entropy

Let $\mathcal{S} = \{1, 2, \ldots, S\}$ be a finite set of symbols (the alphabet). The space of all probability distributions on $\mathcal{S}$ is the probability simplex

an $(S{-}1)$-dimensional manifold with boundary, embedded in $\mathbb{R}^S$. Each point $p \in \Delta_{S-1}$ assigns probability $p(s)$ to symbol $s$.

Entropy quantifies uncertainty: $H(p) = 0$ when $p$ is a point mass (the outcome is deterministic), and $H(p)$ is maximized when $p$ is uniform. We prove this bound precisely as a corollary of the next result. For the binary case $S = 2$, the entropy is a function of a single parameter $p \in [0, 1]$, peaking at $H = 1$ bit when $p = 0.5$:

1.2. Kullback-Leibler divergence

Geometrically, $D_{\mathrm{KL}}$ is a Bregman divergence on $\Delta_{S-1}$ generated by the negative entropy functional $\Phi(p) = \sum_s p(s) \log p(s)$. It is not symmetric and does not satisfy the triangle inequality, so it is not a metric. It does, however, define a canonical notion of directed distance on the simplex. Its Hessian at a point $p$ recovers the Fisher information metric $g_{ij}(p) = \delta_{ij}/p(i)$, the unique (up to scale) Riemannian metric on the interior of $\Delta_{S-1}$ invariant under sufficient statistics ^[4].

Proof. Since $-\!\log$ is strictly convex, Jensen's inequality gives

Equality in Jensen's inequality holds if and only if $q(s)/p(s)$ is constant $p$-almost surely. Since both $p$ and $q$ sum to 1, the constant must be 1, forcing $p = q$. ∎

Proof. For the lower bound: each term $-p(s)\log p(s) \geq 0$ since $p(s) \in [0,1]$, with equality for all $s$ if and only if $p$ is a point mass.

For the upper bound: let $u = (1/S, \ldots, 1/S)$. By [gibbs],

so $H(p) \leq \log S$, with equality if and only if $p = u$. ∎

1.3. Communication channels

In geometric language, the channel $T$ induces a pushforward $q = T_* p$ on the output simplex $\Delta_{R-1}$, and the Bayesian posterior defines a map $\pi \colon \mathcal{R} \to \Delta_{S-1}$ by $\pi(r) = q(\cdot \mid r)$. The posterior map sends each received symbol $r$ to a point on the source simplex, encoding what the receiver has learned about $s$ after observing $r$.

1.4. Conditional entropy and mutual information

Proof. Starting from [eq:mi-eq] and using $q(s \mid r) = p(s,r)/q(r)$:

The first sum is $-H(S \mid R)$ by [eq:cond-entropy-eq]. For the second sum, marginalize over $r$:

Combining: $I(S;R) = -H(S \mid R) + H(S)$. ∎

Proof. From the proof of [mi-entropy]:

where we used $p(s,r) = q(r)\,q(s \mid r)$ to factor the outer sum. ∎

Proof. By [mi-kl], $I(S;R)$ is a convex combination of KL divergences. Each term satisfies $D_{\mathrm{KL}}(q(\cdot \mid r) \,\Vert\, p(\cdot)) \geq 0$ by [gibbs], so $I(S;R) \geq 0$. Equality holds if and only if $D_{\mathrm{KL}}(q(\cdot \mid r) \,\Vert\, p(\cdot)) = 0$ for all $r$ with $q(r) > 0$, which by [gibbs] requires $q(s \mid r) = p(s)$ for all $s, r$. This is precisely the condition that $S$ and $R$ are independent. ∎

The visualization below makes this concrete for a binary channel. The prior $p(s{=}1) = 0.6$ (gold circle) sits on the simplex $\Delta_1 = [0,1]$. As the channel noise $\varepsilon$ decreases, the two posteriors (blue dots, one per received symbol) spread further from the prior, and the mutual information grows.

2. The Gambler's Problem

Consider a sequence of independent horse races, each with $S$ horses. In each race, horse $s$ wins with probability $p(s)$. The track pays odds of $\alpha_s$ to 1 on horse $s$: a unit bet returns $\alpha_s$ if $s$ wins and zero otherwise.

Before each race, the gambler receives a symbol $r$ through a noisy channel with transition probabilities $p(r \mid s)$. After observing $r$, the gambler allocates a fraction $a(s \mid r) \geq 0$ of current capital to each horse $s$, retaining a holdback

If horse $s$ wins, the gambler's wealth is multiplied by $b_r + \alpha_s\, a(s \mid r)$.

By the strong law of large numbers applied to the i.i.d. sequence of log-returns, $(1/N)\log_2(W_N/W_0) \to G$ almost surely as $N \to \infty$. After $N$ races the gambler's wealth satisfies $W_N \approx W_0 \cdot 2^{NG}$, so maximizing $G$ maximizes the asymptotic exponential growth rate. The Kelly criterion is the strategy $\{a^*(s \mid r),\, b^*_r\}$ that maximizes $G$ subject to the budget constraint [eq:budget-eq] for each $r$.

For a binary bet ($p = 0.65$, market price $q = 0.50$), the growth rate $G(f)$ as a function of bet fraction is strictly concave, guaranteeing a unique optimum. The gold dot sweeps across all possible bet sizes. Betting too little wastes edge; betting too much inverts the growth rate, leading to ruin. The shaded region marks the zone where $G(f) > 0$.

3. Optimal Allocation

3.1. Decomposition per received symbol

Proof. Write $p(s,r) = q(r)\,q(s \mid r)$ and factor the outer sum over $r$. The budget constraint [eq:budget-eq] couples only variables sharing the same $r$, so the per-$r$ subproblems are independent. Since each $q(r) > 0$, the global maximum of $G$ is achieved by maximizing each $G_r$ separately. ∎

3.2. The Kelly theorem

By [decomposition], fix a received symbol $r$ and write $p(s) = q(s \mid r)$, $a(s) = a(s \mid r)$, $b = b_r$ for brevity. The subproblem is:

Substituting $b = 1 - \sum_t a(t)$ and differentiating with respect to $a(s)$, the chain rule gives

where $\delta_{s,s'}$ is the Kronecker delta (the $-1$ arises because increasing $a(s)$ decreases $b$). Therefore:

where the constant $k = \log e = \log_2 e \approx 1.4427$ arises from differentiating $\log_2$.

Proof. The growth rate $G$ is strictly concave in the $a(s)$ (it is a positively weighted sum of logarithms of affine functions), so the Karush-Kuhn-Tucker conditions are necessary and sufficient. Since $k = \log e > 0$, dividing [eq:partial-eq] by $k$ yields:

where $C = \sum_{s'} p(s')/(b + \alpha_{s'}\,a(s'))$ is independent of $s$.

Step 1 (Express allocations in terms of $C$). From the active-set condition [eq:active-eq]:

giving $a(s) = p(s)/C - b/\alpha_s$.

Step 2 (Determine $C$). Decompose the sum defining $C$ over the active and inactive sets:

so $C(1 - \sigma) = (1-p)/b$, equivalently $b = (1-p)/(C(1-\sigma))$.

Step 3 (Apply the budget constraint). From $b + \sum_{s \in \lambda} a(s) = 1$:

Substituting $b(1-\sigma) = (1-p)/C$:

so $C = 1$.

Step 4 (Final formulas). Substituting $C = 1$: $b^* = (1-p)/(1-\sigma)$ and $a^*(s) = p(s) - b^*/\alpha_s$ for $s \in \lambda$. The membership condition $a^*(s) > 0$ is equivalent to $p(s) > b^*/\alpha_s$, i.e., $p(s)\,\alpha_s > b^*$. ∎

4. Growth Rate as Mutual Information

Proof. With $\alpha_s = 1/p(s)$, the full sum $\sum_s 1/\alpha_s = \sum_s p(s) = 1$. For each received symbol $r$, consider the candidate strategy $b_r = 0$, $a(s \mid r) = q(s \mid r)$ for all $s$.

Feasibility. The budget constraint is satisfied: $b_r + \sum_s a(s \mid r) = 0 + \sum_s q(s \mid r) = 1$. Each $a(s \mid r) = q(s \mid r) \geq 0$.

Optimality. Evaluate the partial derivative [eq:partial-eq] at this candidate:

Since $G_r$ is concave and all partial derivatives vanish simultaneously, $a(s \mid r) = q(s \mid r)$ is the global maximizer.

Growth rate. The optimal per-symbol growth rate is:

Averaging over received symbols:

where the last equality is [mi-kl]. ∎

Geometrically, the optimal Kelly strategy maps each received symbol $r$ to the corresponding posterior $\pi(r)$ on the simplex $\Delta_{S-1}$, and the growth rate measures the expected KL divergence traversed from the prior $p$ to the posterior $\pi(r)$. The gambler's capital grows precisely because the private signal moves the posterior away from the prior, and this displacement, averaged over the output distribution, is the mutual information.

Proof. When $S$ and $R$ are independent, $q(s \mid r) = p(s)$ for all $r$, so $I(S;R) = 0$ by [mi-nonneg]. ∎

5. Binary Prediction Markets

We specialize to two outcomes: YES ($s = 1$) and NO ($s = 2$). The gambler believes $P(\text{YES}) = p$ while the market prices a YES contract at $q$, giving odds $\alpha_{\text{YES}} = 1/q$ and $\alpha_{\text{NO}} = 1/(1-q)$. These are fair odds since $1/\alpha_{\text{YES}} + 1/\alpha_{\text{NO}} = q + (1-q) = 1$.

Proof. Apply [kelly-optimal] with active set $\lambda = \{\text{YES}\}$, so $\sigma = 1/\alpha_{\text{YES}} = q$ and $p_\lambda = p$. Then

For the growth rate, the wealth multiplier when YES wins is $b^* + \alpha_{\text{YES}} f^* = (1-p)/(1-q) + (p-q)/(q(1-q))$. Combining over a common denominator:

When NO wins, the multiplier is $b^* + 0 = (1-p)/(1-q)$. Therefore:

This is the KL divergence from the market distribution $(q, 1{-}q)$ to the gambler's true distribution $(p, 1{-}p)$. ∎

The visualization below shows $G(f)$ for even-money odds ($q = 0.5$) as the true probability $p$ sweeps from 0.52 to 0.80. The gold dot marks the optimal fraction $f^* = (p - q)/(1 - q)$, and the faded traces show snapshots at intermediate values of $p$.

The next figure illustrates the binary Kelly growth rate as a KL divergence on the simplex $\Delta_1 = [0, 1]$. The top panel shows the market price $q$ (hollow) and the gambler's true probability $p$ (gold) on the simplex. The bottom panel plots $D_{\mathrm{KL}}((p, 1{-}p) \,\Vert\, (q, 1{-}q))$ as a function of $p$.

Finally, we simulate the long-run behavior. Four wealth trajectories are drawn for each of three strategies: full Kelly (gold), half Kelly (blue), and double Kelly (coral). Full Kelly maximizes asymptotic growth but with high variance. Half Kelly sacrifices some growth for smoother paths. Double Kelly overbets, producing frequent ruin despite the positive edge. Dashed lines show the theoretical expected growth rate $g \cdot N$.

6. Application to Algorithmic Trading

The Kelly framework extends naturally from prediction markets to algorithmic trading systems, though several practical modifications are essential.

6.1. From discrete bets to continuous sizing

In practice, a trading system faces a stream of opportunities rather than a single repeated bet. Each opportunity $i$ has an estimated edge $\hat{p}_i$ and market-implied probability $q_i$. The Kelly fraction for opportunity $i$ is $f^*_i = (\hat{p}_i - q_i)/(1 - q_i)$, and the capital allocated is $f^*_i \cdot W$ where $W$ is current wealth.

Two complications arise immediately. First, the true probability $p$ is never known exactly. The gambler works with an estimate $\hat{p}$ that carries both systematic bias and estimation variance. Second, multiple positions may be open simultaneously, introducing correlation structure that the single-bet Kelly formula ignores.

6.2. Fractional Kelly and estimation error

When the estimate $\hat{p}$ is noisy, full Kelly is too aggressive. To see why, consider the expected growth rate under misspecification. If the true probability is $p$ but the gambler bets as though it were $\hat{p}$, the realized growth rate is

which is strictly less than $G^*$ whenever $\hat{p} \neq p$. The loss from overestimating edge is asymmetrically worse than the loss from underestimating it, because the logarithm penalizes losses more than it rewards gains. This asymmetry motivates fractional Kelly: bet a fraction $\kappa \in (0, 1)$ of the full Kelly amount. Common choices are $\kappa = 0.5$ (half Kelly) or $\kappa = 0.25$ (quarter Kelly).

The fractional Kelly strategy trades expected growth for reduced variance. With fraction $\kappa$, the growth rate becomes approximately $G(\kappa) \approx \kappa G^* - \kappa^2 G^* / 2$, achieving $75\%$ of maximum growth at half Kelly while cutting the variance of log-returns in half ^[5].

6.3. Portfolio Kelly and correlation

When multiple positions are open simultaneously, the correct generalization replaces the scalar Kelly fraction with a vector problem. Given $n$ simultaneous opportunities with mean excess return vector $\mu$ and covariance matrix $\Sigma$, the growth-rate-maximizing portfolio weight vector is

the familiar Markowitz mean-variance portfolio, which turns out to be the continuous-time limit of the Kelly criterion ^[6]. For a trading system running multiple concurrent positions on Polymarket or similar venues, this means accounting for correlation between outcomes. Correlated positions effectively concentrate risk, requiring smaller individual allocations than the independent-bet formula suggests.

6.4. Practical guidelines

For an algorithmic system implementing Kelly sizing:

1. Estimate conservatively. Calibrate $\hat{p}$ on out-of-sample data using proper scoring rules (Brier score, log score). Prefer models that are well-calibrated over those that maximize accuracy.

2. Use fractional Kelly. Start with $\kappa = 0.25$ and increase only as confidence in the probability estimates grows. The growth rate curve is flat near the optimum (see Figure 1), so undersizing costs far less than oversizing.

3. Respect the budget constraint. The total allocation across all active bets must satisfy $\sum_i f_i \leq 1$. When many opportunities appear simultaneously, the constraint forces prioritization by edge quality $p_i \alpha_i$, exactly as [active-set-remark] describes.

4. Rebalance on new information. Each new observation updates the posterior $q(s \mid r)$, changing the Kelly fraction. The capital path is not set-and-forget but adapts continuously as the information channel evolves.

5. Account for fees and slippage. Transaction costs reduce the effective odds $\alpha_s$. If the market charges a fee $\phi$, replace $\alpha_s$ with $\alpha_s(1 - \phi)$ in all formulas. Small edges can be entirely consumed by fees, leaving no profitable bet.

7. Conclusion

Kelly's 1956 result reveals a deep structural identity: the maximum rate at which a gambler can grow capital through repeated wagers is exactly the mutual information $I(S;R)$ between the source of uncertainty and the signal available to the bettor. This connection is not a loose analogy. It is a theorem, derived here from the KKT conditions of a constrained concave optimization and the divergence decomposition of mutual information.

The proof passes through three stages. First, the growth rate $G$ decomposes into independent subproblems, one per received symbol. Second, solving each subproblem via the active-set conditions yields the closed-form allocation $a^*(s) = q(s \mid r) - b^*/\alpha_s$. Third, under fair odds, the optimal growth rate for each received symbol reduces to a KL divergence from the prior to the posterior, and averaging recovers the mutual information.

For binary prediction markets, the result simplifies further: the Kelly fraction is $f^* = (p - q)/(1 - q)$ and the growth rate is the KL divergence $D_{\mathrm{KL}}((p, 1{-}p) \,\Vert\, (q, 1{-}q))$. No betting system can beat this rate, and no positive rate is achievable without an informational edge. These are not heuristics but consequences of the concavity of the logarithm and the geometry of the probability simplex.

In practice, uncertainty in the probability estimates, correlated positions, and transaction costs all require modifications. Fractional Kelly provides a principled way to trade growth for robustness, and the portfolio generalization handles simultaneous bets. The core message endures: information is capital, and the rate of capital growth is bounded by the rate of information.

Appendix: Kelly's Original Slash Notation

References

1. Kelly, J. L. Jr. (1956). A New Interpretation of Information Rate. Bell System Technical Journal, 35(4), 917–926. DOI
2. Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423. DOI
3. Cover, T. M. and Thomas, J. A. (2006). Elements of Information Theory, 2nd ed. Wiley.
4. Amari, S. and Nagaoka, H. (2000). Methods of Information Geometry. AMS.
5. MacLean, L. C., Thorp, E. O., and Ziemba, W. T. (2011). The Kelly Capital Growth Investment Criterion. World Scientific. DOI
6. Thorp, E. O. (2006). The Kelly Criterion in Blackjack, Sports Betting, and the Stock Market. In Handbook of Asset and Liability Management, Vol. 1. DOI