Kelly Criterion from Shannon Information Theory
In 1956, John Kelly Jr. at Bell Labs published a paper connecting two apparently unrelated problems: the reliable transmission of information over noisy channels, and the optimal sizing of repeated bets. His central result was that a gambler with access to a noisy private wire to a racetrack who bets at odds consistent with the true probabilities can grow capital at an exponential rate equal to the Shannon mutual information between the source and the received signal. The optimal strategy is now called the Kelly criterion and prescribes exact fractions of wealth to wager on each outcome.
This post derives Kelly's result from first principles. We build the information-theoretic foundations (§1), formulate the gambler's optimisation problem (§2), solve it via Karush-Kuhn-Tucker conditions (§3), prove the connection to mutual information (§4), specialise to binary prediction markets (§5), and discuss application to algorithmic trading (§6).
Conventions. All logarithms are base 2 unless otherwise noted. Information is measured in bits. We write conditional probabilities as and and use the geometric language of the probability simplex throughout. Kelly's 1956 paper uses a slash in place of the conditioning bar (e.g., for ); an appendix restates the key results in his original notation for readers following the paper directly.
1. Shannon's Communication Theory
Shannon's 1948 theory models communication as a probabilistic process. A source emits symbols from a fixed, finite set of possibilities: letters in a language, bits in a wire protocol, horses in a race. The restriction to finitely many symbols loses nothing here. Every prediction market resolves to one of finitely many outcomes (typically YES or NO) and every horse race has finitely many horses. The mathematical structure we need begins with this finite set and the space of probability distributions over it.
1.1. Probability simplex and entropy
Let be a finite set of symbols (the alphabet). The space of all probability distributions on is the probability simplex
an -dimensional manifold with boundary, embedded in . Each point assigns probability to symbol .
The entropy of a distribution is
with the convention (justified by continuity, since ).
Entropy quantifies uncertainty: when is a point mass (the outcome is deterministic), and is maximised when is uniform. We prove this bound as a corollary of the next result. For the binary case the entropy is a function of a single parameter and peaks at bit when :
1.2. Kullback-Leibler divergence
For distributions with absolutely continuous with respect to (i.e., ), the KL divergence from to is
Geometrically is a Bregman divergence on generated by the negative entropy functional . It is asymmetric and fails the triangle inequality. Hence it is a directed distance on the simplex. Its Hessian at a point recovers the Fisher information metric , the unique (up to scale) Riemannian metric on the interior of invariant under sufficient statistics [4].
For any distributions ,
with equality if and only if .
Proof. Since is strictly convex, Jensen's inequality gives
Equality in Jensen's inequality holds if and only if is constant -almost surely, and equality in the last step holds if and only if vanishes off the support of . Together these force with , that is . ∎
For any ,
The lower bound is achieved if and only if is a point mass. The upper bound is achieved if and only if is the uniform distribution .
Proof. For the lower bound: each term since , with equality for all if and only if is a point mass.
For the upper bound: let . By [gibbs],
so , with equality if and only if . ∎
1.3. Communication channels
A discrete memoryless channel consists of an input alphabet , an output alphabet , and a stochastic map defined by the transition probabilities
Given a source distribution , the joint distribution, output marginal, and posterior are:
In geometric language the channel induces a pushforward on the output simplex , and the Bayesian posterior defines a map by . The posterior map sends each received symbol to a point on the source simplex that encodes what the receiver has learned about after observing .
1.4. Conditional entropy and mutual information
The conditional entropy of given is
where is the entropy of the posterior distribution .
The mutual information between and is
Proof. Starting from [eq:mi-eq] and using :
The first sum is by [eq:cond-entropy-eq]. For the second sum, marginalise over :
Combining: . ∎
Proof. From the proof of [mi-entropy]:
where we used to factor the outer sum. ∎
, with equality if and only if and are independent.
Proof. By [mi-kl], is a convex combination of KL divergences. Each term satisfies by [gibbs], so . Equality holds if and only if for all with , which by [gibbs] requires for all . This is the condition that and are independent. ∎
On the simplex , the posterior map traces out a cloud of points around the prior . The mutual information is the average KL divergence from to these posterior points, weighted by the output marginal. A noiseless channel sends to a vertex of the simplex (a point mass) and the divergence reaches its maximum . A completely noisy channel satisfies for all and the divergence is zero. The mutual information measures how far on average the posterior moves from the prior.
The visualisation below shows this for a binary channel. The prior (gold circle) sits on the simplex . As the channel noise decreases, the two posteriors (blue dots, one per received symbol) spread further from the prior and the mutual information grows.
2. Gambler's Problem
Consider a sequence of independent horse races, each with horses. In each race, horse wins with probability . The track pays odds of for 1 on horse : a unit bet returns if wins and zero otherwise.
Before each race, the gambler receives a symbol through a noisy channel with transition probabilities . After observing , the gambler allocates a fraction of current capital to each horse and retains a holdback
If horse wins the gambler's wealth is multiplied by .
The exponential growth rate of the gambler's capital is the expected log-return per race:
By the strong law of large numbers applied to the i.i.d. sequence of log-returns, almost surely as . After races the gambler's wealth satisfies . Hence maximising maximises the asymptotic exponential growth rate. The Kelly criterion is the strategy that maximises subject to the budget constraint [eq:budget-eq] for each .
For a binary bet (, market price ) the growth rate as a function of bet fraction is strictly concave and so has a unique optimum. The gold dot sweeps across all possible bet sizes. Betting too little wastes edge; betting too much inverts the growth rate and leads to ruin. The shaded region marks the zone where .
3. Optimal Allocation
3.1. Decomposition per received symbol
The growth rate decomposes as , where
Since and the variables for distinct values of are disjoint, maximising is equivalent to independently maximising each .
Proof. Write and factor the outer sum over . The budget constraint [eq:budget-eq] couples only variables sharing the same , so the per- subproblems are independent. Since each , the global maximum of is achieved by maximising each separately. ∎
3.2. Kelly theorem
By [decomposition] fix a received symbol and write , , for brevity. The subproblem is:
Substituting and differentiating with respect to , the chain rule gives
where is the Kronecker delta (the arises because increasing decreases ). Therefore:
where the constant arises from differentiating .
Let be the probability that outcome occurs and the odds. Suppose an optimal allocation keeps a positive holdback . This is automatic when the track takes a cut, . Define the active set and its complement . Write
A growth-rate-maximising allocation is:
An outcome belongs to the active set if and only if its expected return exceeds the holdback: . The allocation is unique when . When the strategy , is optimal. Under fair odds it is not unique: a holdback pays the same as bets on every outcome.
Proof. The growth rate is a positively weighted sum of logarithms of the affine functions . Hence it is concave in the and the Karush-Kuhn-Tucker conditions are necessary and sufficient. It is strictly concave when the affine map from to the payoff vector is injective. That map has determinant . Hence the maximiser is unique when . With the constraint is inactive. Since , dividing [eq:partial-eq] by yields:
where is independent of .
Step 1 (Allocations in terms of ). From the active-set condition [eq:active-eq]:
Hence .
Step 2 (Value of ). Decompose the sum defining over the active and inactive sets:
Hence or equivalently .
Step 3 (Allocation constraint). From :
Substituting :
Hence .
Step 4 (Final formulas). Substituting : and for . The membership condition is equivalent to , i.e., . ∎
With side information, [decomposition] reduces the problem to one instance of [kelly-optimal] per received symbol , with in place of . The optimal strategy is and for , where and .
The active set and the holdback are mutually dependent: while depends on through and . In practice sort outcomes by in decreasing order and add them to one at a time, recomputing at each step, until the membership condition fails.
4. Growth Rate as Mutual Information
The odds are fair when , so that the reciprocal odds form a probability distribution on outcomes. In particular gives true fair odds reflecting the source distribution.
Under true fair odds , the maximum growth rate with side information equals the mutual information:
Proof. With , the full sum . For each received symbol , consider the candidate strategy , for all .
Feasibility. The budget constraint is satisfied: . Each .
Optimality. Evaluate the partial derivative [eq:partial-eq] at this candidate:
Since is concave and all partial derivatives vanish simultaneously, is a global maximiser. Other maximisers exchange part of the bets for a holdback and give the same payoffs.
Growth rate. The optimal per-symbol growth rate is:
Averaging over received symbols:
where the last equality is [mi-kl]. ∎
Geometrically the optimal Kelly strategy maps each received symbol to the corresponding posterior on the simplex and the growth rate is the expected KL divergence from the prior to the posterior . The gambler's capital grows because the private signal moves the posterior away from the prior. The average of this displacement over the output distribution is the mutual information.
Without side information (i.e., when and are independent), the maximum growth rate under true fair odds is . No betting strategy can achieve positive long-run growth against odds that match the true probabilities without an informational edge.
Proof. When and are independent, for all , so by [mi-nonneg]. ∎
5. Binary Prediction Markets
We specialise to two outcomes: YES () and NO (). Let the gambler's belief be and let the market price a YES contract at . The odds are and . These are fair odds since .
For a binary prediction market with true probability and market price where (positive edge on YES) the Kelly-optimal fraction of capital to bet on YES is
and the resulting growth rate is
Proof. Under these fair odds the optimum is not unique. We take the one with no stake on NO. Apply the KKT conditions of [kelly-optimal] with active set . They hold with , the NO condition with equality. Here and . Then
For the growth rate, the wealth multiplier when YES wins is . Combining over a common denominator:
When NO wins, the multiplier is . Therefore:
This is the KL divergence from the market distribution to the gambler's true distribution . ∎
When the edge is on NO. By symmetry the optimal fraction to bet on NO is , and the growth rate is again . In either direction the Kelly growth rate for a binary prediction market is the KL divergence between the gambler's beliefs and the market's implied probabilities.
The visualisation below shows for even-money odds () as the true probability sweeps from 0.52 to 0.80. The gold dot marks the optimal fraction , and the faded traces show snapshots at intermediate values of .
The next figure illustrates the binary Kelly growth rate as a KL divergence on the simplex . The top panel shows the market price (hollow) and the gambler's true probability (gold) on the simplex. The bottom panel plots as a function of .
Finally we simulate the long-run behaviour. Four wealth trajectories are drawn for each of three strategies: full Kelly (gold), half Kelly (blue) and double Kelly (coral). Each path is 200 bets at , . Full Kelly maximises asymptotic growth with high variance. Half Kelly gives up some growth for smoother paths. Double Kelly overbets: its growth rate is slightly negative despite the positive edge, and its paths often end below the starting wealth. Dashed lines show the expected log-wealth .
6. Application to Algorithmic Trading
The Kelly framework extends from prediction markets to algorithmic trading systems with several practical modifications.
6.1. Discrete bets and continuous sizing
In practice a trading system faces a stream of opportunities. Each opportunity has an estimated edge and market-implied probability . The Kelly fraction for opportunity is , and the capital allocated is where is current wealth.
Two complications arise immediately. First the true probability is never known exactly. The gambler works with an estimate that has both systematic bias and estimation variance. Second multiple positions may be open simultaneously and introduce correlation structure that the single-bet Kelly formula ignores.
6.2. Fractional Kelly and estimation error
When the estimate is noisy full Kelly is too aggressive. Consider the expected growth rate under misspecification. If the true probability is but the gambler bets as though it were , the realised growth rate is
By [gibbs] it is strictly less than whenever . The loss from overestimating edge is asymmetrically worse than the loss from underestimating it because the logarithm penalises losses more than it rewards gains. This asymmetry motivates fractional Kelly: bet a fraction of the full Kelly amount. Common choices are (half Kelly) or (quarter Kelly).
The fractional Kelly strategy gives up expected growth for reduced variance. With fraction the growth rate is to second order . Half Kelly achieves about of maximum growth with a quarter of the variance of log-returns [5].
6.3. Portfolio Kelly and correlation
When multiple positions are open simultaneously the correct generalisation replaces the scalar Kelly fraction with a vector problem. Given simultaneous opportunities with mean excess return vector and covariance matrix , the growth-rate-maximising portfolio weight vector is to second order in the weights (exactly in the continuous-time limit)
It is the Markowitz mean-variance portfolio with unit risk aversion and the continuous-time form of the Kelly criterion [6]. A trading system running multiple concurrent positions on Polymarket or similar venues must account for correlation between outcomes. Correlated positions concentrate risk and require smaller individual allocations than the independent-bet formula suggests.
6.4. Practical guidelines
For an algorithmic system implementing Kelly sizing:
-
Conservative estimation. Calibrate on out-of-sample data using proper scoring rules (Brier score, log score). Prefer well-calibrated models to accuracy-maximising ones.
-
Fractional Kelly. Start with and increase only as confidence in the probability estimates grows. The growth rate curve is flat near the optimum (see the figure in §2). A moderate undersize loses little growth. Oversizing loses more: growth falls to about zero at twice the Kelly fraction.
-
Allocation constraint. The total allocation across all active bets must satisfy . When many opportunities appear simultaneously the constraint forces prioritisation by edge quality as in [active-set-remark].
-
Rebalancing on new information. Each new observation updates the posterior and changes the Kelly fraction.
-
Fees and slippage. Transaction costs reduce the effective odds . A fee charged as a fraction of the gross amount returned on a winning bet (stake included) replaces with in all formulas. A parimutuel track take has this form. Other fee models give other effective odds: a commission on net winnings gives , and a fee per unit staked and paid on top of the stake gives . Fees can consume a small edge entirely and leave no profitable bet.
7. Conclusion
Kelly's 1956 result is a structural identity: at odds matching the true probabilities the maximum rate at which a gambler can grow capital through repeated wagers is the mutual information between the source of uncertainty and the signal available to the bettor. The result is derived here from the KKT conditions of a constrained concave optimisation and the divergence decomposition of mutual information.
The growth rate decomposes into independent subproblems, one per received symbol. The active-set conditions give the closed-form allocation . Under true fair odds the optimal growth rate for each received symbol is a KL divergence from the prior to the posterior and averaging recovers the mutual information.
For binary prediction markets the result simplifies further: the Kelly fraction is and the growth rate is the KL divergence . No betting system can beat this rate, and no positive rate is achievable without an informational edge.
In practice uncertainty in the probability estimates, correlated positions and transaction costs all require modifications. Fractional Kelly gives up growth for lower variance and the portfolio generalisation handles simultaneous bets. Information is capital and the rate of capital growth is bounded by the rate of information.
Kelly's Original Slash Notation
Expand to see the key results restated in Kelly's 1956 notation
Kelly's paper writes conditional probabilities with a slash: in place of , and in place of . The full notation dictionary is:
| This post | Kelly (1956) | Meaning |
|---|---|---|
| Source probability | ||
| Channel transition | ||
| Output marginal | ||
| Posterior | ||
| Joint probability | ||
| Bet fraction | ||
| Source entropy | ||
| Conditional entropy |
Kelly numbers only three displays: (1) the growth rate without holdback, (2) the growth rate with holdback and (3) its single-channel form . Page numbers below are those of the Bell System Technical Journal printing.
Growth rate (Kelly's Eq. 2, p. 923). In slash notation:
Allocation constraint. For each received symbol :
Optimal allocation (Kelly, p. 924). Writing for the posterior, the KKT conditions yield:
Proof sketch in slash notation. Kelly (p. 924) states the stationarity conditions for Eq. 3 with a multiplier : for , , and for . He finds . Eliminating through the constraint gives the form used in this post:
with . Setting the active-set equations to zero:
From the active-set condition, , giving . Summing the definition of over and separately yields . The budget constraint then gives and hence .
Kelly's fundamental result (Kelly, p. 922). Under fair odds , the optimal strategy is , , and the growth rate is:
Kelly writes the result as , where is the conditional entropy in slash notation. In the notation of this post it reads .
References
- Kelly, J. L. Jr. (1956). A New Interpretation of Information Rate. Bell System Technical Journal, 35(4), 917–926. DOI
- Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423. DOI
- Cover, T. M. and Thomas, J. A. (2006). Elements of Information Theory, 2nd ed. Wiley.
- Amari, S. and Nagaoka, H. (2000). Methods of Information Geometry. AMS.
- MacLean, L. C., Thorp, E. O., and Ziemba, W. T. (2011). The Kelly Capital Growth Investment Criterion. World Scientific. DOI
- Thorp, E. O. (2006). The Kelly Criterion in Blackjack, Sports Betting, and the Stock Market. In Handbook of Asset and Liability Management, Vol. 1. DOI