Search
View source code
Display the source code in std/numeric.d from which this page was generated on github.
Report a bug
Quickly fork, edit online, and submit a pull request for this page. Requires a signed-in GitHub account. This works well for small changes. If you'd like to make larger changes you may want to consider using local clone.

# Function `std.numeric.kullbackLeiblerDivergence`

Computes the Kullback-Leibler divergence between input ranges `a` and `b`, which is the sum `ai * log(ai / bi)`. The base of logarithm is 2. The ranges are assumed to contain elements in `[0, 1]`. Usually the ranges are normalized probability distributions, but this is not required or checked by `kullbackLeiblerDivergence`. If any element `bi` is zero and the corresponding element `ai` nonzero, returns infinity. (Otherwise, if `ai == 0 && bi == 0`, the term `ai * log(ai / bi)` is considered zero.) If the inputs are normalized, the result is positive.

``` CommonType!(ElementType!Range1,ElementType!Range2) kullbackLeiblerDivergence(Range1, Range2) (   Range1 a,   Range2 b ) if (isInputRange!Range1 && isInputRange!Range2); ```

## Example

``````import std.math : approxEqual;

double[] p = [ 0.0, 0, 0, 1 ];
writeln(kullbackLeiblerDivergence(p, p)); // 0
double[] p1 = [ 0.25, 0.25, 0.25, 0.25 ];
writeln(kullbackLeiblerDivergence(p1, p1)); // 0
writeln(kullbackLeiblerDivergence(p, p1)); // 2
writeln(kullbackLeiblerDivergence(p1, p)); // double.infinity
double[] p2 = [ 0.2, 0.2, 0.2, 0.4 ];
assert(approxEqual(kullbackLeiblerDivergence(p1, p2), 0.0719281));
assert(approxEqual(kullbackLeiblerDivergence(p2, p1), 0.0780719));
``````

## Authors

Andrei Alexandrescu, Don Clugston, Robert Jacques, Ilya Yaroshenko