---
title: "Introduction to RenyiExtropy"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Introduction to RenyiExtropy}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment  = "#>"
)
library(RenyiExtropy)
```

## Overview

**RenyiExtropy** provides a unified interface for computing entropy and
extropy measures for discrete probability distributions.  All functions
accept a probability vector (or matrix for joint measures) and return a
single numeric value measured in **nats** (natural-logarithm base).

The package covers three families of measures:

* **Entropy** — Shannon, Renyi, Tsallis, normalised, joint, conditional
* **Extropy** — classical (Shannon), Renyi, conditional Renyi, maximum Renyi
* **Divergences** — KL, Jensen-Shannon, cross-entropy

---

## Probability vectors

Every function that accepts a vector `p` requires it to satisfy:

* numeric, no `NA`/`NaN`
* all elements in \[0, 1\]
* elements sum to 1 (within tolerance 1e-8)
* length ≥ 2

```{r prob-vector}
p <- c(0.2, 0.5, 0.3)   # valid 3-outcome distribution
```

---

## Shannon entropy

The Shannon entropy is defined as
$$H(\mathbf{p}) = -\sum_{i=1}^n p_i \log p_i$$
and equals zero for a degenerate distribution and $\log n$ for the
uniform distribution.

```{r shannon}
shannon_entropy(p)               # three-outcome distribution
shannon_entropy(rep(0.25, 4))    # uniform: H = log(4)
shannon_entropy(c(1, 0, 0))      # degenerate: H = 0
normalized_entropy(p)            # H(p) / log(n), always in [0, 1]
```

---

## Renyi entropy

The Renyi entropy of order $q > 0$ is
$$H_q(\mathbf{p}) = \frac{1}{1-q} \log\!\left(\sum_i p_i^q\right)$$

For $q \to 1$ it reduces to the Shannon entropy; the function
automatically returns the Shannon limit when $|q - 1| < 10^{-8}$.

```{r renyi-entropy}
renyi_entropy(p, q = 2)         # collision entropy
renyi_entropy(p, q = 0.5)       # Renyi entropy, q = 0.5
renyi_entropy(p, q = 1)         # limit: equals shannon_entropy(p)
shannon_entropy(p)               # same value
```

---

## Tsallis entropy

The Tsallis entropy is
$$S_q(\mathbf{p}) = \frac{1 - \sum_i p_i^q}{q - 1}$$
a non-extensive generalisation of Shannon entropy widely used in
statistical physics.

```{r tsallis}
tsallis_entropy(p, q = 2)
tsallis_entropy(p, q = 1)       # limit: equals shannon_entropy(p)
```

---

## Extropy

Extropy is the dual complement of entropy, measuring information via
complementary probabilities:
$$J(\mathbf{p}) = -\sum_{i=1}^n (1-p_i)\log(1-p_i)$$

`extropy()` and `shannon_extropy()` compute the same quantity.

```{r extropy}
extropy(p)
shannon_extropy(p)               # identical
```

---

## Renyi extropy

The Renyi extropy of order $q$ is
$$J_q(\mathbf{p}) = \frac{-(n-1)\log(n-1) + (n-1)\log\!\left(\sum_i(1-p_i)^q\right)}{1-q}$$

For $n = 2$ it equals the Renyi entropy.  For $q \to 1$ it returns the
classical extropy.

```{r renyi-extropy}
renyi_extropy(p, q = 2)
renyi_extropy(p, q = 1)          # limit: equals extropy(p)

# n = 2: Renyi extropy == Renyi entropy
renyi_extropy(c(0.4, 0.6), q = 2)
renyi_entropy(c(0.4, 0.6), q = 2)

# Maximum Renyi extropy over n outcomes (uniform distribution)
max_renyi_extropy(3)
renyi_extropy(rep(1/3, 3), q = 2)
```

---

## Joint and conditional entropy

`joint_entropy()` accepts a matrix of joint probabilities; rows
correspond to outcomes of $X$, columns to outcomes of $Y$.

```{r joint}
Pxy <- matrix(c(0.2, 0.3, 0.1, 0.4), nrow = 2, byrow = TRUE)

joint_entropy(Pxy)               # H(X, Y)
conditional_entropy(Pxy)         # H(Y | X) = H(X,Y) - H(X)

# Independent distributions: H(X,Y) = H(X) + H(Y)
px <- c(0.4, 0.6)
py <- c(0.3, 0.7)
Pxy_indep <- outer(px, py)
joint_entropy(Pxy_indep)
shannon_entropy(px) + shannon_entropy(py)
```

---

## Conditional Renyi extropy

```{r cond-renyi}
conditional_renyi_extropy(Pxy, q = 2)
conditional_renyi_extropy(Pxy, q = 1)   # limit: conditional Shannon extropy
```

---

## Divergences

### KL divergence

The Kullback-Leibler divergence $D_\text{KL}(P \| Q)$ is asymmetric
and can be infinite when $q_i = 0$ but $p_i > 0$ (a warning is emitted
in that case).

```{r kl}
q_dist <- c(0.3, 0.4, 0.3)
kl_divergence(p, q_dist)         # KL(P || Q)
kl_divergence(q_dist, p)         # KL(Q || P) -- asymmetric
kl_divergence(p, p)              # 0
```

### Jensen-Shannon divergence

The Jensen-Shannon divergence is the symmetrised, always-finite version,
bounded in $[0, \log 2]$:

```{r js}
js_divergence(p, q_dist)         # symmetric
js_divergence(q_dist, p)         # same value
js_divergence(p, p)              # 0
js_divergence(c(1, 0), c(0, 1)) # maximum: log(2)
log(2)
```

### Cross-entropy

Cross-entropy relates to KL divergence via
$H(P, Q) = H(P) + D_\text{KL}(P \| Q)$:

```{r cross}
cross_entropy(p, q_dist)
shannon_entropy(p) + kl_divergence(p, q_dist)   # same value
cross_entropy(p, p)                              # equals shannon_entropy(p)
```

---

## Input validation

All functions produce informative errors when given invalid inputs:

```{r validation, error = TRUE}
shannon_entropy(c(0.2, 0.3, 0.1))   # does not sum to 1
renyi_entropy(p, q = NA)            # NA not allowed
max_renyi_extropy(1)                # n must be >= 2
kl_divergence(p, c(0.5, 0.5))      # length mismatch
```