---
title: "Uncertainty in Biodiversity Indicators"
author: "Shawn Dove"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Uncertainty in Biodiversity Indicators}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.width = 8,
  fig.height = 6,
  out.width = "95%"
)
```

## Introduction 🛡️

Quantifying uncertainty is crucial for robust biodiversity assessments. The **b3gbi** package provides a flexible and powerful way to calculate confidence intervals (CIs) for biodiversity indicators using bootstrapping methods, leveraging the [**dubicube**](https://github.com/b-cubed-eu/dubicube) package for robust cube-level resampling.

**dubicube** is developed alongside **b3gbi** within the B-Cubed project and is installed from R-universe:

```{r install-dubicube, eval = FALSE}
install.packages("dubicube",
                 repos = c("https://b-cubed-eu.r-universe.dev",
                           "https://cloud.r-project.org"))
```

If **dubicube** is not installed, `add_ci()` falls back to indicator-level bootstrapping (see below) and tells you so.

To maintain a clean and efficient workflow, uncertainty calculation is decoupled from the initial indicator calculation. This "two-step" process allows users to first explore their data and indicators quickly, and then perform computationally intensive bootstrapping only when needed.

## The Two-Step Workflow

The standard workflow for adding uncertainty to an indicator is as follows:

1.  **Calculate the Indicator**: Use a time-series wrapper function (e.g., `total_occ_ts()`, `pielou_evenness_ts()`).
2.  **Add Confidence Intervals**: Pass the resulting indicator object to the `add_ci()` function.

### Example: Total Occurrences with Uncertainty

In this example, we calculate total occurrences for mammals in Denmark and then add 95% confidence intervals using cube-level bootstrapping.

```{r example, eval = FALSE}
library(b3gbi)

# 1. Process the cube
denmark_cube <- process_cube(system.file("extdata", 
                                         "denmark_mammals_cube_eqdgc.csv", 
                                         package = "b3gbi"))

# 2. Calculate a time series of total occurrences
# By default, CIs are NOT calculated during this step
occ_ts <- total_occ_ts(denmark_cube)

# 3. Add confidence intervals using add_ci()
# With dubicube installed, this uses cube-level bootstrapping by default
occ_ts_with_ci <- add_ci(occ_ts, num_bootstrap = 100) # Using 100 for speed in this example

# 4. Plot the result
plot(occ_ts_with_ci, title = "Total Occurrences with 95% CI")
```

## Bootstrapping Levels

The `add_ci()` function supports two levels of bootstrapping, selectable via the `bootstrap_level` argument. The default, `bootstrap_level = "auto"`, uses cube-level bootstrapping when **dubicube** is installed and indicator-level bootstrapping otherwise. The level that was used is stored in the `ci_method` element of the result and shown when it is printed.

### 1. Cube-Level Bootstrapping (`bootstrap_level = "cube"`)

This is the **recommended method** (and the default when **dubicube** is installed). It resamples the raw occurrence records within the data cube. The function automatically determines whether to use group-specific resampling (for total occurrences, where each year is resampled separately) or whole-cube resampling (for all other indicators, including the species-level ones) based on the indicator type.

*   **Pros**: Mathematically more robust; captures the underlying sampling uncertainty of the original data.
*   **Cons**: Computationally more intensive, especially for large cubes.

It leverages the `dubicube` package to ensure that indicators are recalculated correctly for each bootstrap replicate.

### 2. Indicator-Level Bootstrapping (`bootstrap_level = "indicator"`)

This method resamples, within each year, the values the indicator is computed from (occurrence records, species, or grid cells) using the **boot** package, instead of recalculating the indicator on a resampled cube.

*   **Pros**: Very fast, even for massive datasets.
*   **Cons**: Less robust; treats these components as independent and identically distributed, which is often not strictly true for biodiversity data.

## Automatic Indicator Configuration

The `add_ci()` function automatically applies appropriate bootstrapping strategies based on the indicator type. This ensures statistically correct confidence intervals without requiring manual configuration:

*   **Group-Specific Bootstrapping**: This method isolates resampling strictly within each grouping variable. For indicators like Total Occurrences (`total_occ`), which calculate a raw count per year, resampling only within that year provides mathematically correct variance without cross-contamination between years.

*   **Whole-Cube Bootstrapping**: This method resamples the entire dataset at once. This is automatically applied to aggregate indicators (evenness, rarity, density metrics) as well as species-level indicators (`spec_occ`, `spec_range`). For species-level indicators, whole-cube resampling is computationally optimized to process thousands of species simultaneously in a single pass while correctly capturing changes in species composition during resampling.

*   **Bounded Indicators**: Evenness metrics (`pielou_evenness`, `williams_evenness`) automatically use logit transformation to ensure confidence intervals remain within valid [0, 1] bounds.

These defaults are applied internally (at cube level), but can be overridden using `boot_args` or `ci_args` (see below) if needed.

## Advanced Configuration

The `add_ci()` function provides several arguments for fine-tuning the CI calculation:

| Argument | Description | Default |
|----------|-------------|---------|
| `num_bootstrap` | Number of bootstrap replicates. | `1000` |
| `ci_type` | Type of bootstrap interval. See [Types of Confidence Intervals](#types-of-confidence-intervals) below. | `"perc"` |
| `confidence_level` | The confidence level (e.g., 0.95). | `0.95` |
| `seed` | Random seed for bootstrapping (the session's random number state is restored afterwards). | `123` |
| `trans` / `inv_trans` | Transformation applied before calculating the intervals, and its inverse. | identity |
| `boot_args` | A list of additional arguments for `dubicube::bootstrap_cube()` (cube level only). | `list()` |
| `ci_args` | A list of additional arguments for `dubicube::calculate_bootstrap_ci()` (cube level only). | `list()` |

## Types of Confidence Intervals

The `ci_type` argument allows you to choose between different methods for calculating bootstrap confidence intervals. At cube level these are passed to `dubicube::calculate_bootstrap_ci()`; at indicator level to `boot::boot.ci()`:

*   **Percentile (`"perc"`)**: (Default) The simplest and most commonly used method. It uses the ordered bootstrap replicates to find the intervals (e.g., the 2.5th and 97.5th percentiles for a 95% CI). It is transformation-invariant and generally robust for biodiversity indicators.
*   **Bias-Corrected and Accelerated (`"bca"`)**: An improvement over the percentile method that adjusts for both bias and skewness in the bootstrap distribution. It is highly recommended for accuracy but requires more computation.
*   **Normal Approximation (`"norm"`)**: Assumes the bootstrap distribution is approximately normal. It uses the standard error of the bootstrap replicates to calculate the interval. While fast, it may be less accurate for skewed distributions typical of biodiversity metrics.
*   **Basic (`"basic"`)**: Also known as the "reverse percentile" method. It uses the percentiles of the distribution of the difference between the bootstrap replicates and the original estimate.

### Customizing the Bootstrap Process

The random seed, interval type and confidence level are set with the `seed`, `ci_type` and `confidence_level` arguments of `add_ci()`. If you need to pass other parameters to the underlying `dubicube` functions (cube level only), you can use the `boot_args` and `ci_args` parameters:

```{r advanced, eval = FALSE}
occ_ts_custom <- add_ci(occ_ts,
                        num_bootstrap = 500,
                        seed = 42,
                        ci_type = "perc",
                        confidence_level = 0.9,
                        boot_args = list(progress = TRUE))
```

### Overriding Indicator Defaults

While `add_ci()` automatically selects appropriate settings, you can override these defaults using `boot_args` and `ci_args`. This is useful when you need specialized behavior:

```{r override, eval = FALSE}
# Example: Calculate CIs for evenness on raw scale (no logit transformation)
evenness_raw <- add_ci(my_evenness_indicator,
                       num_bootstrap = 1000,
                       ci_args = list(h = identity,
                                      hinv = identity))

# Example: Force bias correction for total_occ
occ_with_bias <- add_ci(my_occ_indicator,
                        num_bootstrap = 1000,
                        ci_args = list(no_bias = FALSE))
```

Common override parameters include:

- `h` / `hinv` (in `ci_args`): Transformation functions (for evenness indicators, these override the default logit transformation)
- `no_bias` (in `ci_args`): Logical, disable bias correction (default varies by indicator)
- `group_specific` (in `boot_args`): Logical, force group-specific vs whole-cube bootstrapping

## Supported Indicators

Confidence intervals can be added post-hoc for the following indicators:

*   Total Occurrences (`total_occ`)
*   Occurrence Density (`occ_density`)
*   Newness (`newness`)
*   Williams Evenness (`williams_evenness`)
*   Pielou Evenness (`pielou_evenness`)
*   Abundance-based Rarity (`ab_rarity`)
*   Area-based Rarity (`area_rarity`)
*   Species Occurrences (`spec_occ`)
*   Species Range (`spec_range`)

### Special Case: Hill Diversity Indicators
While most indicators rely on `dubicube` for calculating confidence intervals, the Hill diversity indicators (`hill0`, `hill1`, `hill2`) compute their metrics using the `iNEXT` package, which runs an intensive coverage-based rarefaction algorithm. `iNEXT` possesses its own highly optimized, native bootstrapping engine.

When you pass a Hill indicator to `add_ci()`, the function will automatically detect it, bypass `dubicube`'s cube-level resampling to prevent computationally disastrous double-bootstrapping, and seamlessly delegate to `iNEXT`'s internal engine (forcing `bootstrap_level = "indicator"`).

### Exclusions

Certain indicators cannot have confidence intervals added via `add_ci()`:

*   **Observed Richness** (`obs_richness`) and **Species Richness Density** (`spec_richness_density`): Bootstrapping observed occurrences can never discover unobserved species, so resampled richness is never above the observed value; consider using Hill numbers for estimated richness instead.
*   **Sample Completeness** (`completeness`): A deterministic coverage estimate; confidence intervals are not supported.
*   **Relative Occupancy** (`relative_occupancy`): Resampling occurrences cannot account for unoccupied grid cells, which are needed to calculate occupancy.
*   **Cumulative Richness** (`cum_richness`): The cumulative nature of this metric makes standard bootstrapping inappropriate.
*   **Occurrence Turnover** (`occ_turnover`): Similar to cumulative richness, the temporal dependency requires specialized methods.
*   **Taxonomic Distinctness** (`tax_distinct`): Requires specialized randomization tests rather than standard bootstrapping.

## Data Exploration with dubicube 🔍

While **b3gbi** focuses on indicator calculation and visualization, the underlying **dubicube** package offers powerful functions for initial data exploration and quality assessment of your biodiversity data cubes.

To learn more about the full capabilities of **dubicube**, including in-depth tutorials and advanced data processing techniques, please visit the [dubicube documentation](https://docs.b-cubed.eu/software/dubicube/readme/).

## Summary

The `add_ci()` function provides a unified and robust interface for adding uncertainty to your biodiversity assessments. By leveraging the power of `dubicube`, **b3gbi** ensures that your results are not just numbers, but scientifically grounded estimates with clearly defined confidence boundaries.

With automatic indicator configuration, most users can rely on sensible defaults while advanced users retain full control over the bootstrapping process through the `boot_args` and `ci_args` parameters.