blockCV

Spatially and environmentally separated folds for cross-validation

The blockCV package creates spatially or environmentally separated training and testing folds for k-fold, leave-group-out, and leave-one-out (LOO) cross-validation. These folds support more realistic evaluation of models fitted to spatially structured data, including remote-sensing classification, soil mapping, and species distribution modelling.

Alongside several fold-construction strategies, blockCV provides tools for checking fold balance, comparing fold separation with the prediction domain, and identifying environmental extrapolation. It can also estimate spatial autocorrelation ranges in point data or continuous raster covariates, providing an initial distance scale to investigate when designing spatial folds.

Main features

• Six fold-construction strategies: spatial blocks (cv_spatial), spatial or environmental clustering (cv_cluster), existing grouping factors (cv_group), buffering (cv_buffer), leave-one-out nearest neighbour distance matching (cv_nndm), and k-fold nearest neighbour distance matching (cv_knndm)
• Hexagonal (default), rectangular, or user-defined spatial blocks, assigned to folds using random, systematic, checkerboard, or predefined selection. Random assignment can search for balanced folds
• Environmental clustering with optional spatial constraints through spatial_weight, and optional over-clustering to improve fold balance
• Geographical or feature-space kNNDM using prediction locations supplied by a raster (r), prediction points (pred_points), or a polygon (model_domain), with block, hierarchical, or k-means grouping
• Response-aware fold summaries and, where supported, balancing for binary, multi-class, continuous, count, and presence-background data. Continuous and count responses can be grouped into quantile bins using num_bins
• Fold diagnostics through cv_summary, cv_distance, and cv_similarity to assess balance, train-test separation, agreement with prediction-domain distances, and environmental extrapolation
• Fold visualisation with cv_plot, including faceted train-test maps for every strategy and combined-fold maps for k-fold methods
• Spatial autocorrelation and interactive block-size tools (cv_spatial_autocor and cv_block_size) for exploring an initial separation distance
• Raster processing with terra, including support for stars, raster, and raster files on disk

What’s new in v4.0

• Added cv_knndm for k-fold nearest neighbour distance matching, with geographical and feature-space matching and block, hierarchical, or k-means grouping
• Added cv_group for leave-group-out cross-validation based on an existing site, plot, campaign, individual, or other grouping factor
• Added cv_summary for one-call fold-quality summaries and warnings, and cv_distance for comparing fold separation with nearest-neighbour distances in the prediction domain
• Expanded cv_similarity with per-fold extrapolation summaries, overall novelty rates, and spatial map visualisation
• Made fold balancing explicit in cv_spatial, cv_cluster, and cv_knndm; added presence-background balancing and quantile binning (num_bins) for continuous or count responses
• Added spatially-constrained environmental clustering through cv_cluster(spatial_weight = ...)
• Added combined-fold maps to cv_plot and informative print methods for fold and diagnostic objects
• Expanded cv_nndm and cv_knndm to accept prediction rasters, prediction points, or model-domain polygons
• Removed the legacy v2.x function names. Other breaking changes include the new structured return value from cv_similarity, the rename of num_plot to num_plots, and interactive-only defaults for several automatic plots, reports, and progress bars

See NEWS.md for the full changelog.

Installation

To install the latest update of the package from GitHub use:

remotes::install_github("rvalavi/blockCV", build_vignettes = TRUE, dependencies = TRUE)

Or installing from CRAN:

install.packages("blockCV", dependencies = TRUE)

Vignettes

The package ships with several tutorials as vignettes:

1. blockCV introduction: how to create block cross-validation folds (tutorial_1)
2. Choosing and diagnosing spatial folds (tutorial_2)
3. Block cross-validation for species distribution modelling (tutorial_3)
4. Using blockCV with caret (tutorial_4)

To read them, install the package with the vignettes built (see Installation and use build_vignettes = TRUE), then open them from R:

# list all tutorials
browseVignettes("blockCV")

# or open one directly
vignette("tutorial_1", package = "blockCV")

Basic usage

The examples below highlight a few common workflows. See the vignettes for more information and complete examples.

# loading the package
library(blockCV)
library(sf) # working with spatial vector data
library(terra) # working with spatial raster data

# load raster data; the pipe operator |> is available in R v4.1 or higher
covars <- system.file("extdata/au/", package = "blockCV") |>
  list.files(full.names = TRUE) |>
  terra::rast()

# load species presence-absence data and convert to sf
pa_data <- read.csv(system.file("extdata/", "species.csv", package = "blockCV")) |>
  sf::st_as_sf(coords = c("x", "y"), crs = 7845)

# spatial blocking by specified range and random assignment
sb <- cv_spatial(
    x = pa_data,          # sf object of sample points (e.g. species data)
    column = "occ",       # optional response column for fold records/balancing
    r = covars,           # a raster for background (optional)
    size = 350000,        # size of the blocks in metres
    k = 5,                # number of folds
    hexagon = TRUE,       # use hexagonal blocks - default
    selection = "random", # random blocks-to-fold
    balance = TRUE,       # find balanced folds
    iteration = 100,      # search for balanced folds
    biomod2 = TRUE        # also create folds for biomod2
)

Use cv_plot() or the generic plot() method to visualise the folds.

plot(sb, pa_data, combine_folds = TRUE)

cv_similarity() compares each testing fold with its corresponding training data to identify environmental extrapolation. Negative MESS values flag test points outside the environmental range represented by their training data. The distribution and map views show how much extrapolation occurs and where, while the returned object also provides per-fold and overall summaries.

sim1 <- cv_similarity(cv = sb, x = pa_data, r = covars, method = "MESS")
sim2 <- cv_similarity(cv = sb, x = pa_data, r = covars, method = "MESS", type = "map")

sim_map <- sim2$plot +
    ggplot2::labs(x = "Longitude", y = "Latitude")

cowplot::plot_grid(sim1$plot, sim_map, nrow = 1)

cv_cluster() can be tailored to different validation goals. For spatial clustering, balance = TRUE forms additional candidate clusters and assigns them to folds to improve record or response-class balance; k_multiplier controls the trade-off with geographical compactness. For environmental clustering, spatial_weight adds a soft geographical constraint so environmentally similar folds are less spatially scattered.

# balanced spatial clustering
set.seed(6)
bc <- cv_cluster(
    x = pa_data,
    column = "occ",
    k = 5,
    balance = TRUE,
    k_multiplier = 3
)

# spatially-constrained environmental clustering
set.seed(6)
sec <- cv_cluster(
    x = pa_data,
    r = covars,
    column = "occ",
    k = 5,
    spatial_weight = 0.4
)

bc_plot <- cv_plot(bc, x = pa_data, combine_folds = TRUE) +
    ggplot2::labs(title = "Balanced spatial clustering")

sec_plot <- cv_plot(sec, x = pa_data, combine_folds = TRUE) +
    ggplot2::labs(title = "Spatially-constrained environmental clustering")

cowplot::plot_grid(bc_plot, sec_plot, nrow = 1)

Create k-fold NNDM folds:

# k-fold nearest neighbour distance matching
knn <- cv_knndm(
    x = pa_data,
    column = "occ", # optionally prefer class-complete folds
    r = covars, # prediction area, or use pred_points/model_domain
    k = 5,
    num_sample = 5000
)

# compare an existing fold design with prediction-domain distances
cv_distance(
    cv = sb,
    x = pa_data,
    r = covars,
    num_sample = 5000
)

Investigate spatial autocorrelation in the landscape to choose a suitable size for spatial blocks:

# exploring the effective range of spatial autocorrelation in raster covariates or sample data
cv_spatial_autocor(
    r = covars, # a SpatRaster object or path to files
    num_sample = 5000, # number of cells to be used
    plot = TRUE
)

For the residual-based block-size guidance in Roberts et al. (2017), fit the model first, add its residuals to the sample points, and pass that residual column to cv_spatial_autocor(x = ..., column = ...). Ranges estimated from the raw response or raster covariates are exploratory proxies and may mis-size blocks for residual autocorrelation.

Alternatively, you can manually choose the size of spatial blocks in an interactive session using a Shiny app.

# a shiny interactive app to aid selecting a size for spatial blocks
cv_block_size(
    r = covars[[1]],
    x = pa_data, # optionally add sample points
    column = "occ",
    min_size = 2e5,
    max_size = 9e5
)

Reporting issues

Please report issues at: https://github.com/rvalavi/blockCV/issues

Acknowledgements

Special thanks to Eleanor Stern, who created the original artwork for the blockCV logo.

Citation

To cite package blockCV in publications, please use:

Valavi R, Elith J, Lahoz-Monfort JJ, Guillera-Arroita G. blockCV: An R package for generating spatially or environmentally separated folds for k-fold cross-validation of species distribution models. Methods Ecol Evol. 2019; 10:225–232. https://doi.org/10.1111/2041-210X.13107