---
title: "Part 1: GBIF Retrieval, Taxonomy, and Filtering"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Part 1: GBIF Retrieval, Taxonomy, and Filtering}
  %\VignetteEngine{knitr::rmarkdown_notangle}
  %\VignetteEncoding{UTF-8}
---

## Transparent setup

```{r setup, include = TRUE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.width = 7,
  fig.height = 5,
  purl = FALSE
)

ext_file <- function(...) {
  path <- system.file("extdata", ..., package = "gbif.range")
  if (nzchar(path)) {
    return(path)
  }
  normalizePath(file.path("..", "inst", "extdata", ...), mustWork = TRUE)
}

library(gbif.range)
```

## Scope

This vignette covers the package components that sit upstream of range inference:

- `get_status()` to inspect the GBIF backbone taxon concept and synonym set,
- `get_gbif_count()` to gauge likely download size,
- `get_gbif()` to retrieve occurrences without GBIF credentials,
- `obs_filter()` to thin or aggregate occurrences for downstream work,
- `make_tiles()` to build explicit GBIF geometry tiles,
- `get_doi()` to turn one or more downloads into a citable GBIF-derived DOI.

The examples below follow a practical sequence that works well in real analyses: resolve taxonomy first, estimate likely download size second, and only then choose the most appropriate retrieval strategy for the spatial scale and record volume of the analysis.

## Why taxonomy comes first

In `gbif.range`, taxonomy is not an afterthought. The package uses the GBIF backbone to decide which accepted taxon concept a query refers to and which synonyms belong to that concept.

This is why `get_status()` is often the best first step:

```{r get-status, eval = FALSE}
tax <- get_status("Cypripedium calceolus", level = "children")
tax
```

When `level = "children"`, the output is deliberately close to the logic used by `get_gbif()`: it returns the accepted name and the synonyms that are actually used for occurrence retrieval, i.e., including subspecies and variety. While `level = "accepted"` is only exploratory and strictly keeps species level names. When ``level = "all"``, additional related names are also included for taxonomic inspection, but those extra names are not used to download occurrences.

The practical consequence is important: `get_gbif()` harmonizes the query to the accepted GBIF taxon key, but the returned occurrence table still preserves both record-level and accepted-name fields.

## Count before download

For broad extents or common taxa, a quick count helps you decide whether a direct retrieval is appropriate:

```{r get-gbif-count, eval = FALSE}
get_gbif_count("Panthera tigris", search = TRUE)
```

This is particularly useful if you need to decide between:

- a direct `get_gbif()` call,
- a more strongly filtered call with a stricter `grain` or `occ_samp`,
- or a downloaded GBIF export that will later be processed with the disk-based batch workflow described in Part 3: `vignette("large-downloaded-gbif-tables", package = "gbif.range")`.

In practice, this is the quickest way to decide whether a direct `get_gbif()` call remains convenient or whether the project has crossed into the "download first, process on disk later" regime.

## Strict versus permissive name matching

An important distinction is controlled by `search`. With `search = TRUE`, `get_gbif()` expects a GBIF backbone match for the focal taxon concept. With `search = FALSE`, the function becomes more permissive and can return records for fuzzy matches or higher-rank names. Let's do a quick test with `get_gbif_count()`, as it behaves the same way as `get_gbif()`:

```{r search-logic, eval = FALSE}
get_gbif_count("Panthera tigris", search = TRUE)      # records for the tiger concept
get_gbif_count("Panthera tigriiis", search = TRUE)    # typically no records
get_gbif_count("Panthera tigriiis", search = FALSE)   # permissive fuzzy retrieval
get_gbif_count("Acer", search = FALSE)                # higher-rank retrieval
```

For most single-species analyses the strict setting is preferable, because it keeps the biological interpretation clear. The permissive mode is more appropriate when the aim is exploratory retrieval, manual inspection, or broader higher-rank data collection.

## Credential-free GBIF retrieval

`get_gbif()` is a credential-free wrapper around `rgbif::occ_search()`
(Chamberlain et al. 2022). The main package contribution is that it combines
taxonomic harmonization, geographic tiling of large extents, and a practical
sequence of post-download filters, custom and via `CoordinateCleaner`
(Zizka et al. 2019) in one function. The retrieval workflow originates from
Chauvier et al. (2021, *Ecological Monographs*).

```{r get-gbif, eval = FALSE}
# Download global tiger occurrences with the default filters.
obs_tiger <- get_gbif(
  "Panthera tigris",
  grain = 100,
  basis = c('OBSERVATION', 'HUMAN_OBSERVATION', 'MACHINE_OBSERVATION',
    'OCCURRENCE', 'MATERIAL_CITATION', 'MATERIAL_SAMPLE','LITERATURE'),
  establishment = c('native','casual','released','reproducing',
    'established','colonising','invasive','widespreadInvasive'),
  time_period = c(1950, 3000)
)

# Inspect the accepted name and synonym mapping used internally.
get_status("Panthera tigris", level = "children")
```

On top of the desired geographic extent (`geo`), the arguments most often worth adjusting are:

- `grain`, which filters by coordinate uncertainty and decimal precision,
- `basis` and `establishment`, which control the allowed record types,
- `time_period`, if the analysis targets a defined time window,
- `occ_samp`, if the geographic extent is so large that a full retrieval is unnecessary or impractical.

These arguments have different roles. `grain` is the main spatial-quality filter. `basis`, `establishment`, and related arguments define which kinds of records are biologically acceptable. `occ_samp` is the pragmatic scaling argument when a full credential-free retrieval would be too slow or too large for the immediate task.

## Credential-based retrieval with `occ_download`

For larger or more reproducible downloads, `get_gbif()` also supports `rgbif::occ_download()` via the `should_use_occ_download` argument. This requires a GBIF account. Credentials can be supplied directly as arguments or stored as environment variables (`GBIF_USER`, `GBIF_PWD`, `GBIF_EMAIL`):

```{r get-gbif-download, eval = FALSE}
# Option 1: pass credentials directly
obs_tiger <- get_gbif(
  "Panthera tigris",
  should_use_occ_download = TRUE,
  occ_download_user  = "my_gbif_username",
  occ_download_pwd   = "my_gbif_password",
  occ_download_email = "my@email.com"
)

# Option 2: store credentials as environment variables (recommended)
Sys.setenv(GBIF_USER  = "my_gbif_username")
Sys.setenv(GBIF_PWD   = "my_gbif_password")
Sys.setenv(GBIF_EMAIL = "my@email.com")

# Download
obs_tiger <- get_gbif(
  "Panthera tigris",
  should_use_occ_download = TRUE
)
```

The `occ_download` route submits a single formal GBIF download request for the full extent, waits for it to complete, and imports the result. Unlike the default `occ_search` mode, no tiling is applied — the entire query is handled in one request. In practice, `occ_download` tends to be slower than `occ_search` for species with few records, since GBIF queues the request server-side regardless of size. It however becomes more pragmatic for heavy downloads or when processing many species in batch, where the overhead of tiling and repeated API calls would otherwise dominate.

## A terrestrial large-extent example

*Bison bison* is another good example of a case where record volume becomes a practical constraint, as large extents can require more deliberate choices about sampling and ecoregions (e.g., occ_samp argument):

```{r Bison-example, eval = FALSE}
# Retrieve a manageable subsample per internally generated tile.
obs_bison <- get_gbif(
  sp_name = "Bison bison",
  occ_samp = 1000
)

# Inspect the internal get_gbif() logic and all related names.
get_status("Bison bison", level = "all")
```

This is exactly the kind of analysis where `get_gbif_count()` is useful upstream. If the expected volume is extremely large and the project targets many taxa, the disk-based workflow in Part 3: `vignette("large-downloaded-gbif-tables", package = "gbif.range")` is usually the more scalable choice.

## Post-download thinning with `obs_filter()`

The package also includes a lightweight grid-based thinning helper. This is useful when many records fall into the same grid cell and you want one retained observation per species per cell before a downstream analysis.

The example below uses the bundled offline GBIF-style example table rather than a live web query.

```{r obs-filter}
# Set
occ_raw <- utils::read.delim(
  ext_file("occ_example_4sps.csv"),
  sep = "\t",
  stringsAsFactors = FALSE
)
occ_raw$input_search <- occ_raw$species
occ_gbif <- getGBIF(occ_raw)

# Build a coarse grid that spans the example records.
grid <- terra::rast(
  xmin = min(occ_gbif$decimalLongitude) - 1,
  xmax = max(occ_gbif$decimalLongitude) + 1,
  ymin = min(occ_gbif$decimalLatitude) - 1,
  ymax = max(occ_gbif$decimalLatitude) + 1,
  resolution = 10,
  crs = "EPSG:4326"
)

# Keep at most one record per species and grid cell.
obs_thin <- obs_filter(occ_gbif, grid)
head(obs_thin)
```

This kind of thinning does not replace ecological cleaning or taxonomic checks, but it can be a useful way to reduce local clustering before plotting or model calibration.

Dense clusters of records can easily dominate visual inspection or downstream calibration even when they add little new geographic information, so this kind of lightweight aggregation is often worth doing early.

## Explicit tiling workflows

`get_gbif()` handles tiling internally, but `make_tiles()` is available when you want to create GBIF-ready geometry tiles yourself:

```{r make-tiles, eval = FALSE}
tiles <- make_tiles(terra::ext(-20, 40, 0, 60), ntiles = 5)
tiles[[1]]
```

This is mostly useful for custom `rgbif` workflows or explicit diagnostics of how an extent is being subdivided.

## Reproducible GBIF citation

`get_doi()` wraps `rgbif::derived_dataset()` to register a citable GBIF-derived dataset DOI from one or more `get_gbif()` outputs. It requires GBIF credentials and extracts the `datasetKey` column from the `getGBIF` object to identify the source datasets. A title, description, and source URL pointing to the workflow documentation are also required:

```{r get-doi, eval = FALSE}
# Need GBIF Credentials
doi_result <- get_doi(
  gbifs       = obs_tiger,
  title       = "Panthera tigris GBIF occurrences for range mapping",
  description = "GBIF occurrence records for Panthera tigris retrieved with gbif.range.",
  source_url  = "https://github.com/8Ginette8/gbif.range",
  user        = "my_gbif_username",
  pwd         = "my_gbif_password"
)
doi_result
```

Note that `get_doi()` works the same regardless of whether `obs_tiger` was retrieved via `occ_search` or `occ_download` — it reads the `datasetKey` column present in both outputs, counts records per source dataset, and passes that summary to `rgbif::derived_dataset()`. This is a small function, but it is scientifically important because it improves traceability and citation of the exact GBIF-derived datasets used in an analysis

## Take-home message

The GBIF-facing side of `gbif.range` is built around a clear sequence:

1. inspect the GBIF backbone taxon concept with `get_status()`,
2. count likely record volume with `get_gbif_count()`,
3. download filtered occurrences with `get_gbif()` — credential-free via `occ_search` (default) or credential-based via `should_use_occ_download = TRUE`,
4. optionally thin or aggregate them with `obs_filter()`,
5. cite the resulting GBIF-derived datasets with `get_doi()`.

That sequence keeps taxonomic interpretation, record selection, and reproducibility visible before range inference begins.

## References

Chamberlain, S., Oldoni, D., & Waller, J. (2022). rgbif: interface to the global biodiversity information facility API. https://doi.org/10.5281/zenodo.6023735

Chauvier, Y., Thuiller, W., Brun, P., Lavergne, S., Descombes, P., Karger, D. N., Renaud, J., & Zimmermann, N. E. (2021). Influence of climate, soil, and land cover on plant species distribution in the European Alps. *Ecological Monographs*, 91(2), e01433. https://doi.org/10.1002/ecm.1433

Zizka, A., Silvestro, D., Andermann, T., Azevedo, J., Duarte Ritter, C., Edler, D., … Antonelli, A. (2019). CoordinateCleaner: Standardized cleaning of occurrence records from biological collection databases. *Methods in Ecology and Evolution*, 10(5), 744–751. https://doi.org/10.1111/2041-210X.13152