---
title: "Metafrontier Malmquist Productivity Index"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Metafrontier Malmquist Productivity Index}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.width = 7,
  fig.height = 5
)
```

```{r setup}
library(metafrontier)
```

## Motivation

Standard Malmquist productivity indices measure productivity change over
time by decomposing it into efficiency change and technical change. When
firms operate under different technologies, however, this decomposition
misses an important dimension: changes in the *technology gap* between a
group's frontier and the global best practice.

The **metafrontier Malmquist TFP index** of O'Donnell, Rao, and Battese
(2008) extends the standard index with a three-way decomposition that
separates within-group dynamics from cross-group technology convergence
or divergence.


## The three-way decomposition

For a firm observed at times $s$ and $t$, the metafrontier Malmquist
index decomposes as:

$$M^* = TEC \times TGC \times TC^*$$

where:

- **TEC** (Technical Efficiency Change) $= TE^{group}_t / TE^{group}_s$
  measures whether the firm moved closer to or further from its own
  group's frontier.

- **TGC** (Technology Gap Change) $= TGR_t / TGR_s$ measures whether the
  group's frontier moved closer to or further from the metafrontier. A
  value above 1 indicates the group is *catching up* technologically.

- **TC\*** (Metafrontier Technical Change) measures the shift of the
  global production possibility frontier itself.

Values above 1 indicate improvement; below 1 indicate deterioration.


## Simulating panel data

The `simulate_metafrontier()` function generates cross-sectional data.
For panel data, we call it repeatedly with varying parameters to create
time-varying technology gaps:

```{r simulate-panel}
set.seed(42)
panels <- lapply(1:4, function(t) {
  sim <- simulate_metafrontier(
    n_groups = 2,
    n_per_group = 50,
    beta_meta = c(1.0, 0.5, 0.3),
    tech_gap = c(0, 0.3 + 0.03 * t),  # G2 falls behind over time
    sigma_u = c(0.2, 0.3),
    sigma_v = 0.15,
    seed = 42 + t
  )
  sim$data$time <- t
  sim$data$id <- seq_len(nrow(sim$data))
  sim$data
})
panel_data <- do.call(rbind, panels)

table(panel_data$group, panel_data$time)
```

In this simulation, Group G1 operates at the metafrontier (zero
technology gap), while G2 has an increasing gap over time. We would
expect TGC < 1 for G2 (falling behind) and TGC $\approx$ 1 for G1.


## Computing the index

```{r malmquist}
malm <- malmquist_meta(
  log_y ~ log_x1 + log_x2,
  data = panel_data,
  group = "group",
  time = "time",
  orientation = "output",
  rts = "crs"
)

malm
```


## Detailed results

The `summary()` method provides group-level and period-level breakdowns:

```{r summary}
summary(malm)
```


## Interpreting the decomposition

The main results table contains one row per firm per consecutive period
pair:

```{r results-table}
head(malm$malmquist, 10)
```

Each row reports:

| Column | Meaning |
|--------|---------|
| `MPI`  | Metafrontier Malmquist TFP index ($M^* = TEC \times TGC \times TC^*$) |
| `TEC`  | Within-group efficiency change |
| `TGC`  | Technology gap change |
| `TC`   | Metafrontier technical change |

The identity can be verified:

```{r verify-identity}
m <- malm$malmquist
complete <- complete.cases(m[, c("MPI", "TEC", "TGC", "TC")])
all.equal(m$MPI[complete], m$TEC[complete] * m$TGC[complete] * m$TC[complete])
```


## Within-group vs metafrontier Malmquist

The object also stores the standard within-group Malmquist decomposition
and the metafrontier-level decomposition:

```{r group-vs-meta}
# Within-group: MPI_group = EC_group x TC_group
head(malm$group_malmquist)

# Metafrontier: MPI_meta = EC_meta x TC_meta
head(malm$meta_malmquist)
```

The within-group index captures only efficiency change and frontier shift
*within* the group. The metafrontier index additionally accounts for
whether the group is converging toward or diverging from the global best
practice.


## Technology gap dynamics

The TGR at each period endpoint is stored in the `tgr` component:

```{r tgr-dynamics}
tgr_df <- malm$tgr

# Mean TGR by group and period
aggregate(cbind(TGR_from, TGR_to) ~ group, data = tgr_df, FUN = mean)
```

For G2, we expect TGR to decline over time (increasing technology gap).
The TGC column confirms this:

```{r tgc-by-group}
aggregate(TGC ~ group, data = tgr_df, FUN = mean)
```


## Returns to scale assumptions

The `rts` argument controls the DEA technology assumption. Under variable
returns to scale, scale effects are netted out:

```{r vrs-comparison}
malm_vrs <- malmquist_meta(
  log_y ~ log_x1 + log_x2,
  data = panel_data,
  group = "group",
  time = "time",
  rts = "vrs"
)

# Compare mean MPI under CRS vs VRS
data.frame(
  CRS = colMeans(malm$malmquist[, c("MPI", "TEC", "TGC", "TC")],
                 na.rm = TRUE),
  VRS = colMeans(malm_vrs$malmquist[, c("MPI", "TEC", "TGC", "TC")],
                 na.rm = TRUE)
)
```


## Using real-world panel data

The `plm` package provides `Produc`, a panel of 48 US states over
1970--1986 with a built-in `region` grouping variable. This is a natural
candidate for metafrontier Malmquist analysis:

```{r produc-example, eval = FALSE}
library(plm)
data("Produc", package = "plm")

malm_us <- malmquist_meta(
  gsp ~ pc + emp,
  data = Produc,
  group = "region",
  time = "year",
  rts = "crs"
)
summary(malm_us)
```

Similarly, `sfaR::utility` provides electric utility data with a binary
`regu` variable (regulated vs. deregulated) over 1986--1996:

```{r utility-example, eval = FALSE}
library(sfaR)
data("utility", package = "sfaR")

malm_util <- malmquist_meta(
  y ~ k + labor + fuel,
  data = utility,
  group = "regu",
  time = "year",
  rts = "vrs"
)
summary(malm_util)
```


## References

- O'Donnell, C.J., Rao, D.S.P. and Battese, G.E. (2008). Metafrontier
  frameworks for the study of firm-level efficiencies and technology
  ratios. *Empirical Economics*, 34(2), 231--255.

- Battese, G.E., Rao, D.S.P. and O'Donnell, C.J. (2004). A metafrontier
  production function for estimation of technical efficiencies and
  technology gaps for firms operating under different technologies.
  *Journal of Productivity Analysis*, 21(1), 91--103.