Introduction to EMLMLasso

library(EMLMLasso)

Overview

EMLMLasso fits linear mixed models with a lasso penalty on the fixed-effects coefficients, for clustered or longitudinal data, via an Expectation-Maximization (EM) algorithm. At each E-step, the random effects are treated as missing data and replaced by their conditional means and covariances; the fixed effects are updated at each M-step with a coordinate-descent lasso fit (glmnet::glmnet()) on the random-effects- adjusted response.

One function, EMLMLasso(), does the fitting, and what lambda is decides the mode:

Simulating example data

simulate_lasso_lmm() generates longitudinal data from a sparse linear mixed model: p fixed-effect covariates, only pstar of which have a non-zero true coefficient, plus a random intercept and random slope on time.

set.seed(2026)
p <- 10
pstar <- 3
dat <- simulate_lasso_lmm(n = 40, nj = 5, p = p, pstar = pstar, sigmae = 1, seed = 2026)
head(dat)
#>   id time          y          x1         x2         x3           x4         x5
#> 1  1    1  1.9120118  0.52058907 -0.4320264  1.3368926  0.962138142 -1.6052617
#> 2  1    2 -0.2994622 -1.07969076  0.4585911  0.5527104 -0.178134779 -0.5024982
#> 3  1    3 -0.9837634  0.13923812 -0.2293520 -1.0565264  0.003289601 -1.1958575
#> 4  1    4 -0.5575649 -0.08474878 -1.8569622 -0.1901830  0.511615166 -1.0710676
#> 5  1    5 -1.0219079 -0.66663962 -0.2896742 -0.9092242 -0.456060688  0.7461867
#> 6  2    1 -4.5323892 -2.51608903  1.7671412 -2.0009155 -0.210904119 -0.4368864
#>           x6           x7         x8         x9         x10
#> 1  1.7705647  0.292106057  1.5912746 -0.7983005 -1.56410465
#> 2  0.9003771  0.735734309 -1.9549243  1.1286730 -0.57399073
#> 3  1.3304134 -0.004291783 -0.9887574 -0.5711334 -0.09445245
#> 4 -0.3492971 -0.446130570  0.9901177  1.1552387 -0.73474588
#> 5  1.0621850  1.014299812  0.9884280 -0.6623258  1.35623937
#> 6 -0.6199544 -1.290849154 -0.5208329  1.2274016  0.15436091

The response is y; the grouping variable is id; the fixed-effect covariates are x1, ..., x10, of which only x1, x2, x3 have a non-zero true coefficient (equal to 1 in this simulation).

Fitting a single lambda

formFixed and formRandom are ordinary R formulas. Here formFixed includes all ten covariates, and formRandom = ~ time requests a random intercept and a random slope on time (the true data-generating structure).

fit <- EMLMLasso(dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time,
                 lambda = 0.15, verbose = FALSE)
fit
#> EM algorithm for lasso-penalized linear mixed models
#> Call:
#> EMLMLasso(data = dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time, 
#>     lambda = 0.15, verbose = FALSE)
#> 
#> lambda: 0.15  | active covariates: 3 of 11 
#> 
#> Fixed effects (non-zero only):
#>     x1     x2     x3 
#> 0.9209 0.7119 0.7202 
#> 
#> Random effects covariance (D):
#>             (Intercept)   time
#> (Intercept)      0.7495 0.6451
#> time             0.6451 0.6889
#> 
#> Residual variance: 1.0296 
#> logLik: -370.1433  AIC: 754.2866  BIC: 777.3748 
#> EM iterations: 200 (did not converge)

coef(), fitted() and residuals() methods are available:

coef(fit)
#>      time        x1        x2        x3        x4        x5        x6        x7 
#> 0.0000000 0.9209416 0.7118716 0.7201684 0.0000000 0.0000000 0.0000000 0.0000000 
#>        x8        x9       x10 
#> 0.0000000 0.0000000 0.0000000

Selecting lambda by BIC

In practice, lambda is rarely known in advance. Leaving it as NULL fits the model across a data-driven grid of lambda values (sized by n_lambdas) and reports the one with the lowest BIC.

search <- EMLMLasso(dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time,
                    n_lambdas = 15)
search
#> EM algorithm for lasso-penalized linear mixed models
#> Call:
#> EMLMLasso(data = dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time, 
#>     n_lambdas = 15)
#> 
#> BIC search over 15 lambda values
#> 
#> lambda: 0.1083  | active covariates: 3 of 11 
#> 
#> Fixed effects (non-zero only):
#>     x1     x2     x3 
#> 0.9717 0.7661 0.7790 
#> 
#> Random effects covariance (D):
#>             (Intercept)   time
#> (Intercept)      0.7751 0.6344
#> time             0.6344 0.6967
#> 
#> Residual variance: 0.9675 
#> logLik: -365.8906  AIC: 745.7812  BIC: 768.8694 
#> EM iterations: 200 (did not converge)

The selected model correctly identifies x1, x2 and x3 as the active covariates, shrinking the other seven to exactly zero.

coef(search)
#>      time        x1        x2        x3        x4        x5        x6        x7 
#> 0.0000000 0.9717481 0.7661102 0.7790091 0.0000000 0.0000000 0.0000000 0.0000000 
#>        x8        x9       x10 
#> 0.0000000 0.0000000 0.0000000

plot() shows BIC across the lambda grid, with the selected value marked:

plot(search)

summary() gives a more detailed readout of the selected model, including which covariates were shrunk to zero:

summary(search)
#> EM algorithm for lasso-penalized linear mixed models
#> Call:
#> EMLMLasso(data = dat, formFixed = y ~ ., groupVar = "id", formRandom = ~time, 
#>     n_lambdas = 15)
#> 
#> Number of groups: 40  | total observations: 200 
#> BIC search over 15 lambda values
#> 
#> Selected lambda (by BIC): 0.1083 
#> 
#> Selected fixed effects ( 3 of 11 non-zero):
#>     x1     x2     x3 
#> 0.9717 0.7661 0.7790 
#> 
#> Shrunk-to-zero covariates: time, x4, x5, x6, x7, x8, x9, x10 
#> 
#> Random effects covariance (D):
#>             (Intercept)   time
#> (Intercept)      0.7751 0.6344
#> time             0.6344 0.6967
#> Residual variance: 0.9675 
#> 
#> logLik: -365.8906  AIC: 745.7812  BIC: 768.8694  ( 7 effective parameters )

A note on the random-effects structure

The covariance for cluster/subject \(i\) is \(\Sigma_i = Z_i D Z_i' + \sigma^2 I\), with \(b_i \sim N(0, D)\) independent of the residual errors. EMLMLasso() assumes Gaussian random effects and errors, and conditional independence within a cluster given the random effects.

A real-data example: riboflavin production

riboflavinV100 (bundled with this package; see ?riboflavinV100) is a longitudinal version of the well-known riboflavin production dataset for Bacillus subtilis, with 100 gene-expression covariates as fixed-effect candidates – a high-dimensional (p >> n) setting. It is worked through separately in vignette("EMLMLasso-riboflavin").

Relationship to Oliveira, Schumacher and Lachos (2026)

This package implements the EMLMLasso algorithm of Oliveira, Schumacher and Lachos (2026) with modifications. Most notably, the penalty λ is selected by BIC over a grid of candidate values, with λ held fixed within each EM fit, instead of being re-selected by cross-validation at every EM iteration as in Algorithm 1 of the paper. Convergence is also assessed on the change in parameter estimates rather than on the relative change in the penalized log-likelihood. Results may therefore differ from those reported in the paper. See ?EMLMLasso for details (full citation below; run citation("EMLMLasso") for a BibTeX-ready entry).

References

Oliveira DCR, Schumacher FL, Lachos VH (2026). The use of the EM algorithm for regularization problems in high-dimensional linear mixed-effects models. Statistical Methods in Medical Research, 35(2), 240-254. doi:10.1177/09622802251399913