Getting started with splineKANr

splineKANr fits Kolmogorov-Arnold Networks (KAN; Liu et al., 2024) whose edge functions are

\[\phi_{ij}(x) = w^{b}_{ij}\,\mathrm{silu}(x) + w^{s}_{ij} \sum_m c_{ijm} B_m(x),\]

with cubic B-splines \(B_m\) on an extended knot grid. Each node sums its incoming edges. Training uses R torch autograd; a base R reference implementation with analytic gradients (backend = "r") evaluates the same model without libtorch and is used to validate the torch backend.

This vignette uses backend r.

1. Fit a model

library(splineKANr)
d  <- kan_synthetic("interaction", n = 400, seed = 1)   # y = x1 * x2
te <- kan_synthetic("interaction", n = 500, seed = 2)

fit <- kan_fit(d$x, d$y, hidden = 4, epochs = 150, batch_size = 64,
               lr = 0.02, patience = 30, grid_update_every = 10,
               grid_update_until = 50, seed = 1, backend = backend)
fit
#> <kan_model> B-spline KAN
#>   width      : [2, 4, 1]
#>   grid / k   : G = 5, k = 3, extrapolate = linear
#>   parameters : 125
#>   epochs run : 135 (early stopped); best epoch 105
#>   best loss  : train 9.962e-05, val 0.0002704 (normalized scale)
kan_metrics(te$y, predict(fit, te$x))
#>         rmse          mae           r2            n 
#> 5.643107e-03 3.295561e-03 9.997377e-01 5.000000e+02

kan_fit() normalizes predictors onto the spline domain with statistics from the training rows only, keeps a validation split for early stopping, restores the best epoch and records the full history:

kan_plot_history(fit)

2. Inspect the learned functions

plot(fit, layer = 1, data = d$x)

kan_plot_network(fit, d$x)

plot(kan_partial_dependence(fit, d$x, "x1"))

kan_activation_check() reports how many layer inputs fall outside the spline domain. With the default extrapolate = "linear" the edges continue linearly there, so gradients do not vanish.

fit$activation_check
#>   layer  input_min input_max domain_min domain_max frac_outside
#> 1     1 -1.0000000  1.000000 -1.0000000   1.000000       0.0000
#> 2     2 -0.8797675  1.156436 -0.8723096   1.145545       0.0125

3. Grid refinement, pruning, regularization

fine <- kan_refine(fit, d$x, grid_size = 10)       # G = 5 -> 10
fine <- kan_fit(d$x, d$y, init = fine, epochs = 50, batch_size = 64,
                lr = 0.01, seed = 1, backend = backend)
kan_metrics(te$y, predict(fine, te$x))
#>         rmse          mae           r2            n 
#> 3.788671e-03 1.779833e-03 9.998818e-01 5.000000e+02

pruned <- kan_prune(fine, d$x)
pruned$pruning$removed_nodes
#> [1] hidden_layer node        
#> <0 rows> (or 0-length row.names)

Sparsity regularization in the style of pykan (L1 + entropy of edge scales) requires the torch backend:

fit_sparse <- kan_fit(d$x, d$y, hidden = 4, epochs = 200,
                      reg = kan_reg_control(lambda = 1e-3))

4. Save and reload

f <- tempfile(fileext = ".rds")
kan_save(fine, f)
same <- kan_load(f)
max(abs(predict(same, te$x) - predict(fine, te$x)))
#> [1] 0

5. Numerical validation

v <- kan_validate_basis(grid_sizes = 5, backend = "r")
v[, c("spline_order", "max_err_splineDesign", "max_err_partition", "pass")]
#>   spline_order max_err_splineDesign max_err_partition pass
#> 1            1         1.110223e-16      0.000000e+00 TRUE
#> 2            2         2.220446e-16      2.220446e-16 TRUE
#> 3            3         3.330669e-16      3.330669e-16 TRUE
g <- kan_validate_gradient(widths = list(c(2, 3, 1)), backend = "r")
g[, c("extrapolate", "comparison", "max_rel_err", "pass")]
#>   extrapolate                comparison  max_rel_err pass
#> 1      linear analytic_r vs finite_diff 5.660148e-07 TRUE
#> 2        none analytic_r vs finite_diff 3.687253e-07 TRUE
#> 3       clamp analytic_r vs finite_diff 2.618747e-06 TRUE

kan_validate_all(backend = "both") additionally compares torch autograd with the analytic gradients when libtorch is installed.

References

Liu, Z. et al. (2024). KAN: Kolmogorov-Arnold Networks. doi:10.48550/arXiv.2404.19756