splineKANr fits Kolmogorov-Arnold Networks (KAN; Liu et
al., 2024) whose edge functions are
\[\phi_{ij}(x) = w^{b}_{ij}\,\mathrm{silu}(x) + w^{s}_{ij} \sum_m c_{ijm} B_m(x),\]
with cubic B-splines \(B_m\) on an
extended knot grid. Each node sums its incoming edges. Training uses R
torch autograd; a base R reference implementation with
analytic gradients (backend = "r") evaluates the same model
without libtorch and is used to validate the torch backend.
This vignette uses backend r.
library(splineKANr)
d <- kan_synthetic("interaction", n = 400, seed = 1) # y = x1 * x2
te <- kan_synthetic("interaction", n = 500, seed = 2)
fit <- kan_fit(d$x, d$y, hidden = 4, epochs = 150, batch_size = 64,
lr = 0.02, patience = 30, grid_update_every = 10,
grid_update_until = 50, seed = 1, backend = backend)
fit
#> <kan_model> B-spline KAN
#> width : [2, 4, 1]
#> grid / k : G = 5, k = 3, extrapolate = linear
#> parameters : 125
#> epochs run : 135 (early stopped); best epoch 105
#> best loss : train 9.962e-05, val 0.0002704 (normalized scale)
kan_metrics(te$y, predict(fit, te$x))
#> rmse mae r2 n
#> 5.643107e-03 3.295561e-03 9.997377e-01 5.000000e+02kan_fit() normalizes predictors onto the spline domain
with statistics from the training rows only, keeps a validation split
for early stopping, restores the best epoch and records the full
history:
kan_activation_check() reports how many layer inputs
fall outside the spline domain. With the default
extrapolate = "linear" the edges continue linearly there,
so gradients do not vanish.
fine <- kan_refine(fit, d$x, grid_size = 10) # G = 5 -> 10
fine <- kan_fit(d$x, d$y, init = fine, epochs = 50, batch_size = 64,
lr = 0.01, seed = 1, backend = backend)
kan_metrics(te$y, predict(fine, te$x))
#> rmse mae r2 n
#> 3.788671e-03 1.779833e-03 9.998818e-01 5.000000e+02
pruned <- kan_prune(fine, d$x)
pruned$pruning$removed_nodes
#> [1] hidden_layer node
#> <0 rows> (or 0-length row.names)Sparsity regularization in the style of pykan (L1 + entropy of edge scales) requires the torch backend:
v <- kan_validate_basis(grid_sizes = 5, backend = "r")
v[, c("spline_order", "max_err_splineDesign", "max_err_partition", "pass")]
#> spline_order max_err_splineDesign max_err_partition pass
#> 1 1 1.110223e-16 0.000000e+00 TRUE
#> 2 2 2.220446e-16 2.220446e-16 TRUE
#> 3 3 3.330669e-16 3.330669e-16 TRUE
g <- kan_validate_gradient(widths = list(c(2, 3, 1)), backend = "r")
g[, c("extrapolate", "comparison", "max_rel_err", "pass")]
#> extrapolate comparison max_rel_err pass
#> 1 linear analytic_r vs finite_diff 5.660148e-07 TRUE
#> 2 none analytic_r vs finite_diff 3.687253e-07 TRUE
#> 3 clamp analytic_r vs finite_diff 2.618747e-06 TRUEkan_validate_all(backend = "both") additionally compares
torch autograd with the analytic gradients when libtorch is
installed.
Liu, Z. et al. (2024). KAN: Kolmogorov-Arnold Networks. doi:10.48550/arXiv.2404.19756