PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss

Deividas Eringis (1) , John Leth (1) , Zheng-Hua Tan (2) , Rafael Wisniewski (1) , Mihaly Petreczky (3)

1 Department of Electronic Systems - Aalborg University
2 Department of Computer Science [Aalborg]
3 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189

In this paper we derive a Probably Approxilmately Correct(PAC)-Bayesian error bound for linear time-invariant (LTI) stochastic dynamical systems with inputs. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. In particular, with the bound derived in this paper relates future average prediction errors with the prediction error generated by the model on the data used for learning. In turn, this allows us to provide finite-sample error bounds for a wide class of learning/system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.

Mots clés: Machine Learning (stat.ML); Machine Learning (cs.LG); FOS: Computer and information sciences

Domaines: Intelligence artificielle [cs.AI]; Automatique / Robotique

Deividas Eringis, John Leth, Zheng-Hua Tan, Rafael Wisniewski, Mihaly Petreczky. PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss. Workshop Frontiers4LCD ICML 2023, Jul 2023, Honolulu, United States. ⟨10.48550/arXiv.2303.16816⟩. ⟨hal-04304089⟩