arXiv research

We find a convex model for traditional nonlinear regression under L2 loss.

New algorithms estimate Jacobian matrices for large-scale machine learning.

This paper presents regression models obtained from a process of blind prediction of peptide binding affinity from provided descriptors for several distinct datasets as part of the 2006 Comparative Evaluation of Prediction Algorithms (COEPRA) contest. This paper finds that kernel partial least squares, a nonlinear part…

Method improves SINDy for noisy nonlinear systems.

The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…

JAXFit speeds up curve fitting on GPUs.

Generalization in nonlinear least squares can be studied via algorithmic stability and effective dimension.

Deep-RLS uses deep learning to improve PCA for better source separation.

This book introduces linear models and their theories rigorously.

Bayesian system ID improves robustness to sparse, noisy data.

The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.

Develops Lagrange-Hamilton geometry for COVID-19 disease dynamics.

A new method estimates parameters of complex models using ordinary least squares.

We learn linear models from nonlinear systems using multiple trajectories and regularization.

Smooth, globally PŁ functions are essentially nonlinear least-squares.

Unified theory and debiasing framework for random oblique projections in high dimensions.

The paper provides a non-asymptotic error bound for linear system identification under nonlinear policies.

We identify linear models from nonlinear systems with initialization constraints.

PGD algorithms solve nonlinear inverse problems with generative priors using noisy measurements.

Bayesian method improves online NARMAX model identification.

Develops geometry for Lotka-Volterra model of species competition.

ADMM algorithm solves nonlinear matrix decompositions efficiently.

Unified multi-view learning framework using OPLS with regularization and deep extensions.

Improved real-time UAV terrain following with RVM-RLS filter.

We study the problem of recovering a structured signal $\mathbf{x}_0$ from high-dimensional data $\mathbf{y}_i=f(\mathbf{a}_i^T\mathbf{x}_0)$ for some nonlinear (and potentially unknown) link function $f$, when the regressors $\mathbf{a}_i$ are iid Gaussian. Brillinger (1982) showed that ordinary least-squares estimate…

New method learns low-dimensional models for systems with non-polynomial terms.

Functional PLS improves prediction and inference for scalar responses from functional predictors.

Machine learning can improve 2SLS first stage predictions, but nonlinear methods often introduce bias.

Optimizes prediction error method for time-varying models.

Method estimates parameters of complex nonlinear systems.

Spectral decomposition of the Koopman operator is attracting attention as a tool for the analysis of nonlinear dynamical systems. Dynamic mode decomposition is a popular numerical algorithm for Koopman spectral analysis; however, we often need to prepare nonlinear observables manually according to the underlying dynami…

A tutorial on non-asymptotic system identification methods.

WENDy now estimates nonlinear ODEs with noisy data.

Proposes KAR for nonlinear causal discovery using kernel methods.

This study defines a multivariate Self--Exciting Threshold Autoregressive with eXogenous input (MSETARX) models and present an estimation procedure for the parameters. The conditions for stationarity of the nonlinear MSETARX models is provided. In particular, the efficiency of an adaptive parameter estimation algorithm…

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size (bandwidth) is still an open important issue especially for learning with small sample s…

CD converges linearly for MCP/SCAD penalized least squares.

We present ADMM-Softmax, an alternating direction method of multipliers (ADMM) for solving multinomial logistic regression (MLR) problems. Our method is geared toward supervised classification tasks with many examples and features. It decouples the nonlinear optimization problem in MLR into three steps that can be solv…

The paper proposes a new system ID method from noisy data.

Stochastic VB improves nonlinear model inference speed and accuracy.

A new tensor completion method using tensor networks with Tucker wrapper.

We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by $y = f( x^\top β^* ) + ε$, where $f$ is nonlinear. To recover $β^*$, we propose an $\ell_1$-regularized least-squares estimator. Unlike classical linear regression, the correspondin…

We use statistical learning methods to construct an adaptive state estimator for nonlinear stochastic systems. Optimal state estimation, in the form of a Kalman filter, requires knowledge of the system's process and measurement uncertainty. We propose that these uncertainties can be estimated from (conditioned on) past…

Illustrates interleaved learning with Kalman Filter for linear least squares.

Simplifies neural regression by combining two sub-networks for predictions and uncertainties.

Paper addresses identifiability and asymptotics of ODE systems from noisy data.

We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance meas…

Cross validation residuals are well known for the ordinary least squares model. Here leave-M-out cross validation is extended to generalised least squares. The relationship between cross validation residuals and Cook's distance is demonstrated, in terms of an approximation to the difference in the generalised residual …