arXiv research

We find a convex model for traditional nonlinear regression under L2 loss.

New algorithms estimate Jacobian matrices for large-scale machine learning.

ADMM algorithm solves nonlinear matrix decompositions efficiently.

Method improves SINDy for noisy nonlinear systems.

This paper presents regression models obtained from a process of blind prediction of peptide binding affinity from provided descriptors for several distinct datasets as part of the 2006 Comparative Evaluation of Prediction Algorithms (COEPRA) contest. This paper finds that kernel partial least squares, a nonlinear part…

Deep-RLS uses deep learning to improve PCA for better source separation.

JAXFit speeds up curve fitting on GPUs.

JULIA combines multi-linear and nonlinear models for tensor completion.

Generalization in nonlinear least squares can be studied via algorithmic stability and effective dimension.

We learn linear models from nonlinear systems using multiple trajectories and regularization.

Agent-based market shows herding cycles with square-root price impact.

Gradient descent on ReLU networks with square loss implicitly favors balanced weights.

PGD algorithms solve nonlinear inverse problems with generative priors using noisy measurements.

The paper provides a non-asymptotic error bound for linear system identification under nonlinear policies.

We identify linear models from nonlinear systems with initialization constraints.

Spectral decomposition of the Koopman operator is attracting attention as a tool for the analysis of nonlinear dynamical systems. Dynamic mode decomposition is a popular numerical algorithm for Koopman spectral analysis; however, we often need to prepare nonlinear observables manually according to the underlying dynami…

Smooth, globally PŁ functions are essentially nonlinear least-squares.

A new method estimates parameters of complex models using ordinary least squares.

The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…

Method estimates parameters of complex nonlinear systems.

This book introduces linear models and their theories rigorously.

Over the last decade, both the neural network and kernel adaptive filter have successfully been used for nonlinear signal processing. However, they suffer from high computational cost caused by their complex/growing network structures. In this paper, we propose two random Euler filters for complex-valued nonlinear filt…

Dual Bayesian Affine Estimators for Wiener-type state-space models

New tools in nonlinear random matrices improve understanding of the Sum of Squares hierarchy.

The past decade has witnessed a successful application of deep learning to solving many challenging problems in machine learning and artificial intelligence. However, the loss functions of deep neural networks (especially nonlinear networks) are still far from being well understood from a theoretical aspect. In this pa…

We study the problem of recovering a structured signal $\mathbf{x}_0$ from high-dimensional data $\mathbf{y}_i=f(\mathbf{a}_i^T\mathbf{x}_0)$ for some nonlinear (and potentially unknown) link function $f$, when the regressors $\mathbf{a}_i$ are iid Gaussian. Brillinger (1982) showed that ordinary least-squares estimate…

Proposes KAR for nonlinear causal discovery using kernel methods.

Develops Lagrange-Hamilton geometry for COVID-19 disease dynamics.

Stochastic VB improves nonlinear model inference speed and accuracy.

WENDy now estimates nonlinear ODEs with noisy data.

Bayesian system ID improves robustness to sparse, noisy data.

Unified theory and debiasing framework for random oblique projections in high dimensions.

This paper introduces a robust mixing model to describe hyperspectral data resulting from the mixture of several pure spectral signatures. This new model not only generalizes the commonly used linear mixing model, but also allows for possible nonlinear effects to be easily handled, relying on mild assumptions regarding…

Machine learning can improve 2SLS first stage predictions, but nonlinear methods often introduce bias.

Optimizes prediction error method for time-varying models.

Improved real-time UAV terrain following with RVM-RLS filter.

We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by $y = f( x^\top β^* ) + ε$, where $f$ is nonlinear. To recover $β^*$, we propose an $\ell_1$-regularized least-squares estimator. Unlike classical linear regression, the correspondin…

It is generally accepted that many time series of practical interest exhibit strong dependence, i.e., long memory. For such series, the sample autocorrelations decay slowly and log-log periodogram plots indicate a straight-line relationship. This necessitates a class of models for describing such behavior. A popular cl…

Develops geometry for Lotka-Volterra model of species competition.

While conformal transformations of the plane preserve Laplace's equation, Lorentz-conformal mappings preserve the wave equation. We discover how simple geometric objects, such as quadrilaterals and pairs of crossing curves, are transformed under nonlinear Lorentz-conformal mappings. Squares are transformed into curvili…

New method estimates convergence bounds for nonlinear Markov chains.

Traditionally, nonlinear inversion, direct inversion, or wave estimation methods have been used for reconstructing images from MRE displacement data. In this work, we propose a convolutional neural network architecture that can map MRE displacement data directly into elastograms, circumventing the costly and computatio…

In deep learning, \textit{depth}, as well as \textit{nonlinearity}, create non-convex loss surfaces. Then, does depth alone create bad local minima? In this paper, we prove that without nonlinearity, depth alone does not create bad local minima, although it induces non-convex loss surface. Using this insight, we greatl…

The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.

Bayesian method improves online NARMAX model identification.

This work presents a non-intrusive model reduction method to learn low-dimensional models of dynamical systems with non-polynomial nonlinear terms that are spatially local and that are given in analytic form. In contrast to state-of-the-art model reduction methods that are intrusive and thus require full knowledge of t…

In order to cope with the increased data volumes generated by modern radio interferometers such as LOFAR (Low Frequency Array) or SKA (Square Kilometre Array), fast and efficient calibration algorithms are essential. Traditional radio interferometric calibration is performed using nonlinear optimization techniques such…

RichID learns optimal control policies from nonlinear observations.