Regression learns Mori-Zwanzig operators for dynamical systems.
problem Learning Mori-Zwanzig operators for complex dynamical systems.
method Statistical regression to extract Markov and memory operators.
result Regression models improve learning of memory-dependent corrections.
The paper reformulates regression in infinite dimensions as an inverse problem, showing it's equivalent to compact inverse problems.
problem Learning a linear operator between Hilbert spaces from empirical observations.
method Reformulates regression as an inverse problem, proving equivalence to compact inverse problems under specific conditions.
result The inverse problem is equivalent to compact inverse problems in terms of spectral properties and regularisation theory.
Randomized algorithm solves vector-valued regression problems with low-rank operators.
problem Vector-valued regression problems involving infinite-dimensional spaces.
method Randomized Reduced Rank Regression (R4) using Gaussian sketching for optimization.
result R4 estimators are efficient and accurate, with empirical risk close to optimal.
This note introduces a regression technique for finding a class of nonlinear integro-differential operators from data. The method parametrizes the spatial operator with neural networks and Fourier transforms such that it can fit a class of nonlinear operators without needing a library of a priori selected operators. We…
Develops a new method for functional regression that works with non-Gaussian data.
problem Limited models for regression in function spaces with Gaussian process priors.
method Introduces Neural Operator Flows (OpFlow) for non-Gaussian function spaces.
result OpFlow enables robust and accurate uncertainty quantification for functional regression.
Short proof shows how ridge regression works with random data.
problem Understanding prediction error in ridge regression with random design.
method Combination of exchangeability arguments, matrix perturbation, and operator convexity.
result Elementary proof of prediction error without complex inequalities.
The report analyzes infinite-dimensional output space regression.
problem Learning theory in vector-valued RKHS regression.
method Integral operator technique with spectral theory for non-compact operators.
result Results with minimal assumptions using Chebyshev's inequality.
Paper introduces a method for operator learning using random features.
problem Estimating maps between infinite-dimensional spaces using input-output pairs.
method Function-valued random features method, building a linear combination of random operators.
result The method provides convergence guarantees and error bounds for nonlinear problems.
Most of real-world graphs are dynamic, i.e., they change over time by a sequence of update operations. While the regression problem has been studied for static graphs and temporal graphs, it is not investigated for general dynamic graphs. In this paper, we study regression over dynamic graphs. First, we present the not…
New method bounds high-dimensional regression without estimating design covariance.
problem High-dimensional linear regression with random design.
method Error-in-operator approach that incorporates design covariance into empirical risk minimization.
result Dimension-free bounds on excess prediction risk derived.
Efficiently predicts long-time dynamics of quantum spin models using MLP regression.
problem Challenges in calculating long-time expectation values for quantum spin models.
method Utilized a multi-layer perceptron (MLP) model for regression on matrix product states (MPS) expectation values.
result Significantly reduced computational cost for generating long-time dynamics while maintaining high accuracy.
Optimal multiscale learning of linear operators
problem Statistical and computational limits of learning bounded linear operators between Sobolev spaces
method Reformulate as an infinite-dimensional matrix regression problem with heterogeneous multiscale structure
result Establish minimax rates and construct a finite-resolution blockwise least-squares estimator attaining these rates
We study the stability properties of nonlinear multi-task regression in reproducing Hilbert spaces with operator-valued kernels. Such kernels, a.k.a. multi-task kernels, are appropriate for learning prob- lems with nonscalar outputs like multi-task learning and structured out- put prediction. We show that multi-task ke…
This paper studies robust regression in the settings of Huber's ε-contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of ε-contamination models for various regression problems including nonpa…
Zoetrope Genetic Programming improves symbolic regression performance.
problem Evolutionary symbolic regression for complex mathematical expressions.
method Zoetropic representation, repeated fusion operations, linear combination, crossover, mutation, selection.
result Zoetrope Genetic Programming achieves state-of-the-art performance and low computational time.
Two methods solve kernel ridge regression problems efficiently.
problem Solving kernel ridge regression problems with large datasets.
method RPCholesky and KRILL preconditioning techniques.
result Efficient solutions to KRR problems with strong guarantees.
In this work, we develop a simple algorithm for semi-supervised regression. The key idea is to use the top eigenfunctions of integral operator derived from both labeled and unlabeled examples as the basis functions and learn the prediction function by a simple linear regression. We show that under appropriate assumptio…
Operator learning approximates complex mappings for PDEs and experimental data.
problem Approximating mappings between infinite-dimensional function spaces for scientific computing.
method Formalizing operator learning as function-to-function regression and incorporating physical constraints.
result Development of rigorous uncertainty quantification frameworks for operator learning.
MetaNOR learns common nonlocal kernels for efficient metamaterial modeling.
problem Efficiently modeling wave propagation in new metamaterials.
method Meta-learns a common nonlocal kernel from existing tasks and transfers this knowledge to new tasks with minimal data.
result Substantial improvements in sampling efficiency for new metamaterials.
Physics-informed GP regression solves eigenvalue problems by identifying non-trivial eigenspaces.
problem Solving eigenvalue problems of linear operators with trivial solutions.
method Constructing a transfer function-type indicator using physics-informed Gaussian Process posterior.
result The posterior covariance is non-trivial only for eigenvalues of the operator, indicating non-trivial eigenspaces.
New EiV models correct bias in operator learning with noisy data.
problem Bias in operator learning due to noisy independent variables.
method Developed EiV models for MOR-Physics and DeepONet.
result EiV models reduce bias in noisy operator learning.
Positive definite operator-valued kernels generalize the well-known notion of reproducing kernels, and are naturally adapted to multi-output learning situations. This paper addresses the problem of learning a finite linear combination of infinite-dimensional operator-valued kernels which are suitable for extending func…
The paper tackles domain generalization using functional regression.
problem Learning a model that generalizes well across different source distributions.
method Functional regression approach to learn a linear operator between marginal and conditional distributions.
result The proposed algorithm achieves finite sample error bounds for the idealized risk.
New method speeds up NIR spectroscopy calibration by 400x.
problem Efficient preprocessing selection in NIR spectroscopy.
method Operator-adaptive PLS and Ridge regression.
result Significant reduction in fitting time with comparable prediction quality.
Metrics assess uncertainty structure and distribution for regression models.
problem Quantifying uncertainty in high-dimensional and nonlinear regression tasks.
method Two bounded comparison metrics for uncertainty structure and distribution.
result DNNs and DNOs provide encouraging uncertainty metric values in high dimensions.
Researchers develop a method to control nonlinear systems with Koopman operator regression.
problem Controlling nonlinear systems with finite action spaces.
method Koopman operator regression for dynamics estimation and model predictive control for control.
result The method yields a linear switching predictive model for control.
New kernels allow learning from non-separable data.
problem Learning from non-separable data.
method Introducing entangled kernels and a two-step algorithm.
result Efficient algorithm for learning entangled kernels.
New method assesses prediction intervals across different operating points.
problem Difficulty in comparing prediction intervals across studies.
method Operating characteristics curves and gain over a simple reference.
result A novel operating point agnostic assessment methodology for prediction intervals.
Study learns convolution operators on compact Abelian groups using regularization.
problem Learning convolution operators on compact Abelian groups.
method Regularization-based approach with ridge regression estimator.
result Characterizes the accuracy of the estimator in terms of finite sample bounds.
Derives adjoint formulas for matrix operations and applies them to specific cases.
problem Computing adjoints for matrix operations and specific matrix types.
method Derives adjoint formulas for matrix operations and applies them to specific cases.
result Closed-form expressions for adjoints in specific matrix types.
Accelerates GPR with localized kernels for faster performance.
problem Speeding up Gaussian process regression.
method Localization kernels applied at each data point to down-weight distant points, leading to a sparsified Gram matrix.
result Significant speedups with competitive performance compared to other methods.
In this paper, we consider the nonparametric least square regression in a Reproducing Kernel Hilbert Space (RKHS). We propose a new randomized algorithm that has optimal generalization error bounds with respect to the square loss, closing a long-standing gap between upper and lower bounds. Moreover, we show that our al…
Transformers learn functionals from distributions without losing information.
problem Lack of rigorous mathematical theory supporting Transformer performance.
method Proposed a Transformer learning framework, attention operator, and distribution regression.
result Transformers can compress distributions into function representations without loss of information.
Bayesian method models multivalued power data from wind farms.
problem Accurate modeling of power curves with multivalued relationships.
method Overlapping mixture of probabilistic regression models.
result Model accurately represents practical power data.
Paper introduces ℓ-DER for regression tasks using morphological operators and convex-concave procedure.
problem Developing a universal approximator for regression tasks.
method Introduces ℓ-DER model, trains it using a convex-concave procedure (CCP) to minimize least-squares. result Outperforms other hybrid morphological models and state-of-the-art approaches.
A neural network solves logistic regression with ℓ1 regularization efficiently.
problem Efficiently solving logistic regression with ℓ1 regularization due to non-differentiability of ℓ1 norm. method A simple projection neural network that avoids auxiliary variables and smooth approximations.
result The neural network converges to a solution of the problem with any initial value and outperforms existing methods.
Modified relative universality for unbiasedness and consistency in dimension reduction.
problem Gap in proof of unbiasedness and Fisher consistency in relative universality.
method Modified definition of relative universality using ǫ-measurability.
result Established unbiasedness and Fisher consistency rigorously.
This paper analyzes divide-and-conquer estimators for functional linear regression without assuming target function in the RKHS.
problem Functional linear regression without target function in RKHS.
method Integral operator approach to establish upper bounds and prove asymptotic optimality.
result Sharp finite sample upper bounds and asymptotic optimality of divide-and-conquer estimators.
Modeling dynamical systems with ordinary differential equations implies a mechanistic view of the process underlying the dynamics. However in many cases, this knowledge is not available. To overcome this issue, we introduce a general framework for nonparametric ODE models using penalized regression in Reproducing Kerne…
Self-test loss functions improve data-driven modeling of weak-form operators and gradient flows.
problem Challenges in selecting test functions for data-driven modeling involving weak-form operators and gradient flows.
method Introducing self-test loss functions that depend on unknown parameters and are quadratic.
result Self-test loss functions conserve energy for gradient flows and coincide with log-likelihood ratios for stochastic differential equations.
We consider the dynamic linear regression problem, where the predictor vector may vary with time. This problem can be modeled as a linear dynamical system, with non-constant observation operator, where the parameters that need to be learned are the variance of both the process noise and the observation noise. While var…
Ability for accurate hospital case cost modelling and prediction is critical for efficient health care financial management and budgetary planning. A variety of regression machine learning algorithms are known to be effective for health care cost predictions. The purpose of this experiment was to build an Azure Machine…
We study pathwise invariances of centred random fields that can be controlled through the covariance. A result involving composition operators is obtained in second-order settings, and we show that various path properties including additivity boil down to invariances of the covariance kernel. These results are extended…
Corrected moment-based methods improve inference in topic model regression.
problem Inferential difficulties in topic model plug-in workflow for regression.
method Corrected spectral moment methods for LDA, response-weighted word moments.
result Direct identification of regression coefficients without estimating topic shares.
New methods create counterfactuals for image regression models.
problem Creating interpretable explanations for regression models in images.
method Two methods using diffusion-based generative models to create counterfactuals.
result Diffusion-based methods produce realistic, semantic, and smooth counterfactuals.
Adapts EGOP to multi-class setting and proposes a simple rough estimator.
problem Recovering relevant directions for multi-class regression.
method Adapt EGOP to multi-class setting, propose a simple rough estimator.
result Simple rough estimator of EJOP remains statistically consistent.
Framework learns physics-informed continuum models from molecular data.
problem Discovering accurate and robust data-driven continuum models from molecular simulation data.
method Operator regression framework using neural networks in modal space with physical inductive biases.
result Learned operators generalize to unseen system characteristics.
Ridge regression performs optimally in noisy environments with heavy-tailed distributions.
problem Performance of ridge regression in noisy environments with heavy-tailed noise.
method Established excess risk bounds using integral operator framework and Fuk-Nagaev inequality.
result Ridge regression achieves optimal convergence rates under heavy-tailed noise, demonstrating robustness.