A new method for support vector regression using a data-driven insensitive parameter.
problem Determining an optimal insensitive parameter in support vector regression.
method A data-driven approach to approximate the insensitive parameter by minimizing a generalized loss function based on the likelihood principle.
result The proposed method outperforms traditional support vector regression methods and has lower computational costs.
Regression learns Mori-Zwanzig operators for dynamical systems.
problem Learning Mori-Zwanzig operators for complex dynamical systems.
method Statistical regression to extract Markov and memory operators.
result Regression models improve learning of memory-dependent corrections.
Machine learning finds new natural laws from noisy data.
problem Discovery of natural laws relies on human inspiration.
method Recursive-LASSO-based symbolic regression (RLS) method.
result Data-driven formulation of natural laws from noisy data.
ERFit identifies dynamic equations from data with minimal supervision.
problem Data-driven sparse system identification in science and engineering.
method Entropic Regression method.
result ERFit package simplifies sparse system identification for various applications.
Improves robustness of high-dimensional regression with rank objective and group lasso regularization.
problem Heavy-tailed noise and outliers in high-dimensional regression.
method Non-smooth Wilcoxon score based rank objective, group lasso regularization, data-driven tuning rule, proximal augmented Lagrangian method.
result Robust estimator with finite-sample error bound and efficient computational method.
This work evaluates and benchmarks calibration metrics for data-driven regression models.
problem Conflicting results from different calibration metrics make it hard to compare and interpret model performance.
method Systematically extracted and benchmarked 14 regression calibration metrics across various data types and recalibration methods.
result Many metrics disagree on the same recalibration result, highlighting the need for careful metric selection.
Gaussian Process Regression and Kernel Ridge Regression are popular nonparametric regression approaches. Unfortunately, they suffer from high computational complexity rendering them inapplicable to the modern massive datasets. To that end a number of approximations have been suggested, some of them allowing for a distr…
We present a regression technique for data-driven problems based on polynomial chaos expansion (PCE). PCE is a popular technique in the field of uncertainty quantification (UQ), where it is typically used to replace a runnable but expensive computational model subject to random inputs with an inexpensive-to-evaluate po…
Develops a new method for uncertainty quantification in high-dimensional learning.
problem Challenges in uncertainty quantification in high-dimensional regression or learning problems.
method Data-driven approach for UQ that corrects bias terms from training data.
result Non-asymptotic confidence intervals that avoid overestimating uncertainty.
We improve random forest consistency and performance with DMRF, a new variant.
problem Improving the consistency and performance of random forest models.
method Developed DMRF, a data-driven multinomial random forest, by modifying proof methods and improving data utilization.
result DMRF achieves strong consistency with probability 1, surpassing previous models in classification tasks.
Framework improves data-driven ROMs for complex systems using Bayesian operator inference.
problem Improving the quality of data-driven reduced-order models for complex dynamical systems.
method Develops an active learning framework using Bayesian operator inference to identify and select training parameters.
result The proposed adaptive sampling strategy consistently yields more stable and accurate ROMs than random sampling.
DDGroup identifies subgroups with uniform linear relationships.
problem Heterogeneous effects of covariates in linear models.
method Data-driven method to identify subgroups with uniform linear relationships.
result DDGroup can discover subgroups with improved performance.
In recent years, data-driven methods have been developed to learn dynamical systems and partial differential equations (PDE). The goal of such work is discovering unknown physics and the corresponding equations. However, prior to achieving this goal, major challenges remain to be resolved, including learning PDE under …
Dividing local Gaussian processes improve real-time prediction efficiency.
problem Efficient online prediction for large data sets.
method Iterative data-driven division of input space for sublinear computational complexity.
result Sublinear computational complexity in real-time prediction.
The study explores how machine learning can enhance scientific research.
problem Improving scientific models with machine learning.
method Analysis of data-driven models versus manually added variables in regression.
result Complex models may not always improve over simpler ones in scientific contexts.
Tuning parameter selection is of critical importance for kernel ridge regression. To this date, data driven tuning method for divide-and-conquer kernel ridge regression (d-KRR) has been lacking in the literature, which limits the applicability of d-KRR for large data sets. In this paper, by modifying the Generalized Cr…
Data-driven models analyze power grids under incomplete physical information, and their accuracy has been mostly validated empirically using certain training and testing datasets. This paper explores error bounds for data-driven models under all possible training and testing scenarios, and proposes an evaluation implem…
LGD algorithm learns optimal hyperparameters for regression tasks.
problem Learning to learn hyperparameters for regression problems.
method Langevin Gradient Descent (LGD) approximates posterior distribution.
result Meta-learning optimal hyperparameters achieves Bayes' optimal solution.
Deep-HGP uses Bayesian nonparametric approach for complex data regression.
problem Complex data regression with compositional structures.
method Deep Gaussian processes with a squared-exponential kernel, data-driven lengthscale parameters.
result Posterior distribution optimally recovers unknown true regression curve in terms of quadratic loss.
New method clusters variables using robust nodewise regression.
problem Variable clustering in multi-factor models.
method Distributionally robust nodewise regression with convex relaxation and ADMM.
result Superior performance in numerical studies.
Local laGPR speeds up multiscale mechanics simulations without neural networks.
problem High computational costs in multiscale mechanics simulations.
method Local approximate Gaussian process regression (laGPR) combined with FE schemes.
result laGPR offers better accuracy than neural networks for stress predictions.
Symbolic regression finds simple formulas for implied volatility.
problem Discovering accurate parametric representations for implied volatility.
method Symbolic regression to find analytic formulas from market data.
result Symbolic regression identifies compact parametrizations with competitive fitting performance.
A new method assesses regression models' global optimality.
problem Challenges in evaluating regression models without access to true data.
method Information Teacher framework based on Shannon mutual information.
result Demonstrates capability to detect global optimality.
Fair ML systems can be safe ML systems by considering uncertainty.
problem Safety of data-driven decision systems is often neglected.
method Viewing ML systems as socio-technical, uncertainty-aware modeling.
result Fair models should be uncertainty-aware, e.g. through distributional regression.
We study nonlinear regression of real valued data in an individual sequence manner, where we provide results that are guaranteed to hold without any statistical assumptions. We address the convergence and undertraining issues of conventional nonlinear regression methods and introduce an algorithm that elegantly mitigat…
Three physics-constrained regression exercises for image velocimetry and turbulence modeling.
problem Image velocimetry and turbulence modeling challenges.
method Physics-constrained regression exercises implemented as toy problems.
result Python codes provided for all exercises.
ADML combines debiased learning with data-driven model selection for efficient inference.
problem Debiased machine learning estimators can be unstable and biased in nonparametric models.
method Data-driven model selection techniques combined with debiased machine learning.
result ADML estimators yield superefficient inference for pathwise differentiable parameters.
Study integrates machine learning with SAA for optimizing decisions based on uncertain parameters and covariates.
problem Optimizing decisions under uncertain parameters and covariates.
method Data-driven frameworks integrating machine learning prediction models within SAA for scenario generation.
result Consistent and asymptotically optimal solutions under certain conditions, with finite sample guarantees.
Data-driven approaches, most prominently deep learning, have become powerful tools for prediction in many domains. A natural question to ask is whether data-driven methods could also be used to predict global weather patterns days in advance. First studies show promise but the lack of a common dataset and evaluation me…
New method refines model predictions as design evolves.
problem Designing objects with desired properties using data-driven methods.
method Formalized as a game, developed autofocusing strategy for model retraining.
result Autofocusing improves model predictions in design space.
The paper tackles data-driven optimal control of unknown nonlinear systems using RKHS.
problem Unknown nonlinear dynamics and stage cost functions.
method Embed state densities into RKHS, learn Markov operators, solve Hamilton-Jacobi-Bellman recursions.
result Solves a wide range of nonlinear control problems, including depth regulation.
Method estimates treatment effects with continuous values, correcting for confounding.
problem Estimating treatment effects with continuous values, dealing with confounding.
method Two-stage kernel ridge regression: first stage learns response, second stage corrects for distribution shift.
result Optimal learning bounds achieved without estimating treatment density, adapts to unknown overlap and kernel spectral decay.
New algorithm tackles regression on manifold data using diffusion and semi-supervised learning.
problem Regression on high-dimensional manifold data with complex structures.
method Diffusion-based spectral algorithm using graph Laplacian and heat kernel.
result Algorithm achieves convergence rate dependent on intrinsic manifold dimension, avoiding curse of dimensionality.
This paper tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression, spline smoothing or locally weighted regression, and the choice of a kernel in multiple kern…
Enhances random forest consistency and introduces DMRF for improved performance.
problem Improving the consistency and efficiency of random forest algorithms.
method Strengthened proof methods and propose DMRF algorithm.
result DMRF achieves better theoretical and experimental performance than previous variants.
Recently, (Blanchet, Kang, and Murhy 2016, and Blanchet, and Kang 2017) showed that several machine learning algorithms, such as square-root Lasso, Support Vector Machines, and regularized logistic regression, among many others, can be represented exactly as distributionally robust optimization (DRO) problems. The dist…
Framework learns physics-informed continuum models from molecular data.
problem Discovering accurate and robust data-driven continuum models from molecular simulation data.
method Operator regression framework using neural networks in modal space with physical inductive biases.
result Learned operators generalize to unseen system characteristics.
Automated digital twin discovery from biological data improves drug discovery and personalized medicine.
problem Developing reliable digital twins from noisy, incomplete biological data.
method Symbolic and sparse regression, Bayesian frameworks, deep learning, and large language models.
result Sparse regression generally outperforms symbolic regression, especially with Bayesian frameworks.
This chapter reviews classic regression methods and their evolution to physics-informed approaches.
problem Finding relationships between variables using regression.
method Introduces traditional and physics-informed regression methods, linking them to computational science.
result Regression methods have evolved from purely statistical to incorporating physical knowledge.
New method quantifies uncertainty in distributed regression.
problem Large datasets make traditional regression techniques ineffective.
method Data-driven approach to uncertainty quantification for averaged estimator.
result Rigorous theoretical guarantees for sup-norm consistency.
Self-test loss functions improve data-driven modeling of weak-form operators and gradient flows.
problem Challenges in selecting test functions for data-driven modeling involving weak-form operators and gradient flows.
method Introducing self-test loss functions that depend on unknown parameters and are quadratic.
result Self-test loss functions conserve energy for gradient flows and coincide with log-likelihood ratios for stochastic differential equations.
Study compares data-driven vs model-based MRS quantification strategies, focusing on resilience to out-of-distribution effects.
problem Resilience to out-of-distribution effects in data-driven MRS quantification.
method Compared three data-driven strategies (supervised regression, self-supervised learning, test-time adaptation) against model-based fitting tools.
result Test-time adaptation proved most resilient to out-of-distribution effects, while self-supervised learning achieved intermediate performance.
A new line search rule improves support recovery in high-dimensional data.
problem Support recovery in high-dimensional data analysis with ℓ0 penalty. method Data-driven line search rule for adaptive step size determination.
result Proves ℓ2 error bound without restrictions on cost functional. This work improves SINDy-type algorithms for system identification using score-guided dictionary selection.
problem Improving accuracy and interpretability in dynamical system identification.
method Score-guided library selection to refine dictionary terms in sparse regression.
result Score-guided methods enhance SINDy's robustness in discovering governing equations.
Random Forest kernels improve performance in various regression and survival tasks.
problem Improving performance of Random Forest in high-dimensional data with noisy features.
method Developed and evaluated data-driven RF kernels for regression, classification, and survival tasks.
result RF kernels are competitive or superior to RF in most scenarios, especially for survival tasks.
Machine learning improves combustion system predictions by integrating physical models.
problem Improving accuracy of complex multi-physics systems like combustion.
method Coupling machine learning algorithms with physical models and constraints.
result Enhanced predictive capabilities in turbulent combustion.
Discovering quasipotential equations from data using machine learning.
problem Understanding escape mechanisms from metastable states in nonlinear systems.
method Combining neural networks and sparse regression to symbolically reconstruct quasipotential equations.
result Model-unbiased analytical forms of quasipotential discovered directly from data.
Bayesian nonparametrics improves data-driven risk optimization under distributional uncertainty.
problem Improving out-of-sample performance in machine learning models due to distributional uncertainty.
method Combining Bayesian nonparametric theory and decision-theoretic preferences to propose a robust optimization criterion.
result The proposed robust optimization procedure provides favorable statistical guarantees and tractable approximations.