Data-driven Distributionally Robust Optimization (DD-DRO) via optimal transport has been shown to encompass a wide range of popular machine learning algorithms. The distributional uncertainty size is often shown to correspond to the regularization parameter. The type of regularization (e.g. the norm used to regularize)…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “data-driven regularization”
The Risk of Machine Learningstat.ML
Many applied settings in empirical economics involve simultaneous estimation of a large number of parameters. In particular, applied economists are often interested in estimating the effects of many-valued treatments (like teacher effects or location effects), treatment effects for many groups, and prediction models wi…
Unified framework for data-driven priors in Bayesian inverse problems
problem Bayesian inverse problems
method Unified framework using score functions
result Evaluation of four data-driven priors
Improves robustness of high-dimensional regression with rank objective and group lasso regularization.
problem Heavy-tailed noise and outliers in high-dimensional regression.
method Non-smooth Wilcoxon score based rank objective, group lasso regularization, data-driven tuning rule, proximal augmented Lagrangian method.
result Robust estimator with finite-sample error bound and efficient computational method.
This paper tackles regularization parameter learning in inverse problems using data-driven bilevel optimization.
problem Finding optimal regularization parameters in inverse problems.
method Data-driven bilevel optimization approach, analyzing performance in large data samples.
result The approach can reduce computational cost through online numerical schemes based on stochastic gradient descent.
Proposes DMOC for more nuanced neural network robustness.
problem Lipschitz continuity is too coarse for nuanced data-dependent behavior.
method Data-driven, architecture-agnostic framework based on DMOC.
result DMOC provides a finer notion of robustness relative to data distribution.
Recently, (Blanchet, Kang, and Murhy 2016, and Blanchet, and Kang 2017) showed that several machine learning algorithms, such as square-root Lasso, Support Vector Machines, and regularized logistic regression, among many others, can be represented exactly as distributionally robust optimization (DRO) problems. The dist…
Adaptive BO improves solder joint reliability by 3% with half the computational cost.
problem Improving solder joint reliability under thermomechanical loading.
method Adaptive Bayesian optimization with Gaussian process regression.
result Adaptive BO outperforms regular BO by 3% on average at any given computational budget.
In deep neural nets, lower level embedding layers account for a large portion of the total number of parameters. Tikhonov regularization, graph-based regularization, and hard parameter sharing are approaches that introduce explicit biases into training in a hope to reduce statistical complexity. Alternatively, we propo…
Physics-Driven Regularization of Deep Neural Networks for Enhanced Engineering Design and Analysiscs.LG
In this paper, we introduce a physics-driven regularization method for training of deep neural networks (DNNs) for use in engineering design and analysis problems. In particular, we focus on prediction of a physical system, for which in addition to training data, partial or complete information on a set of governing la…
New methods improve estimation of nonhomogeneous Poisson processes from limited data.
problem Estimating nonhomogeneous Poisson processes from limited data.
method Formulated as a learning generalization problem, proposed adaptive and data-driven binning methods.
result Improved estimation of nonhomogeneous Poisson processes with limited data.
New method approximates M-estimator and predictions without solving fixed-point equations.
problem Characterize behavior of M-estimator and predictions in single index models.
method Develops data-driven observable adjustments to proximal operators.
result Empirical distributions of M-estimator and predictions are approximated without solving fixed-point equations.
We consider the problem of impulse response estimation of stable linear single-input single-output systems. It is a well-studied problem where flexible non-parametric models recently offered a leap in performance compared to the classical finite-dimensional model structures. Inspired by this development and the success…
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.
Data-driven optimization improves mean-variance portfolios by penalizing norms.
problem Estimation error in mean-variance optimization.
method Augment MVO with norm penalties, use neural networks for optimization, and compute derivatives implicitly.
result Data-driven optimization reduces portfolio risk compared to standard MVO.
Many challenging image processing tasks can be described by an ill-posed linear inverse problem: deblurring, deconvolution, inpainting, compressed sensing, and superresolution all lie in this framework. Traditional inverse problem solvers minimize a cost function consisting of a data-fit term, which measures how well a…
New method uses deep learning to solve inverse problems with provable guarantees.
problem Solving inverse problems with high-quality results and provable guarantees.
method Convex-Nonconvex (CNC) framework with input weakly convex neural network (IWCNN).
result The method provides provably convergent regularization for inverse problems.
Bayesian framework for image inversion using regularization by denoising.
problem Image inversion and regularization in imaging tasks.
method Bayesian approach with Langevin-within-split Gibbs sampling.
result Demonstrates the effectiveness of the proposed method through numerical experiments.
Introduces self-regularization for analyzing learning algorithms.
problem Analyzing and optimizing learning algorithms without explicit regularization.
method Develops a self-regularization framework for learning algorithms.
result Provides statistical analysis and minmax-optimal rates for self-regularized algorithms.
Bayesian nonparametrics improves data-driven risk optimization under distributional uncertainty.
problem Improving out-of-sample performance in machine learning models due to distributional uncertainty.
method Combining Bayesian nonparametric theory and decision-theoretic preferences to propose a robust optimization criterion.
result The proposed robust optimization procedure provides favorable statistical guarantees and tractable approximations.
Paper develops a method to learn optimal sparsity-promoting regularizers for linear inverse problems.
problem Solving linear inverse problems with sparse solutions.
method Bilevel optimization framework to select an optimal synthesis operator .
result Established well-posedness and theoretical guarantees for the learning process.
Unified framework for DRO and DTA using Bayesian nonparametrics.
problem Combining DRO and DTA under ambiguity.
method Unified framework using DP and HDPs, with outlier robustness.
result Favorable performance in prediction accuracy and stability.
We consider deep feedforward neural networks with rectified linear units from a signal processing perspective. In this view, such representations mark the transition from using a single (data-driven) linear representation to utilizing a large collection of affine linear representations tailored to particular regions of…
Designs a robust data-driven decision-making model to handle multiple overfitting sources.
problem Overfitting in data-driven models due to statistical error, data noise, and data misspecification.
method Holistic distributionally robust optimization formulation combining Kullback-Leibler and Lévy-Prokhorov approaches.
result Guaranteed holistic protection against statistical error, data noise, and data misspecification.
Inverse Problems in medical imaging and computer vision are traditionally solved using purely model-based methods. Among those variational regularization models are one of the most popular approaches. We propose a new framework for applying data-driven approaches to inverse problems, using a neural network as a regular…
We present an adaptive regularization algorithm that can be effectively applied to the optimization problem in deep learning framework. Our regularization algorithm aims to take into account the fitness of data to the current state of model in the determination of regularity to achieve better generalization. The degree…
The paper uses neural networks to learn system dynamics from data with Lipschitz regularization.
problem Learning governing equations from time-sampled data.
method Lipschitz regularized deep neural networks for ODE system identification.
result Lipschitz regularization improves the smoothness and generalization of the learned function.
A framework for analyzing regularizers to ensure trustworthy theory-driven model estimation.
problem Uncertain choice of regularizers can compromise the interpretability of deep grey-box models.
method Adapting neural net architecture and training objective to analyze regularizer behavior empirically.
result Empirical analysis of regularizers helps in making a justified choice for trustworthy theory-driven model estimation.
Theoretical comparison of three invariance approaches in deep linear networks.
problem Understanding invariance in deep linear networks.
method Data augmentation, regularization, and hard-wiring approaches.
result Regularization introduces additional critical points, but they remain saddles except for the global optimum.
Method improves simulation accuracy by mitigating distribution shift in hybrid systems.
problem Mitigating distribution shift in machine-learning augmented hybrid simulation.
method Tangent-space regularized estimator to control distribution shift.
result Marked improvements in simulation accuracy, especially for systems with high distribution shift.
A neural network learns a convex regularizer for better image reconstruction.
problem Improving image reconstruction in inverse problems.
method Adversarial training of a data-adaptive ICNN as a convex regularizer.
result The convex regularizer leads to better convergence and error reduction in image reconstruction.
Spectral regularization improves learning over combinatorial spaces with limited data.
problem Learning pseudo-Boolean functions with scarce labeled data.
method Regularizing the spectral representation of learned functions using the L_1 norm.
result Regularization allows for data-frugal learning and achieves statistically optimal generalization performance.
The performance of spectral clustering can be considerably improved via regularization, as demonstrated empirically in Amini et. al (2012). Here, we provide an attempt at quantifying this improvement through theoretical analysis. Under the stochastic block model (SBM), and its extensions, previous results on spectral c…
Paper proposes DC functions for better regularization of inverse problems with theoretical guarantees.
problem Improving regularization for ill-posed inverse problems.
method Introduces difference-of-convex (DC) functions and uses them with optimization algorithms like DCA and PSM.
result DC functions yield improved performance and theoretical guarantees compared to weakly convex functions.
This study explores star-shaped regularizers learned from critic-based losses.
problem Understanding the structure of regularizers learned from critic-based losses.
method Optimizing critic-based loss functions over star-shaped regularizers.
result Derives exact expressions for optimal regularizers in certain cases.
Extends dimension reduction to data-driven settings without gradients.
problem Gradient-based dimension reduction limitations in data-driven settings.
method Score ratio matching framework, tailored parameterization, regularization, eigenvalue deflation.
result Outperforms standard score-matching for problems with low-dimensional structure.
Symmetry-regularized Neural ODEs improve model stability and interpretability.
problem Improving the stability and physical interpretability of Neural ODEs.
method Integrating Lie symmetries and conservation laws into the loss function.
result Symmetry-regularized Neural ODEs enhance model stability and interpretability.
New method clusters variables using robust nodewise regression.
problem Variable clustering in multi-factor models.
method Distributionally robust nodewise regression with convex relaxation and ADMM.
result Superior performance in numerical studies.
New method aggregates nodes in sparse graphical models.
problem Estimating edge-sparse graphical models.
method Tree-aggregated graphical lasso (tag-lasso) method.
result Aggregates nodes in a data-driven fashion using a tree.
Generative models improve MRI reconstruction by learning image structure.
problem Improving MRI image quality from undersampled data.
method Using variational autoencoders (VAEs) to learn image structure and covariance.
result The proposed method outperforms other regularization techniques on MRI datasets.
ADML combines debiased learning with data-driven model selection for efficient inference.
problem Debiased machine learning estimators can be unstable and biased in nonparametric models.
method Data-driven model selection techniques combined with debiased machine learning.
result ADML estimators yield superefficient inference for pathwise differentiable parameters.
Physics-guided neural network improves power flow analysis.
problem Infeasibility of traditional numerical approaches due to outdated or unavailable PF equations.
method Proposes a physics-guided neural network to learn PF mappings from historical data while constraining by physical laws.
result Physics-guided neural network achieves better performance and generalizability than unconstrained data-driven approaches.
New PRGP model improves traffic flow estimation.
problem Lack of models combining physics and ML for traffic flow.
method Physics regularized Gaussian process (PRGP) with discrete formulations.
result PRGP model outperforms calibrated physics models and ML methods.
A new matrix factorization method for high-dimensional data.
problem Exploiting sparse structures in complex data for better interpretability.
method Bayesian shrinkage priors and flexible sparse patterns modeled through row and column dependencies.
result Demonstrated practical advantages through simulation and soccer heatmap analysis.
DRIVE improves IV estimation by accounting for distributional uncertainties.
problem Challenges in IV estimation due to untestable model assumptions and poor finite sample properties.
method DRIVE is a distributionally robust IV estimation method that minimizes a square root TSLS objective with a Wasserstein ambiguity set.
result DRIVE achieves consistency without requiring regularization parameter to vanish, ensuring robustness to distributional uncertainties.
Correlational Lagrangian Schrödinger Bridge: Learning Dynamics with Population-Level Regularizationcs.LG
CLSB models system dynamics from cross-sectional data with population-level regularization.
problem Challenges in modeling system dynamics from limited cross-sectional samples and heterogeneous individual behaviors.
method Introduces CLSB framework for learning dynamics, regularized for population-level temporal variations.
result Empirically superior in single-cell sequencing data analyses, e.g., simulating cell development and drug response.
New insights into learning for blind inverse problems with theoretical guarantees.
problem Learning in blind inverse problems where both signal and operator are unknown.
method Data-driven approaches using Linear Minimum Mean Square Estimators (LMMSEs) with theoretical analysis.
result Established equivalences with Tikhonov-regularized formulations and derived finite-sample error bounds.
Generative adversarial network for probabilistic forecasting of random systems.
problem Forecasting random dynamical systems without distributional assumptions.
method Recurrent neural network and generative adversarial network (GAN) with regularization based on maximum mean discrepancy (MMD).
result The proposed model successfully forecasts complex stochastic processes with multiple-step predictions.