The paper studies convergence rates of Tsallis entropic regularization in optimal transport.
problem Optimal transport with regularization.
method Γ-convergence and quantization/shadow arguments.
result Derives convergence rate of Tsallis entropic regularization.
Improved optimal regularity for harmonic almost complex structures.
problem Establishing optimal regularity for harmonic almost complex structures.
method Quantitative stratification method and rectifiability of singular strata.
result Optimal regularity theory for energy minimizing harmonic almost complex structures.
The paper explores optimal regularizers for data sources, linking them to star bodies.
problem Understanding optimal regularizers for data sources.
method Investigates optimal regularizers for data distributions using star bodies and dual Brunn-Minkowski theory.
result Identifies optimal regularizers and assesses amenability to convex regularization.
Study on convergence rates for optimal transport with regularization.
problem Convergence analysis of divergence-regularized optimal transport.
method Novel methodology using quantization and martingale couplings.
result Sharp rates for various divergences and transport costs.
Study on regularity of optimal transport maps on convex domains with quadratic cost.
problem Regularity of optimal transport maps between convex domains with quadratic cost.
method Analysis of C α C^α C α -densities and C 1 , α C^{1, α} C 1 , α boundary conditions, monotonicity formula for optimal transport maps. result Proves C 1 , 1 − ε C^{1, 1-\varepsilon} C 1 , 1 − ε -regularity for nondegenerate C α C^α C α -densities and C 2 , α C^{2, α} C 2 , α -regularity for C 1 , α C^{1, α} C 1 , α boundary. Adaptive reduction scheme approximates optimal policy in regularized MDPs.
problem Finding near optimal policy in regularized MDPs with biased solutions.
method Adaptive reduction of regularization parameter λ to approximate optimal policy.
result Iteration complexity reduced for obtaining ε-optimal policy.
Extends optimal regularity and compactness to vector bundles over non-Riemannian manifolds.
problem Optimal regularity and compactness for connections on vector bundles.
method Derive RT-equations, establish existence theory, handle curvature up to L 1 L^1 L 1 . result Optimal regularity and compactness extended to vector bundles over non-Riemannian manifolds.
Study optimal ridge regularization for out-of-distribution prediction.
problem Optimal ridge regularization for predicting out-of-distribution data.
method Established conditions for optimal regularization under covariate and regression shifts, proving monotonic risk in data aspect ratio.
result Negative regularization can be optimal under shifts, even with isotropic or underparameterized training features.
We propose and study a general framework for regularized Markov decision processes (MDPs) where the goal is to find an optimal policy that maximizes the expected discounted total reward plus a policy regularization term. The extant entropy-regularized MDPs can be cast into our framework. Moreover, under our framework, …
Tikhonov regularization is robust under specific martingale constraints in distributionally robust optimization.
problem Distributionally robust optimization and regularization of learning models.
method Optimal transport approach with martingale constraints.
result Tikhonov regularization is optimal transport robust under specified martingale constraints.
Optimizes biharmonic map regularity using stratification methods.
problem Improving the known almost optimal regularity of biharmonic maps.
method Quantitative stratification method.
result Optimal regularity results for minimizing biharmonic maps.
New framework for robust regularization under uncertain data distributions.
problem Addressing ill-posed inverse problems and statistical estimation under distributional uncertainty.
method Distributionally robust optimal regularization using convex duality.
result Identifies robust regularizers that remain effective under data distributional perturbations.
Study optimal transport on simplex boundary, proving transport map and potential regularity.
problem Regularity of transport map and potential on simplex boundary.
method Boundary regularity results for optimal transport maps, exploiting simplex symmetries.
result Regularity properties of transport map and its convex potential.
New method averages SGD iterates to achieve adjustable regularization.
problem Overfitting in machine learning models.
method Averaging SGD iterates for regularized solutions.
result Obtain regularized solutions without tuning parameters.
Study uses property elicitation to understand how fairness regularizers affect optimal decisions.
problem Understanding how fairness regularizers change the optimal decision in predictive algorithms.
method Property elicitation to analyze the relationship between loss, regularization, and optimal decision.
result Necessary and sufficient condition for when a property changes with the addition of a regularizer.
Proposes reducing random error in stochastic optimization by variance regularization.
problem Random error accumulation in stochastic optimization algorithms.
method Regularizes learning-rate based on mini-batch variances.
result Speeds up convergence and stabilizes stochastic optimization.
Paper develops a method to learn optimal sparsity-promoting regularizers for linear inverse problems.
problem Solving linear inverse problems with sparse solutions.
method Bilevel optimization framework to select an optimal synthesis operator B B B . result Established well-posedness and theoretical guarantees for the learning process.
Entropy regularized algorithms such as Soft Q-learning and Soft Actor-Critic, recently showed state-of-the-art performance on a number of challenging reinforcement learning (RL) tasks. The regularized formulation modifies the standard RL objective and thus generally converges to a policy different from the optimal gree…
Weight decay is one of the standard tricks in the neural network toolbox, but the reasons for its regularization effect are poorly understood, and recent results have cast doubt on the traditional interpretation in terms of L 2 L_2 L 2 regularization. Literal weight decay has been shown to outperform L 2 L_2 L 2 regularization for…
The optimization of the variance supplemented by a budget constraint and an asymmetric ℓ 1 \ell_1 ℓ 1 regularizer is carried out analytically by the replica method borrowed from the theory of disordered systems. The asymmetric regularizer allows us to penalize short and long positions differently, so the present treatment in…
This paper explores how entropic regularization improves Wasserstein estimators' performance.
problem Improving the approximation and estimation properties of Wasserstein estimators.
method Entropic regularization of optimal transport costs to smooth Wasserstein estimators.
result Entropic regularization can achieve comparable statistical performance to un-regularized estimators at lower computational cost.
New approach to portfolio optimization shows entropy regularization is ineffective.
problem Entropy regularization in mean-variance portfolio optimization under drift uncertainty.
method Combining Bayesian filtering and stochastic policy optimization.
result Entropy regularization does not accelerate learning about unknown drift.
Paper introduces robust market making using Wasserstein distance and entropy regularization.
problem Market making robustness under uncertainty.
method Wasserstein distance, entropy regularization, convex optimization, optimal radius selection.
result The robust market making problem can be reformulated as a convex optimization problem.
Study MinMax methods for optimization problems, including optimal transport.
problem Optimization problems, especially optimal transport.
method MinMax framework, regularization, neural networks, approximation theorems.
result Justification of neural networks for solving optimization problems.
Algorithm finds optimal regularizers for online linear optimization.
problem Finding optimal regularizers to minimize regret in online linear optimization.
method Algorithm takes input sets and outputs an optimal regularizer for FTRL.
result Algorithm guarantees regret within a constant factor of the best possible learning algorithm.
Unified framework for understanding and optimizing training acceleration.
problem Challenges in optimizing training with regularization and acceleration techniques.
method Explains how AdaGrad, RMSProp, and Adam accelerate training, and derives a generalization for L 1 L_1 L 1 -regularization. result Derives a unified mathematical framework for understanding and optimizing training acceleration.
Regularized policies are robust to adversarial rewards.
problem Understanding the effects of regularization on policy exploration and robustness.
method Using Fenchel duality to derive the dual problem of the regularized RL objective, showing the optimal policy is robust to adversarial rewards.
result Regularized policies are optimal for a reinforcement learning problem under adversarial reward conditions.
Regularization can improve machine learning models' robustness against poisoning attacks.
problem Poisoning attacks degrade machine learning models' performance by manipulating a fraction of the training data.
method Proposes a novel optimal attack formulation considering the effect of hyperparameters on regularization, leading to better evaluation of robustness.
result Demonstrates that L 2 L_2 L 2 regularization can help mitigate the impact of poisoning attacks. Optimal ridge regularization computed iteratively from generative parameters.
problem Finding the optimal ridge regularization strength for linear regression.
method Iterative procedure to compute optimal regularization strength numerically.
result The proposed procedure attains near-optimal generalization across various conditions.
Proves optimal regularity for sphere minimizers in 3-sphere.
problem Finding optimal regularity for sphere minimizers.
method Proves C 1 , 1 C^{1,1} C 1 , 1 regularity for minimizers of prescribed mean curvature over isotopy classes. result Proves optimal C 1 , 1 C^{1,1} C 1 , 1 regularity for minimizers. Path regularization reveals convex optimization in deep ReLU networks.
problem Understanding the optimization landscape of deep neural networks.
method Introducing path regularization to make the training problem convex and sparsity-inducing.
result Path regularized parallel ReLU networks are a parsimonious convex model in high dimensions.
PAR provides a flexible framework for quantization in optimization problems.
problem Challenges in optimization problems over discrete or quantized variables.
method Piecewise-affine regularization (PAR) for modeling and computational optimization.
result PAR-regularized loss functions exhibit high quantization at critical points in the overparameterized regime.
In this paper we combine two important extensions of ordinary least squares regression: regularization and optimal scaling. Optimal scaling (sometimes also called optimal scoring) has originally been developed for categorical data, and the process finds quantifications for the categories that are optimal for the regres…
DRAG decreases regularization to accelerate semi-discrete OT convergence.
problem Mitigating bias in semi-discrete OT problems with entropic regularization.
method DRAG: Decreasing Regularization Averaged Gradient, a stochastic gradient descent algorithm.
result DRAG achieves unbiased O ( 1 / t ) \mathcal{O}(1/t) O ( 1/ t ) sample and iteration complexity for OT cost and potential estimation, and O ( 1 / t ) \mathcal{O}(1/\sqrt{t}) O ( 1/ t ) rate for OT map. Many machine learning algorithms minimize a regularized risk, and stochastic optimization is widely used for this task. When working with massive data, it is desirable to perform stochastic optimization in parallel. Unfortunately, many existing stochastic optimization algorithms cannot be parallelized efficiently. In t…
Trajectory optimization using a learned model of the environment is one of the core elements of model-based reinforcement learning. This procedure often suffers from exploiting inaccuracies of the learned model. We propose to regularize trajectory optimization by means of a denoising autoencoder that is trained on the …
We present an adaptive regularization algorithm that can be effectively applied to the optimization problem in deep learning framework. Our regularization algorithm aims to take into account the fitness of data to the current state of model in the determination of regularity to achieve better generalization. The degree…
Develops regularity theory for Beckmann's optimal transport problem.
problem Minimizing total squared flux in continuous transport from source to target.
method Unconstrained Lagrangian formulation, variational first order optimality conditions, Schauder estimates.
result Exact Hölder regularity of potential, flux, and flow generating on bounded, regular domains.
Introduces self-regularization for analyzing learning algorithms.
problem Analyzing and optimizing learning algorithms without explicit regularization.
method Develops a self-regularization framework for learning algorithms.
result Provides statistical analysis and minmax-optimal rates for self-regularized algorithms.
Paper analyzes sample complexity of offline MABs with KL regularization.
problem Optimizing sample complexity for offline decision-making with KL-regularized metrics.
method Sharp analysis of KL-PCB, providing upper and lower bounds.
result Characterizes sample complexity for offline MABs with KL regularization.
New method smooths optimization for sparse regularization.
problem Non-smooth, non-convex optimization problems for sparsity.
method Overparameterization and smooth surrogate penalties.
result Surrogate objective has identical global and local minima.
The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a …
Study optimal transport on globally hyperbolic spacetimes, focusing on weak Kantorovich potentials' regularity.
problem Investigate regularity of weak Kantorovich potentials on globally hyperbolic spacetimes.
method Apply insights from Riemannian and Lorentzian cases to study π π π -solutions. result Conclude existence, uniqueness, and structure of optimal transport maps.
A new model corrects inhomogeneity in Optimal Transport with Boundary.
problem Inhomogeneity in UROT models for Optimal Transport with Boundary.
method Proposed a modified entropic regularization term to make UROT models homogeneous.
result Homogeneous UROT model preserves properties of standard UROT while correcting inhomogeneity.
New regularizer for machine learning using private data.
problem Machine learning with private data.
method Distributionally-robust optimization with locally-differentially-private datasets.
result New regularizer for training linear regression models.
We propose a general framework for entropy-regularized average-reward reinforcement learning in Markov decision processes (MDPs). Our approach is based on extending the linear-programming formulation of policy optimization in MDPs to accommodate convex regularization functions. Our key result is showing that using the …
New algorithm stabilizes bi-level hyperparameter optimization.
problem Stability issues in bi-level hyperparameter optimization.
method Uses Moreau-Yosida regularization to stabilize convergence.
result Significant improvement in loss values with fixed computation budget.
Choquet regularization improves exploration in RL.
problem Improving exploration in reinforcement learning.
method Introducing Choquet regularizers to measure and manage exploration, reformulating RL problems and deriving explicit solutions.
result Explicit optimal distributions and Choquet regularizers for various exploratory samplers.