Paper develops a method to learn optimal sparsity-promoting regularizers for linear inverse problems.
problem Solving linear inverse problems with sparse solutions.
method Bilevel optimization framework to select an optimal synthesis operator B. result Established well-posedness and theoretical guarantees for the learning process.
Uniform small energy regularity for fractional geometric problems proved.
problem Proving regularity for fractional geometric problems.
method Analyzing parabolic boundary reaction Ginzburg-Landau problems and fractional harmonic maps to spheres.
result Uniform small energy regularity results for s∈(0,1), answering a posed question. New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.
problem Comparative analysis of regularization norms in ill-posed problems.
method Small noise analysis framework for Tikhonov and RKHS regularizations.
result Optimal convergence rates achieved with adaptive fractional RKHS, but hyper-parameters decay too fast.
A fast sketching algorithm solves regularized least squares problems efficiently.
problem Solving large-scale optimization problems with convex or nonconvex regularization.
method Sketching for Regularized Optimization (SRO) algorithm that generates a sketch of the original data matrix and solves the sketched problem.
result General theoretical results for the approximation error between the original and sketched problems, including minimax rates for sparse signal estimation.
Proves C1,1 regularity for multiple membrane solutions.
problem Stationary C1,α solutions to the multiple membrane problem. method Uses C1,1-regularity estimate. result Proves C1,1 regularity for stationary solutions. Inverse problems arise in a number of domains such as medical imaging, remote sensing, and many more, relying on the use of advanced signal and image processing approaches -- such as sparsity-driven techniques -- to determine their solution. This paper instead studies the use of deep learning approaches to approximate …
Rapidly growing product lines and services require a finer-granularity forecast that considers geographic locales. However the open question remains, how to assess the quality of a spatio-temporal forecast? In this manuscript we introduce a metric to evaluate spatio-temporal forecasts. This metric is based on an Opti- …
The paper analyzes reg-SGD for convex problems, proving convergence and quantifying the rate of convergence.
problem Minimizing convex, L-smooth functions in a Hilbert space.
method Regularized stochastic gradient descent with decaying regularization.
result Strong convergence to the minimum-norm solution without boundedness assumptions.
Solving l1 regularized optimization problems is common in the fields of computational biology, signal processing and machine learning. Such l1 regularization is utilized to find sparse minimizers of convex functions. A well-known example is the LASSO problem, where the l1 norm regularizes a quadratic function. A multil…
New iterative regularization method tackles non-smooth, non-strongly convex functionals.
problem Tackles non-smooth, non-strongly convex functionals in regularization problems.
method Primal-dual algorithm with convergence and stability analysis.
result First iterative regularization procedure for non-smooth, non-strongly convex functionals.
Paper proves smoothness of solutions to a complex geometric problem.
problem Smoothness of solutions to the degenerate Lp Dual Minkowski problem. method Inspired by Guan and Li's approach for the Aleksandrov problem, the authors derive C1,1 estimates. result Proves solutions are C1,1 regular. New regularization method reduces support of empirical risk minimization solutions.
problem Regularization in empirical risk minimization with relative entropy.
method Introduces Type-II regularization, characterizes solutions, analyzes properties of relative entropy.
result Type-II regularization collapses solution support into reference measure's support.
Choquet regularization improves exploration in RL.
problem Improving exploration in reinforcement learning.
method Introducing Choquet regularizers to measure and manage exploration, reformulating RL problems and deriving explicit solutions.
result Explicit optimal distributions and Choquet regularizers for various exploratory samplers.
The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a …
Study on geometric variational problems for existence, regularity, and uniqueness of solutions.
problem Geometric variational problems, focusing on existence, regularity, and uniqueness of solutions.
method Formulated in Federer and Fleming's theory of currents, discussed the existence theory, and presented core ideas of the (interior) regularity theory for area-minimizing currents and optimal transport paths. Two original results on generic uniqueness of solutions were presented.
result Generic uniqueness of solutions for both Plateau's problem and optimal branched transport problem.
Investigates regularity of solutions to complex Hessian equation.
problem Regularity of solutions to complex Hessian equation.
method Analyzes solutions to Dirichlet problem with specific density condition.
result Establishes conditions for regularity of solutions.
Paper solves a complex stopping problem using regularization and HJB equations.
problem Time-inconsistent mean-variance optimal stopping problem
method Vanishing regularization method to derive HJB equations and prove existence of solutions
result Formally recovers variational inequalities for original problem
The paper studies the loss landscape of regularized deep matrix factorization, revealing unique and sharp minimizers.
problem Understanding the loss landscape and minimizers of regularized deep matrix factorization problems.
method Theoretical analysis of ℓ2-regularized deep matrix factorization/deep linear network training problems with squared-error loss. result The unique end-to-end minimizer exists for all target matrices except for a set of Lebesgue measure zero.
Study on 3-manifolds finds regular conformal metrics for rough metrics.
problem Characterize conformal metrics for rough Riemannian metrics on 3-manifolds.
method Analogous to the Yamabe problem, study conformal classes and regularity.
result Characterize when a more regular representative exists in the conformal class.
The study shows how certain ODEs and integrals are regular under Borel summation.
problem Analyzing the regularity of solutions to ODEs and integration problems.
method Using geometric perspective on Laplace and Borel transforms, the study examines level 1 ODEs and exponential period integrals over Lefschetz thimbles.
result Solutions of certain ODEs and integration problems are Borel regular.
Local regularization fails in transductive learning for some multiclass problems.
problem Whether local regularization can learn all transductive multiclass problems.
method Provided a negative answer by exhibiting a specific multiclass problem.
result Local regularization cannot learn all transductive multiclass problems.
The paper analyzes Tikhonov regularization in Hilbert scales for statistical inverse problems.
problem Statistical inverse problems in Hilbert scales with general noise.
method Tikhonov regularization scheme with conditional stability estimates and high probability error bounds.
result Explicit rates of convergence for oversmoothing and regular cases over defined regularity classes.
New geometric insights reveal properties of adversarial training problems.
problem Adversarial training in binary classification.
method Equivalence with regularized risk minimization and convex relaxations.
result Existence of minimal and maximal solutions, and regular solutions.
Regularized regression problems are ubiquitous in statistical modeling, signal processing, and machine learning. Sparse regression in particular has been instrumental in scientific model discovery, including compressed sensing applications, variable selection, and high-dimensional analysis. We propose a broad framework…
Regularized policies are robust to adversarial rewards.
problem Understanding the effects of regularization on policy exploration and robustness.
method Using Fenchel duality to derive the dual problem of the regularized RL objective, showing the optimal policy is robust to adversarial rewards.
result Regularized policies are optimal for a reinforcement learning problem under adversarial reward conditions.
Path regularization reveals convex optimization in deep ReLU networks.
problem Understanding the optimization landscape of deep neural networks.
method Introducing path regularization to make the training problem convex and sparsity-inducing.
result Path regularized parallel ReLU networks are a parsimonious convex model in high dimensions.
New methods tackle statistical inverse problems with random data.
problem Statistical inverse problems with random experimental design.
method Spectral regularization, regularization by projection, convex penalties.
result Minimax rates in expectation and probability for convergence.
Study of regularized least squares in RKKS with indefinite kernels.
problem Asymptotic properties of regularized least squares with indefinite kernels in RKKS.
method Introducing a bounded hyper-sphere constraint, theoretical demonstration of globally optimal solution, modified error decomposition techniques, matrix perturbation theory.
result Derivation of learning rates in RKKS, same as RKHS under certain conditions.
Sparsity inducing regularization is an important part for learning over-complete visual representations. Despite the popularity of ℓ1 regularization, in this paper, we investigate the usage of non-convex regularizations in this problem. Our contribution consists of three parts. First, we propose the leaky capped …
A neural network learns a convex regularizer for better image reconstruction.
problem Improving image reconstruction in inverse problems.
method Adversarial training of a data-adaptive ICNN as a convex regularizer.
result The convex regularizer leads to better convergence and error reduction in image reconstruction.
Study MinMax methods for optimization problems, including optimal transport.
problem Optimization problems, especially optimal transport.
method MinMax framework, regularization, neural networks, approximation theorems.
result Justification of neural networks for solving optimization problems.
A regularized optimization problem over a large unstructured graph is studied, where the regularization term is tied to the graph geometry. Typical regularization examples include the total variation and the Laplacian regularizations over the graph. When applying the proximal gradient algorithm to solve this problem, t…
New framework for robust regularization under uncertain data distributions.
problem Addressing ill-posed inverse problems and statistical estimation under distributional uncertainty.
method Distributionally robust optimal regularization using convex duality.
result Identifies robust regularizers that remain effective under data distributional perturbations.
This paper explores how entropic regularization improves Wasserstein estimators' performance.
problem Improving the approximation and estimation properties of Wasserstein estimators.
method Entropic regularization of optimal transport costs to smooth Wasserstein estimators.
result Entropic regularization can achieve comparable statistical performance to un-regularized estimators at lower computational cost.
Develops regularity theory for Beckmann's optimal transport problem.
problem Minimizing total squared flux in continuous transport from source to target.
method Unconstrained Lagrangian formulation, variational first order optimality conditions, Schauder estimates.
result Exact Hölder regularity of potential, flux, and flow generating on bounded, regular domains.
Variational problems that involve Wasserstein distances and more generally optimal transport (OT) theory are playing an increasingly important role in data sciences. Such problems can be used to form an examplar measure out of various probability measures, as in the Wasserstein barycenter problem, or to carry out param…
Regularization for matrix factorization (MF) and approximation problems has been carried out in many different ways. Due to its popularity in deep learning, dropout has been applied also for this class of problems. Despite its solid empirical performance, the theoretical properties of dropout as a regularizer remain qu…
Classifies tilings of hyperbolic plane by regular polygons.
problem Decidability of tiling patterns in hyperbolic plane.
method Finite set of local and inductive combinatorial constraints.
result First known weakly aperiodic protosets of regular polygons in hyperbolic plane.
This paper addresses the problem of inferring a regular expression from a given set of strings that resembles, as closely as possible, the regular expression that a human expert would have written to identify the language. This is motivated by our goal of automating the task of postmasters of an email service who use r…
We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the …
New method averages SGD iterates to achieve adjustable regularization.
problem Overfitting in machine learning models.
method Averaging SGD iterates for regularized solutions.
result Obtain regularized solutions without tuning parameters.
Paper solves a complex equation for unbounded convex sets.
problem Solving the Lp dual Minkowski problem for unbounded closed sets. method Using variational properties of Monge-Ampère functionals, the paper proves existence, regularity, and uniqueness of solutions.
result Existence, regularity, and uniqueness of solutions to the Monge-Ampère type equation for p≥1. New regularizer for machine learning using private data.
problem Machine learning with private data.
method Distributionally-robust optimization with locally-differentially-private datasets.
result New regularizer for training linear regression models.
New method uses deep learning to solve inverse problems with provable guarantees.
problem Solving inverse problems with high-quality results and provable guarantees.
method Convex-Nonconvex (CNC) framework with input weakly convex neural network (IWCNN).
result The method provides provably convergent regularization for inverse problems.
PAR provides a flexible framework for quantization in optimization problems.
problem Challenges in optimization problems over discrete or quantized variables.
method Piecewise-affine regularization (PAR) for modeling and computational optimization.
result PAR-regularized loss functions exhibit high quantization at critical points in the overparameterized regime.
We consider the quantifier-free languages, Bc and Bc0, obtained by augmenting the signature of Boolean algebras with a unary predicate representing, respectively, the property of being connected, and the property of having a connected interior. These languages are interpreted over the regular closed sets of n-dimension…
We find the optimal Tikhonov regularizer for linear inverse problems without prior knowledge.
problem Finding the optimal regularizer for linear inverse problems in imaging.
method Characterization of the optimal regularizer and learning from data.
result The optimal regularizer is independent of the forward operator and depends only on the mean and covariance of the random variable.
We estimate Radon-Nikodym derivatives using regularization in reproducing kernel Hilbert spaces.
problem Estimating Radon-Nikodym derivatives in various applications.
method General regularization scheme in reproducing kernel Hilbert spaces.
result High order accuracy in reconstructing Radon-Nikodym derivatives at any point.