Convex optimization with sparsity-promoting convex regularization is a standard approach for estimating sparse signals in noise. In order to promote sparsity more strongly than convex regularization, it is also standard practice to employ non-convex optimization. In this paper, we take a third approach. We utilize a no…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “convex regularization”
Optimally shows the distance between perturbed convex functions and their Γ-regularizations.
problem Understanding the difference between perturbed convex functions and their Γ-regularizations.
method Analyzing the compactly supported perturbation and the Γ-regularization of a strictly convex function.
result The optimal estimate of the distance between perturbed convex functions and their Γ-regularizations is shown to be .
Study iterative regularization for linear models with convex bias, improving robust sparse recovery.
problem Improving robust sparse recovery with iterative regularization for linear models.
method Primal-dual gradient approach, analyzing convergence in presence of noise, combining regularization and optimization.
result Theoretical results show state-of-the-art performances with computational speed-ups.
Study on regularity of optimal transport maps on convex domains with quadratic cost.
problem Regularity of optimal transport maps between convex domains with quadratic cost.
method Analysis of -densities and boundary conditions, monotonicity formula for optimal transport maps.
result Proves -regularity for nondegenerate -densities and -regularity for boundary.
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networkscs.LG
Path regularization reveals convex optimization in deep ReLU networks.
problem Understanding the optimization landscape of deep neural networks.
method Introducing path regularization to make the training problem convex and sparsity-inducing.
result Path regularized parallel ReLU networks are a parsimonious convex model in high dimensions.
Convex solutions to a specific equation are smooth when the phase is smooth enough.
problem Regularity of solutions to the Lagrangian mean curvature equation.
method Showed regularity for convex solutions under Hölder continuity conditions on the phase.
result Convex viscosity solutions are regular if the Lagrangian phase is Hölder continuous.
Optimal Regularization Under Uncertainty: Distributional Robustness and Convexity Constraintsmath.OC
New framework for robust regularization under uncertain data distributions.
problem Addressing ill-posed inverse problems and statistical estimation under distributional uncertainty.
method Distributionally robust optimal regularization using convex duality.
result Identifies robust regularizers that remain effective under data distributional perturbations.
A fast sketching algorithm solves regularized least squares problems efficiently.
problem Solving large-scale optimization problems with convex or nonconvex regularization.
method Sketching for Regularized Optimization (SRO) algorithm that generates a sketch of the original data matrix and solves the sketched problem.
result General theoretical results for the approximation error between the original and sketched problems, including minimax rates for sparse signal estimation.
This work analyzes Batch Normalization through convex optimization, providing insights and improved training methods.
problem Understanding and improving the effectiveness of Batch Normalization in deep neural networks.
method Introducing convex duality to model weight-decay regularized ReLU networks with BN, and designing an explicit regularization approach.
result Gradient Descent provides an algorithmic bias effect on BN networks, which can be explicitly encoded into the convex objective.
Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifierscs.LG
Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical robustness. In principle, convex relaxation can provide tight bounds if the solution to the relaxed problem is feasible for the original non…
In this article, we describe symplectic and complex toric spaces associated to the five regular convex polyhedra. The regular tetrahedron and the cube are rational and simple, the regular octahedron is not simple, the regular dodecahedron is not rational and the regular icosahedron is neither simple nor rational. We re…
Convex surfaces derived from specific Riemannian manifolds with high regularity.
problem Proving convexity of surfaces derived from Riemannian manifolds.
method Analyzing solutions to the very weak Monge-Ampère equation.
result Proved convexity of weakly regular surfaces with nonnegative intrinsic curvature.
A neural network learns a convex regularizer for better image reconstruction.
problem Improving image reconstruction in inverse problems.
method Adversarial training of a data-adaptive ICNN as a convex regularizer.
result The convex regularizer leads to better convergence and error reduction in image reconstruction.
Paper solves Carathéodory's conjecture for -regular convex surfaces.
problem Carathéodory's conjecture about convex surfaces.
method Index formula derived from Lorentz--Minkowski 4-space analysis.
result Affirmative solution to conjecture for -regular surfaces.
Diffeomorphisms of convex polytopes form a Lie group.
problem Understanding transformations of convex polytopes.
method Forming a Lie group from diffeomorphisms of convex polytopes.
result The group of diffeomorphisms of a convex polytope is a regular Lie group.
The paper tackles safe reinforcement learning with convex regularization.
problem Safe reinforcement learning in complex, high-dimensional settings with safety constraints.
method Doubly-regularized RL framework combining reward and parameter regularization, formulated as a convex regularized objective with parametrized policies on an infinite-dimensional statistical manifold.
result Exponential convergence guarantees under sufficient regularization, robust theoretical insights and guarantees for safe RL.
The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a …
Polynomial-time convex optimization for CNNs with ReLU activations.
problem Training Convolutional Neural Networks (CNNs) with ReLU activations.
method Developed a convex analytic framework using semi-infinite duality to formulate equivalent convex optimization problems for CNN architectures.
result Proved that two-layer CNNs can be globally optimized via an norm regularized convex program.
New proof for convex solutions of Monge-Ampère equation.
problem Interior regularity of strictly convex solutions
method Doubling inequality for Hessian in extrinsic distance function
result Interior regularity established
Classifies regularity for Lagrangian mean curvature type equations.
problem Classifying regularity for Lagrangian mean curvature type equations.
method Generalized constant rank theorem for Legendre transform, constructed convex solutions, and showed regularity conditions.
result Optimal regularity conditions for Lagrangian mean curvature type equations.
We analyze learning curves of RF models with convex regularization and derive precise asymptotic expressions.
problem Understanding the learning curves of RF models with general convex regularization.
method Novel multi-level application of the convex Gaussian min max theorem (CGMT) to compute precise asymptotic expressions.
result Precise asymptotic expressions for learning curves of RF models with separable strongly convex regularization or regularization.
Study on the regularity of -Gauss curvature flow near flat interfaces.
problem Regularity of -Gauss curvature flow near flat interfaces.
method Analysis of convex hypersurface near the interface.
result Regularity of the convex hypersurface near the interface.
Unified framework for training neural networks with non-smooth, non-convex regularizers.
problem Training neural networks with non-smooth, non-convex regularizers.
method ProxGen framework for stochastic proximal gradient descent.
result ProxGen framework achieves the same convergence rate as standard methods and outperforms subgradient-based approaches.
Unified framework reveals regularization mechanism in deep ReLU networks via convex optimization.
problem Understanding the success of deep neural networks.
method Developed a unified framework using convex optimization to reveal regularization mechanisms.
result ReLU networks can be globally optimized via convex programs, enforcing sparsity.
Relaxed Sparse Eigenvalue Conditions for Sparse Estimation via Non-convex Regularized Regressioncs.LG
Non-convex regularizers usually improve the performance of sparse estimation in practice. To prove this fact, we study the conditions of sparse estimations for the sharp concave regularizers which are a general family of non-convex regularizers including many existing regularizers. For the global solutions of the regul…
The paper explores optimal regularizers for data sources, linking them to star bodies.
problem Understanding optimal regularizers for data sources.
method Investigates optimal regularizers for data distributions using star bodies and dual Brunn-Minkowski theory.
result Identifies optimal regularizers and assesses amenability to convex regularization.
Study improves estimation of functions from noisy data using convex penalties.
problem Estimating functions from noisy point evaluations of linear operators.
method Tikhonov regularization with convex and -homogeneous penalty functionals.
result Derives concentration rates for regularized solutions in symmetric Bregman distance.
Dropout is a simple but effective technique for learning in neural networks and other settings. A sound theoretical understanding of dropout is needed to determine when dropout should be applied and how to use it most effectively. In this paper we continue the exploration of dropout as a regularizer pioneered by Wager,…
In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finite-dimensional bounded convex set with exponential concave loss functions and an arbitrary convex regularization. To the best of our knowledg…
The paper analyzes reg-SGD for convex problems, proving convergence and quantifying the rate of convergence.
problem Minimizing convex, L-smooth functions in a Hilbert space.
method Regularized stochastic gradient descent with decaying regularization.
result Strong convergence to the minimum-norm solution without boundedness assumptions.
Proves flows of two-convex Lagrangians are regular, global, and converge.
problem Proves regularity, global existence, and convergence of Lagrangian mean curvature flows in the two-convex case.
method Uses a newly discovered monotone quantity to control two-convexity.
result Proves results for the mean curvature flow of area-decreasing Lagrangian submanifolds.
NAPP-ERM improves ERM with differential privacy guarantees by iteratively achieving target regularization and delivering strong convexity.
problem Over-regularization in privacy-preserving ERM approaches.
method Noise-Augmented Privacy-Preserving Empirical Risk Minimization (NAPP-ERM) with a dual-purpose l2 regularizer and privacy budget retrieval strategy.
result Mitigates over-regularization and achieves strong convexity through a single regularizer.
In this paper, we consider stochastic dual coordinate (SDCA) {\em without} strongly convex assumption or convex assumption. We show that SDCA converges linearly under mild conditions termed restricted strong convexity. This covers a wide array of popular statistical models including Lasso, group Lasso, and logistic reg…
We establish interior regularity for convex viscosity solutions of the special Lagrangian equation. Our result states that all such solutions are real analytic in the interior of the domain.
Paper proposes DC functions for better regularization of inverse problems with theoretical guarantees.
problem Improving regularization for ill-posed inverse problems.
method Introduces difference-of-convex (DC) functions and uses them with optimization algorithms like DCA and PSM.
result DC functions yield improved performance and theoretical guarantees compared to weakly convex functions.
In this work we establish the equivalence of algorithmic regularization and explicit convex penalization for generic convex losses. We introduce a geometric condition for the optimization path of a convex function, and show that if such a condition is satisfied, the optimization path of an iterative algorithm on the un…
The main goal of this work is equipping convex and nonconvex problems with Barzilai-Borwein (BB) step size. With the adaptivity of BB step sizes granted, they can fail when the objective function is not strongly convex. To overcome this challenge, the key idea here is to bridge (non)convex problems and strongly convex …
Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.
problem Solving composite convex optimization problems with composite regularizers.
method Analyzed proximal stochastic gradient method and randomized incremental proximal method under relaxed variance assumptions.
result Proves convergence rate for last iterate of both algorithms under componentwise convexity and smoothness.
SAGA is a fast incremental gradient method on the finite sum problem and its effectiveness has been tested on a vast of applications. In this paper, we analyze SAGA on a class of non-strongly convex and non-convex statistical problem such as Lasso, group Lasso, Logistic regression with regularization, linear r…
Maximum regularized likelihood estimators (MRLEs) are arguably the most established class of estimators in high-dimensional statistics. In this paper, we derive guarantees for MRLEs in Kullback-Leibler divergence, a general measure of prediction accuracy. We assume only that the densities have a convex parametrization …
Paper solves a complex equation for unbounded convex sets.
problem Solving the dual Minkowski problem for unbounded closed sets.
method Using variational properties of Monge-Ampère functionals, the paper proves existence, regularity, and uniqueness of solutions.
result Existence, regularity, and uniqueness of solutions to the Monge-Ampère type equation for .
We develop exact representations of training two-layer neural networks with rectified linear units (ReLUs) in terms of a single convex program with number of variables polynomial in the number of training samples and the number of hidden neurons. Our theory utilizes semi-infinite duality and minimum norm regularization…
In this paper we consider regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer.…
Study shows SGD's generalization is not explained by implicit bias.
problem Explaining the generalization ability of overparameterized learning algorithms.
method Revisited Stochastic Convex Optimization with SGD, demonstrating limitations of implicit bias.
result No distribution-independent or distribution-dependent implicit regularizer can explain SGD's generalization.
Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularizationstat.ML
Convex clustering is a promising new approach to the classical problem of clustering, combining strong performance in empirical studies with rigorous theoretical foundations. Despite these advantages, convex clustering has not been widely adopted, due to its computationally intensive nature and its lack of compelling v…
Regularized empirical risk minimization with constrained labels (in contrast to fixed labels) is a remarkably general abstraction of learning. For common loss and regularization functions, this optimization problem assumes the form of a mixed integer program (MIP) whose objective function is non-convex. In this form, t…
New method uses deep learning to solve inverse problems with provable guarantees.
problem Solving inverse problems with high-quality results and provable guarantees.
method Convex-Nonconvex (CNC) framework with input weakly convex neural network (IWCNN).
result The method provides provably convergent regularization for inverse problems.
We consider the problem of supervised learning with convex loss functions and propose a new form of iterative regularization based on the subgradient method. Unlike other regularization approaches, in iterative regularization no constraint or penalization is considered, and generalization is achieved by (early) stoppin…