Study shows unique sharp local minimum in ℓ 1 \ell_1 ℓ 1 -minimization for dictionary learning.
problem Global recovery of a dictionary from random linear combinations of atoms.
method Norm condition, explicit bound, perturbation-based test, Block Coordinate Descent algorithm.
result Reference dictionary is the unique sharp local minimum of the ℓ 1 \ell_1 ℓ 1 objective function. Lower bound on minimum vertex degree for non-negative Lin-Lu-Yau curvature on graphs.
problem Determining the minimum vertex degree for non-negative Lin-Lu-Yau curvature.
method Investigation of Ollivier-Ricci curvature and Lin-Lu-Yau modification on locally finite graphs.
result Lower bound on minimum vertex degree ensuring non-negative Lin-Lu-Yau curvature.
In this note, we obtain a sharp volume estimate for complete gradient Ricci solitons with scalar curvature bounded below by a positive constant. Using Chen-Yokota's argument we obtain a local lower bound estimate of the scalar curvature for the Ricci flow on complete manifolds. Consequently, one has a sharp estimate of…
Truncated SGD with heavy-tailed noise eliminates sharp local minima.
problem Avoiding sharp local minima in deep learning models.
method Truncated SGD with heavy-tailed gradient noise.
result Truncated SGD can eliminate sharp local minima entirely from its training trajectory.
In this paper, based on the local comparison principle in [12], we study the local behavior of the difference of two spacelike graphs in a neighborhood of a second contact point. Then we apply it to the constant mean curvature equation in 3-dimensional Lorentz-Minkowski space L 3 \mathbb{L}^3 L 3 and get the uniqueness of cr…
Averaged SGD optimizes a smoothed objective, leading to better generalization.
problem Improving generalization performance in machine learning models.
method Analyzed the smoothed objective function of SGD and proved that averaged SGD can optimize this smoothed function efficiently.
result Averaged SGD can efficiently optimize a smoothed objective, leading to better generalization.
Improved portfolio optimization method yields better risk-adjusted returns.
problem Optimizing global minimum variance portfolios with reduced risk.
method k-fold boosted k − k- k − BAHC covariance cleaning procedure for correlation matrices. result Our method outperforms other filtering methods in Sharpe ratios, despite higher turnover.
SGD batch size affects autoencoder global minima sparsity and sharpness.
problem Investigating how batch size impacts autoencoder learning.
method Non-convex autoencoder training with SGD, varying batch sizes.
result SGD batch size influences global minimum sparsity and sharpness.
Despite the non-convex nature of their loss functions, deep neural networks are known to generalize well when optimized with stochastic gradient descent (SGD). Recent work conjectures that SGD with proper configuration is able to find wide and flat local minima, which have been proposed to be associated with good gener…
SGD transitions between maxima and minima with varying time scales.
problem Understanding SGD's behavior near critical points in noisy landscapes.
method Analyzing SGD convergence and escape dynamics in 1D landscapes with infinite- and finite-variance noise.
result SGD reliably moves to the basin's minimum unless close to a local maximum, where it can linger.
The paper analyzes how good initial guesses affect the amount of data needed for low-rank matrix recovery.
problem Theoretical guarantee of local optimization algorithms requires excessive data to prevent spurious local minima.
method Quantifies the relationship between initial guess quality and sample complexity using restricted isometry constant.
result A linear improvement in initial guess quality leads to a constant factor improvement in sample complexity.
Proposes a new model to maximize out-of-sample Sharpe ratios by forecasting tangency portfolios.
problem Maximizing Sharpe ratios when returns and covariances are not stationary.
method Forecast the tangency portfolio using vector autoregressions and invest in the minimum Euclidean distance portfolio.
result Empirically validated superior out-of-sample Sharpe ratios.
In this paper we compute the sharp lower bounds for the crossing number of n n n -string k k k -loop essential tangles. For essential tangles with only string components, we characterise the ones with the minimum crossing number for a given number of components, both when the tangle has knotted strings or only unknotted stri…
SGD noise helps select flat minima by concentrating in sharp directions and being proportional to loss value.
problem Understanding the implicit regularization of SGD and selecting flat minima in over-parameterized models.
method Relating SGD's linear stability to the Frobenius norm of the Hessian and analyzing the alignment property of SGD noise.
result Flat minima are linearly stable for SGD, and their sharpness is bounded independently of model size and sample size.
A new tradeoff between regularization and sharpness improves model performance in overparameterized settings.
problem Improving model performance in overparameterized settings with minimum-norm interpolators.
method Proposes a regularization-sharpness tradeoff for overparameterized linear regression with an ℓ^p penalty.
result Empirical validation shows the tradeoff terms can distinguish performant linear interpolators.
Sharp bounds found on nonabelian quotients of surface braid groups.
problem Finding the smallest nonabelian quotients of surface braid groups.
method Sharp lower bounds and classification of quotients.
result Quotients of minimum order are either symmetric groups or 2-step nilpotent p-groups.
Estimates Bartnik mass for metrics with nonnegative Gauss curvature.
problem Estimating Bartnik mass for specific metric configurations.
method Using area, total mean curvature, and a metric roundness measure.
result Estimate approaches sharp value for round spheres.
SGD favors flat minima exponentially more than sharp minima in deep learning.
problem Understanding how SGD selects flat minima in deep learning.
method Developed a density diffusion theory (DDT) to analyze minima selection.
result SGD exponentially favors flat minima over sharp minima due to Hessian-dependent noise.
Stochastic gradient descent (SGD) is almost ubiquitously used for training non-convex optimization tasks. Recently, a hypothesis proposed by Keskar et al. [2017] that large batch methods tend to converge to sharp minimizers has received increasing attention. We theoretically justify this hypothesis by providing new pro…
SAM optimizes deep networks by oscillating between sides of the minimum.
problem Improving performance of deep networks.
method Gradient-based optimization method that oscillates between sides of the minimum.
result SAM effectively performs gradient descent on the spectral norm of the Hessian, encouraging drift towards wider minima.
The paper calculates minimum Dehn colors for knots using symmetric local biquandle cocycles.
problem Determining the minimum number of Dehn colors for knots.
method Using symmetric local biquandle cocycle invariants to evaluate minimum Dehn colors.
result There exist knots distinguished by minimum numbers of Dehn colors.
SAM selects flatter minima late in training, improving generalization.
problem Improving neural network generalization under various settings.
method Sharpness-Aware Minimization (SAM) applied late in training.
result SAM efficiently selects flatter minima late in training, improving generalization.
This paper introduces minimum-risk recalibration for probabilistic classifiers, improving their reliability and accuracy.
problem Improving the reliability and accuracy of probabilistic classifiers.
method Minimum-risk recalibration within the MSE decomposition framework, analyzing UMB method and label shift adaptation.
result The optimal number of bins for UMB scales with n 1 / 3 n^{1/3} n 1/3 , resulting in a risk bound of approximately O ( n − 2 / 3 ) O(n^{-2/3}) O ( n − 2/3 ) . Langevin dynamics (LD) has been proven to be a powerful technique for optimizing a non-convex objective as an efficient algorithm to find local minima while eventually visiting a global minimum on longer time-scales. LD is based on the first-order Langevin diffusion which is reversible in time. We study two variants th…
Every local minimum in non-convex machine learning is globally optimal.
problem Non-convex optimization challenges in machine learning.
method Proves every local minimum achieves globally optimal value of perturbable gradient basis model.
result Theoretical support for non-convex machine learning similar to convex machine learning.
The paper constructs upper bounds for cost minimization in shallow neural networks.
problem Cost minimization in underparametrized shallow ReLU networks.
method Explicit construction of upper bounds based on the geometric structure of classification data.
result An upper bound on the minimum of the cost function of order O ( δ P ) O(δ_P) O ( δ P ) , with exact degenerate local minimum in the special case M = Q M=Q M = Q . Hill-ADAM optimizes loss landscapes by exploring state space deterministically.
problem Escaping local minima in loss landscapes.
method Hill-ADAM alternates between minimizing and maximizing error to explore the loss space.
result Hill-ADAM finds the global minimum state in loss landscapes.
Proposes a method to solve deep neural networks' local minimum problem.
problem Local minimum problem in deep neural networks training.
method Transforms cross-entropy loss into risk-averse error criterion, adjusts RSI, and uses convexity region.
result Trained deep learning machine is expected to be inside a global minimum's attraction basin.
Tests Sharpe ratio for skill vs luck in asset management.
problem Accuracy of Sharpe ratio in measuring skill vs luck.
method Statistical tests to assess the significance of Sharpe ratios.
result Tests reveal the statistical significance of Sharpe ratios and their impact of auto-correlation.
The paper measures non-convexity of real algebraic curves near a strict local minimum.
problem Measuring the non-convexity of real algebraic curves near a strict local minimum.
method Introduced a new combinatorial object, the Poincare-Reeb graph, to encode and quantify the shape of curves.
result The Poincare-Reeb graph is a plane tree and can be used to study the asymptotic behaviour of level curves near a strict local minimum.
In this paper, we theoretically prove that adding one special neuron per output unit eliminates all suboptimal local minima of any deep neural network, for multi-class classification, binary classification, and regression with an arbitrary loss function, under practical assumptions. At every local minimum of any deep n…
New proof shows how to identify DAGs with weakly increasing errors.
problem Identifying the true DAG in models with weakly increasing error variances.
method Minimum-trace DAG method and hill climbing algorithm with R2R neighborhood.
result Hill climbing algorithm without strict local optima under weakly increasing error variances.
One of the main difficulties in analyzing neural networks is the non-convexity of the loss function which may have many bad local minima. In this paper, we study the landscape of neural networks for binary classification tasks. Under mild assumptions, we prove that after adding one special neuron with a skip connection…
This study compares Markowitz and Single-Index models for Malaysian stocks.
problem Optimizing portfolio selection for Malaysian stocks using different models.
method Applied Markowitz and Single-Index models to 10-year historical data of 10 stocks and a risk-free asset.
result Comparison of minimum variance and maximum Sharpe portfolios for both models under various constraints.
We study the blow-up behaviour of minimizing sequences for the singular Moser-Trudinger functional on compact surfaces. Assuming non-existence of minimum points, we give an estimate for the infimum value of the functional. This result can be applied to give sharp Onofri-type inequalities on the sphere in the presence o…
Develops local curvature estimates for mean curvature flow.
problem Sharp curvature pinching estimates for mean curvature flow.
method Local version of Huisken-Stampacchia iteration.
result Local curvature estimates do not depend on noncollapsing quality.
G-TRACER optimizes deep learning by promoting flat minima.
problem Promoting generalization in deep learning architectures.
method Geometric TRACE Ratio regularization, curvature-regularized optimizers.
result Converges to a neighborhood of local minima of unregularized objective.
New method for Sharpe ratio analysis in high dimensions using residual-based nodewise regression.
problem Consistency of Sharpe ratio estimators in high-dimensional portfolios.
method Residual-based nodewise regression for estimating precision matrix of errors and returns.
result Consistent Sharpe ratio estimators in various portfolio settings.
New algorithm finds local minima in non-convex problems efficiently.
problem Finding local minima in non-convex finite-sum minimization problems.
method Stochastic Trust Region (STR) algorithm combining inexact gradient and Hessian estimation.
result STR finds ( ε , ε ) (ε, \sqrtε) ( ε , ε ) -approximate local minimum with improved efficiency. Study shows LLC correlates with neural network compressibility.
problem Evaluating limits of neural network compression.
method Extended minimum description length principle using singular learning theory.
result Complexity estimates based on LLC are linearly correlated with compressibility.
We design a non-convex second-order optimization algorithm that is guaranteed to return an approximate local minimum in time which scales linearly in the underlying dimension and the number of training examples. The time complexity of our algorithm to find an approximate local minimum is even faster than that of gradie…
We propose a semismooth Newton algorithm for pathwise optimization (SNAP) for the LASSO and Enet in sparse, high-dimensional linear regression. SNAP is derived from a suitable formulation of the KKT conditions based on Newton derivatives. It solves the semismooth KKT equations efficiently by actively and continuously s…
Study analyzes stock performance before, during, and after the pandemic.
problem Impact of the pandemic on stock performance and risk.
method Daily data of most traded companies in Colombia from 2015 to 2023, using minimum variance approach.
result Portfolio returns and risks varied significantly during the pandemic.
New method uses Gaussian processes to find local minima efficiently.
problem Finding all local minima of a black-box function with unknown derivatives.
method Sequentially selects input points to update GP derivatives' confidence intervals.
result Theoretical analysis and numerical experiments show the method's effectiveness.
New model shows SGD can prefer sharp or flat solutions based on label noise.
problem Understanding SGD's preference for flat or sharp solutions during training.
method Solved an analytically solvable model to explore SGD behavior.
result Data distribution determines sharpness at convergence; isotropic label noise leads to flat minimum preference.
Sharp conditions found for solving heat equation on Riemannian manifolds.
problem Solving semilinear heat equation on Riemannian manifolds.
method Sharp conditions derived for local-in-time solvability.
result Sharp conditions on solvability given for complete and connected manifolds.
The L1 loss landscape of neural nets near local minima behaves differently, revealing exponential decay and increased vertex density.
problem Understanding the L1 loss landscape of neural nets near local minima.
method Iterative minimization of the loss function on adjacent vertices of the Deep ReLU Simplex algorithm.
result Exponential decay of loss levels and increased vertex density around local minima.
Sharp estimate for nodal domains intersecting a ball on a Riemannian manifold.
problem Local bounds for nodal domains on Riemannian manifolds.
method Combining Remez inequality for eigenfunctions and Landis growth lemma in narrow domains.
result Proved a sharp estimate of nodal domains intersecting a ball.