Develops an accelerated algorithm for solving nonmonotone generalized equations.
problem Solving nonmonotone generalized equations with possibly non-accelerated schemes.
method Combines Nesterov's acceleration and variance-reduction techniques for a class of generalized equations.
result Achieves O ( 1 / k 2 ) \mathcal{O}(1/k^2) O ( 1/ k 2 ) convergence rates, improving upon non-accelerated counterparts. Large stepsizes can accelerate gradient descent for logistic regression.
problem Optimizing logistic regression with large stepsizes.
method Gradient descent with large stepsize for ℓ 2 \ell_2 ℓ 2 -regularized logistic regression. result Large stepsizes can achieve O ~ ( κ ) \widetilde{\mathcal{O}}(\sqrtκ) O ( κ ) convergence, improving over O ~ ( κ ) \widetilde{\mathcal{O}}(\sqrtκ) O ( κ ) from classical theory. We propose a randomized nonmonotone block proximal gradient (RNBPG) method for minimizing the sum of a smooth (possibly nonconvex) function and a block-separable (possibly nonconvex nonsmooth) function. At each iteration, this method randomly picks a block according to any prescribed probability distribution and solves…
Data coarse graining improves model performance by filtering out less relevant features.
problem Lossy data transformations lose information but can improve model generalization.
method Data coarse graining schemes that systematically discard features based on relevance to the learning task.
result A 'high-pass' scheme helps models generalize better by filtering out less relevant features.
A new L-BFGS method tackles large-scale optimization with fewer evaluations.
problem Efficiently solving large-scale unconstrained optimization problems.
method Proposes a regularized L-BFGS method with line search techniques.
result Shows global convergence and robust performance in numerical tests.
In this paper we study a broad class of structured nonlinear programming (SNLP) problems. In particular, we first establish the first-order optimality conditions for them. Then we propose sequential convex programming (SCP) methods for solving them in which each iteration is obtained by solving a convex programming pro…
The paper explores arbitrage opportunities in derivative markets under specific conditions.
problem Arbitrage opportunities in derivative markets under different conditions.
method Analyzes the relationship between pricing kernel monotonicity and stochastic arbitrage opportunities.
result Pricing kernel nonmonotonicity is equivalent to stochastic arbitrage opportunities under adequacy.
We consider the problem of minimizing a Lipschitz differentiable function over a class of sparse symmetric sets that has wide applications in engineering and science. For this problem, it is known that any accumulation point of the classical projected gradient (PG) method with a constant stepsize 1 / L 1/L 1/ L satisfies the $L…
New method solves root-finding problems with faster convergence.
problem Finite-sum co-coercive equations
method Variance-reduced Krasnoselkii--Mann methods
result Achieves both O ( 1 / k 2 ) \mathcal{O}(1/k^2) O ( 1/ k 2 ) and o ( 1 / k 2 ) o(1/k^2) o ( 1/ k 2 ) convergence rates This paper advances extragradient methods for solving inclusions under co-hypomonotonicity.
problem Solving inclusions with non-Lipschitz mappings.
method Unified and generalized extragradient methods under co-hypomonotonicity.
result Achieved O ( 1 / k ) \mathcal{O}(1/k) O ( 1/ k ) convergence rates for residual norms. We consider the problem of minimizing a difference-of-convex (DC) function, which can be written as the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous possibly nonsmooth concave function. We refine the convergence analysis in [38] for the proximal DC algorithm …
Iteratively reweighted ℓ 1 \ell_1 ℓ 1 algorithm is a popular algorithm for solving a large class of optimization problems whose objective is the sum of a Lipschitz differentiable loss function and a possibly nonconvex sparsity inducing regularizer. In this paper, motivated by the success of extrapolation techniques in accele…
We consider a class of constrained optimization problems with a possibly nonconvex non-Lipschitz objective and a convex feasible set being the intersection of a polyhedron and a possibly degenerate ellipsoid. Such problems have a wide range of applications in data science, where the objective is used for inducing spars…
Develops variance-reduced methods for solving generalized equations.
problem Solving a class of generalized equations, including minimization, minimax, and variational inequalities.
method Integrates accelerated operator splitting, fixed-point methods, and variance reduction techniques.
result Achieves both O ( 1 / k 2 ) \mathcal{O}(1/k^2) O ( 1/ k 2 ) and o ( 1 / k 2 ) o(1/k^2) o ( 1/ k 2 ) convergence rates on the expected squared norm of the FBS residual. Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus…
The paper studies how more data affects prediction risk in high-dimensional models.
problem The impact of increasing data on prediction risk in high-dimensional models.
method Derives central limit theorem and provides finite-sample distribution and confidence interval for prediction risk.
result Demonstrates 'more data hurt' phenomenon in high-dimensional least squares estimation.
New variance-reduction methods solve stochastic composite inclusions.
problem Solving nonmonotone stochastic composite inclusions.
method Developed unbiased and biased variance-reduced estimators for FRBS method.
result Achieved best oracle complexities for finite-sum and expectation settings.
Improves CNN stability by translating classical signal denoising methods.
problem Stability of CNNs is poorly understood.
method Interprets classical signal denoising methods as ResNet architectures.
result Translates diffusivities, shrinkage functions, and regularizers into CNN activation functions.
We present a heuristic based algorithm to induce \textit{nonmonotonic} logic programs that will explain the behavior of XGBoost trained classifiers. We use the technique based on the LIME approach to locally select the most important features contributing to the classification decision. Then, in order to explain the mo…
In financial markets, greater volatility is usually considered synonym of greater risk and instability. However, large market downturns and upturns are often preceded by long periods where price returns exhibit only small fluctuations. To investigate this surprising feature, here we propose using the mean first hitting…
In this paper, we further study the forward-backward envelope first introduced in [28] and [30] for problems whose objective is the sum of a proper closed convex function and a twice continuously differentiable possibly nonconvex function with Lipschitz continuous gradient. We derive sufficient conditions on the origin…
New algorithms solve stochastic variational inequalities without bounded variance assumption.
problem Solving stochastic variational inequalities without bounded variance assumption.
method Developed algorithms for two classes of problems: monotone and structured nonmonotone VIs.
result Oracle complexity of O(ε^-4) for solving VIs with unbounded domains and possibly unbounded variance.
Analysis of ridge regression under concept shift reveals nontrivial effects on generalization performance.
problem Understanding and mitigating the impact of distribution shift in machine learning models.
method Derivation of exact prediction risk expression in the thermodynamic limit for ridge regression under concept shift.
result Reveals a phase transition and nonmonotonic data dependence of test performance under concept shift.
UBI model proves financial equilibrium exists.
problem Proving existence of financial equilibrium with UBI.
method Backward stochastic differential equation (BSDE) approach.
result Equilibrium exists in UBI model.
Improved variational inequality algorithms using adaptive step sizes.
problem Solving monotone variational inequalities and convex-concave min-max problems efficiently.
method Adaptive step sizes that eliminate hyperparameters and global Lipschitz continuity requirements.
result Eliminated the need for the golden ratio in the algorithm and improved complexity bounds.
Synthetic data augmentation can improve imbalanced classification metrics.
problem Improving imbalanced classification metrics
method Developing a framework for analyzing the effects of synthetic data augmentation on score-based classification
result Augmentation can improve AUROC, AUPRC, balanced accuracy, and F1 score
FACMAC combines deep policy gradients with factored critic for multi-agent reinforcement learning.
problem Cooperative multi-agent reinforcement learning in discrete and continuous action spaces.
method FACMAC uses a centralised but factored critic, combining per-agent utilities into a joint action-value function.
result FACMAC outperforms MADDPG and other baselines on multi-agent particle environments and StarCraft II tasks.
This paper extends financial theory to measure learnable market structure under computational constraints.
problem Understanding learnable market structure under bounded computational capacity.
method Introduces financial epiplexity as a measure of learnable market structure, extending classical information theory.
result Proves that equal entropy does not imply equal epiplexity and derives thresholds for useful regimes.
The study uses Random Matrix Theory to identify structural changes in stock markets during shocks.
problem Understanding structural changes in stock markets during exogenous shocks.
method Random Matrix Theory and complexity gap analysis.
result The complexity gap collapses during shocks, indicating strong synchronization, and widens before shocks, signaling a rich structure.
We simplify neural networks to 3D to study their topological changes.
problem Understanding how neural network layers affect low-dimensional topological invariants.
method Limiting each layer to a width of 3D space, tracking changes in linking numbers.
result ResNets and transformers are equally powerful in changing linking numbers.
In the context of sparse recovery, it is known that most of existing regularizers such as ℓ 1 \ell_1 ℓ 1 suffer from some bias incurred by some leading entries (in magnitude) of the associated vector. To neutralize this bias, we propose a class of models with partial regularizers for recovering a sparse solution of a linear …
Proves weak convergence equals mean convergence in GGC.
problem Proving convergence in GGC distributions.
method Using generalized gamma convolution (GGC) and expected utility maximization.
result Weak convergence implies mean convergence in GGC.
Introduces generalized almost statistical convergence and its properties.
problem Developing a new convergence concept for sequences.
method Introducing generalized almost statistical convergence and proving its properties.
result Existence of a GAS convergent sequence that is neither statistical nor almost convergent.
This is an intuitive survey of extrinsic and intrinsic notions of convergence of manifolds complete with pictures of key examples and a discussion of the properties associated with each notion. We begin with a description of three extrinsic notions which have been applied to study sequences of submanifolds in Euclidean…
The abstract discusses convergence properties of Lipschitz functions and sets defined by equations.
problem Convergence of Lipschitz functions and sets defined by equations.
method Painlevé-Kuratowski convergence applied to Lipschitz functions and sets defined by equations.
result Generalizations and reverses of classical theorems on convergence of functions and sets.
Study shows intrinsic timed Hausdorff convergence leads to Gromov-Hausdorff and big bang convergence.
problem Distance between Lorentzian manifolds.
method Intrinsic timed Hausdorff convergence.
result Intrinsic timed Hausdorff convergence implies Gromov-Hausdorff and big bang convergence.
Studied SGD convergence under weak conditions.
problem Convergence of SGD in nonconvex optimization.
method Analyzed biased nonconvex SGD under mild conditions.
result Provided convergence rates and complexities.
Study on convergence rate of Q Q Q -curvature flow in 6 dimensions.
problem Analyzing the convergence rate of Q Q Q -curvature flow in 6 dimensions. method Provided an example of a slowly converging Q 6 Q_6 Q 6 -curvature flow in dimension 6. result The Q Q Q -curvature flow in 6 dimensions does not always converge exponentially, unlike in 2 dimensions. Establishes geometric convergence of iterative optimization algorithms.
problem Analyzes convergence of iterative optimization algorithms under general assumptions.
method General framework for iterative optimization algorithms, proving asymptotic geometric convergence and providing convergence rates.
result Asymptotic geometric convergence of iterative optimization algorithms with exact rate.
Uniform counting formulas for orthogeodesics in Kleinian groups converge.
problem Counting orthogeodesics in Kleinian groups converging to a limit.
method Spectral gap of the limit manifold and geodesic flow mixing property.
result Asymptotically uniform counting formulas for orthogeodesics.
The paper connects different convergence concepts in geometric analysis.
problem Comparing convergence concepts in geometric analysis.
method Relating L p L^p L p convergence and volume convergence to Intrinsic Flat and Gromov-Hausdorff convergence. result Conditions for convergence of Riemannian manifolds under specific conditions.
The paper explores null distance convergence for warped product spacetimes.
problem Defining convergence for sequences of spacetimes as metric spaces.
method Using the null distance to define convergence of spacetimes.
result Optimal convergence theorem for warped product spacetimes.
New quasi-Newton method guarantees global superlinear convergence.
problem Global convergence and superlinear convergence of quasi-Newton methods.
method Hybrid proximal extragradient method with online learning for Hessian approximation.
result First globally convergent quasi-Newton method with explicit superlinear convergence rate.
We investigate finite-time decoupled convergence in nonlinear two-time-scale stochastic approximation.
problem Achieving decoupled convergence in nonlinear two-time-scale stochastic approximation.
method Nested local linearity assumption, suitable step size selection, convergence analysis of matrix cross term, fourth-order moment convergence rates.
result Finite-time decoupled convergence rates can be achieved in nonlinear two-time-scale stochastic approximation with proper step size selection.
The article introduces a new convergence concept for Lorentzian spaces and applies it to generalized cones.
problem Stability of curvature bounds in generalized Lorentzian cones.
method Introduces ℓ \ell ℓ -convergence for Lorentzian pre-length spaces, applies it to generalized cones, and proves stability of curvature bounds. result Sharp timelike curvature and curvature-dimension bounds for generalized cones are established.
AdaBoost's classifier and margins converge to a known value.
problem Convergence properties of AdaBoost algorithm.
method Formal proofs of convergence properties of AdaBoost's classifier and margins.
result AdaBoost's classifier and margins converge to a known value.
Study shows gap between uniform convergence and test error in random feature models.
problem Understanding the gap between uniform convergence and test error in random feature models.
method Analytical expressions for uniform convergence over norm balls, interpolators, and minimum norm interpolator risk derived and proved.
result Uniform convergence over interpolators still gives a non-trivial bound of test error even when classical uniform convergence is vacuous.
The Sinkhorn-Knopp derivatives converge with linear rate.
problem Optimal transport problem with entropic regularization.
method Iterative proportional fitting procedure.
result Derivatives converge with linear rate.