In this paper, we propose a novel sufficient decrease technique for stochastic variance reduced gradient descent methods such as SVRG and SAGA. In order to make sufficient decrease for stochastic optimization, we design a new sufficient decrease criterion, which yields sufficient decrease versions of stochastic varianc…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,786 papers · 148 categories
Trend · papers per month
48 results for “Sufficient decrease”
In this paper, we propose a novel sufficient decrease technique for variance reduced stochastic gradient descent methods such as SAG, SVRG and SAGA. In order to make sufficient decrease for stochastic optimization, we design a new sufficient decrease criterion, which yields sufficient decrease versions of variance redu…
The paper analyzes the convergence of CART under a SID condition, improving previous results.
problem Investigating the convergence rate of CART under a sufficient impurity decrease condition.
method Established an upper bound on prediction error under SID condition, introduced easily verifiable conditions.
result Improved convergence rate of CART under SID condition, demonstrated examples of error bound limitations.
Study connects curvature bounds to map existence and flow solutions.
problem Existence of lower scalar curvature bounds and measures.
method Relates curvature bounds to map existence and backward limit of Ricci flow solutions.
result Sufficient condition for existence of limiting scalar curvature measure.
Numerical observations on martingale couplings are confirmed under certain conditions.
problem Understanding the validity of numerical observations on maximizers and minimizers of martingale couplings.
method Investigation of sufficient conditions and counterexamples for the property to hold.
result The non-decreasing property of martingale couplings is preserved for maximizers under specific conditions.
Topological entropy decreases strictly along Ricci flow near hyperbolic metrics.
problem Understanding entropy changes in flows near hyperbolic metrics.
method Analysis of geodesic flow on Riemannian manifolds with variable negative curvature.
result Topological entropy strictly decreases along normalized Ricci flow near hyperbolic metrics.
New conditions for weighted composition operators in group homomorphisms.
problem Conditions for weighted composition operators in group homomorphisms.
method Range decreasing group homomorphisms.
result New insights into weighted composition operators and their algebraic structure.
Novel evolutionary strategy solves stochastic constrained optimization problems.
problem Optimizing objective functions with stochastic constraints in reinforcement learning.
method Design of a novel optimization algorithm with a sufficient decrease mechanism for stochastic constrained problems.
result Demonstrated convergence of the algorithm on control tasks and constrained optimization problems.
Given a geometrically finite hyperbolic cone-manifold, with the cone singularity sufficiently short, we construct a one parameter family of cone-manifolds decreasing the cone angle to zero. We also control the geometry of this one parameter family via the Schwarzian derivative of the projective boundary and the length …
In this paper, we study the relation of the monotonicity of Hawking Mass and geometric flow problems. We show that along the Hamilton-DeTurck flow with bounded curvature coupled with the modified mean curvature flow, the Hawking mass of the hypersphere with a sufficiently large radius in Schwarzschild spaces is monoton…
TREGO improves EGO for global optimization of high-dimensional problems.
problem Efficient Global Optimization struggles with high dimensions and lacks theoretical guarantees.
method TREGO alternates between EGO steps and local steps within a trust region.
result TREGO outperforms EGO and other methods in black-box optimization problems.
The object of our investigation is a point that gives the maximum value of a potential with a strictly decreasing radially symmetric kernel. It defines a center of a body in Rm. When we choose the Riesz kernel or the Poisson kernel as the kernel, such centers are called a radial center or an illuminating center, respec…
Selective regression allows abstention to improve fairness criteria.
problem Selective regression can exacerbate disparities between subgroups.
method Proposes new fairness criteria and two approaches to mitigate performance disparity.
result Proposed fairness criteria ensures performance improvement for every subgroup with reduced coverage.
Many optimization algorithms converge to stationary points. When the underlying problem is nonconvex, they may get trapped at local minimizers and occasionally stagnate near saddle points. We propose the Run-and-Inspect Method, which adds an "inspect" phase to existing algorithms that helps escape from non-global stati…
We study the use of knowledge distillation to compress the U-net architecture. We show that, while standard distillation is not sufficient to reliably train a compressed U-net, introducing other regularization methods, such as batch normalization and class re-weighting, in knowledge distillation significantly improves …
We propose a method for recognizing moving vehicles, using data from roadside audio sensors. This problem has applications ranging widely, from traffic analysis to surveillance. We extract a frequency signature from the audio signal using a short-time Fourier transform, and treat each time window as an individual data …
Consistency conditions for affine term structure modelscond-mat.other
ATSM are widely applied for pricing of bonds and interest rate derivatives but the consistency of ATSM when the short rate, r, is unbounded from below remains essentially an open question. First, the standard approach to ATSM uses the Feynman-Kac theorem which is easily applicable only when r is bounded from below. Sec…
Decentralized Bayesian learning reduces KL-divergence exponentially.
problem Efficiently learning posterior distributions in a decentralized setting.
method Decentralized Langevin dynamics in a non-convex setting.
result The algorithm converges to the target posterior distribution with exponential decrease in KL-divergence and polynomial decrease in error contributions.
This paper optimizes high-dimensional oblique splits for decision trees, enhancing performance and computational efficiency.
problem Enhancing decision tree performance and computational efficiency in high-dimensional data.
method Established Sufficient Impurity Decrease (SID) convergence for -sparse oblique splits, proposing progressive trees for iterative refinement.
result Demonstrated that SID function class expands with -sparsity, enabling capture of complex data-generating processes.
Adam and RMSProp are two of the most influential adaptive stochastic algorithms for training deep neural networks, which have been pointed out to be divergent even in the convex setting via a few simple counterexamples. Many attempts, such as decreasing an adaptive learning rate, adopting a big batch size, incorporatin…
Normalization methods play an important role in enhancing the performance of deep learning while their theoretical understandings have been limited. To theoretically elucidate the effectiveness of normalization, we quantify the geometry of the parameter space determined by the Fisher information matrix (FIM), which als…
We consider training over-parameterized two-layer neural networks with Rectified Linear Unit (ReLU) using gradient descent (GD) method. Inspired by a recent line of work, we study the evolutions of network prediction errors across GD iterations, which can be neatly described in a matrix form. When the network is suffic…
In this article, we derive a Bayesian model to learning the sparse and low rank PARAFAC decomposition for the observed tensor with missing values via the elastic net, with property to find the true rank and sparse factor matrix which is robust to the noise. We formulate efficient block coordinate descent algorithm and …
It is known that evolution strategies in continuous domains might not converge in the presence of noise. It is also known that, under mild assumptions, and using an increasing number of resamplings, one can mitigate the effect of additive noise and recover convergence. We show new sufficient conditions for the converge…
Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by d…
Poly-view contrastive learning improves image representation learning.
problem Learning representations from multiple related views in image data.
method Developed new representation learning objectives for poly-view tasks using information maximization and sufficient statistics.
result Poly-view contrastive models trained for fewer epochs and with smaller batch sizes outperform models trained for more epochs and with larger batch sizes.
Let be a reduced alternating diagram of a non-split link and be the link whose diagram is obtained from by a crossing change. If is alternating, then . In this paper we explore when holds and obtain a simple sufficient and necessary cond…
How many samples are sufficient to guarantee that the eigenvectors and eigenvalues of the sample covariance matrix are close to those of the actual covariance matrix? For a wide family of distributions, including distributions with finite second moment and distributions supported in a centered Euclidean ball, we prove …
Investigating the Impact of Model Width and Density on Generalization in Presence of Label Noisecs.LG
This work reveals how label noise can cause a final ascent in neural network performance curves.
problem The impact of label noise on the performance of neural networks.
method Theoretical analysis and extensive experiments on various neural network architectures.
result Label noise can lead to a final ascent in the test loss curve, improving generalization at intermediate model widths.
Study optimal stopping times for multi-dimensional processes with non-exponential discounting.
problem Optimal stopping in multi-dimensional processes with non-exponential discounting.
method Probabilistic potential theory to establish existence of optimal equilibria.
result Existence of optimal equilibria for multi-dimensional stopping problems.
Investment decisions shift earlier as patience decreases, with implications for pasting conditions.
problem Investment timing under decreasing impatience.
method Game-theoretic framework with continuous-time capacity expansion problem.
result Decreasing impatience leads to earlier investment decisions, but can violate smooth pasting conditions.
Characterizes a general range decreasing group homomorphism.
problem Understanding range decreasing group homomorphisms in the entire mapping group.
method Characterization of a general range decreasing group homomorphism.
result Computes a particular class of homomorphisms and identifies all range decreasing group homomorphisms on specific mapping groups.
Study shows HFT benefits large traders under certain conditions.
problem Influence of high-frequency traders (HFTs) on large traders.
method Analyzes the impact of HFT front-running on large traders under different conditions.
result HFT benefits large traders when there is high-speed noise trading and vague HFT predictions.
Margulis space-times with parabolic holonomy elements are stable under sufficiently small deformations.
problem Stability of Margulis space-times with parabolic holonomy elements
method Combining compactification and partial generalization of earlier work
result Openness result on the number of conjugacy classes of parabolic elements under deformation
Most random ReLU networks are vulnerable to small, Euclidean adversarial perturbations.
problem Vulnerability of ReLU networks to adversarial attacks.
method Analysis of random ReLU networks with decreasing dimensions, using gradient flow and descent.
result Most examples can be perturbed by small Euclidean distances via gradient methods.
The study examines how investor protection and past information affect stock returns and interest rates.
problem Empirical regularities related to investor protection and past information in asset pricing models.
method Developed a dynamic asset pricing model with a controlling shareholder and good/bad memory in budget dynamics.
result Good/bad memory of investors on historical market information affects stock returns and interest rates, strengthening investor protection in high ownership concentration.
D2SRM solves complex PDEs using deep learning.
problem High-dimensional, Hessian-dependent fully nonlinear parabolic PDEs.
method Single scalar space-time network generating derivative-consistent approximations trained through residuals and penalties.
result Well-posedness and convergence theory established for globally Lipschitz equations.
We expect manifolds obtained by Dehn filling to inherit properties from the knot manifold. To what extent does that hold true for the Heegaard structure? We study four changes to the Heegaard structure that may occur after filling: (1) Heegaard genus decreases, (2) a new Heegaard surface is created, (3) a non-stabilize…
Mortgages and Refinancingq-fin.PR
In general, homeowners refinance in response to a decrease in interest rates, as their borrowing costs are lowered. However, it is worth investigating the effects of refinancing after taking the underlying costs into consideration. Here we develop a synthetic mortgage calculator that sufficiently accounts for such cost…
We propose a method to decrease the number of hidden units of the restricted Boltzmann machine while avoiding decrease of the performance measured by the Kullback-Leibler divergence. Then, we demonstrate our algorithm by using numerical simulations.
New method improves convergence for smooth games.
problem Improving convergence for smooth games.
method Stochastic Hamiltonian Gradient Methods (SHGD).
result SHGD converges linearly to the neighbourhood of a stationary point.
LMC loss barrier decreases to zero with large network width.
problem Understanding the LMC phenomenon in neural networks.
method Fine-grained analysis of LMC for two-layer ReLU networks.
result LMC loss barrier decreases to zero at a rate of O(m^-1/2) for large network width.
Informed traders strategically reveal noisier signals, making prices less responsive to public information.
problem How informed traders strategically reveal signals impacts market prices and utility.
method Modeling a market with an informed trader, an uninformed trader, and liquidity providers, proving equilibrium existence.
result In equilibrium, the insider strategically reveals a noisier signal, making prices less responsive to public information.
Sharp estimate for flow in any dimension.
problem Interior gradient estimate for graphical mean curvature flow.
method Proving sharp interior gradient estimate for area decreasing graphical mean curvature flow in arbitrary codimension.
result Generalized result in arbitrary codimension.
Large batch sizes reduce gradient variance in DP-SGD, improving privacy.
problem Understanding why large batch sizes work in DP-SGD.
method Decomposed total gradient variance into subsampling and noise-induced variances, proving batch size independence in the limit.
result Large batch sizes reduce effective total gradient variance, improving privacy in DP-SGD.
Paper proves genus of surfaces decreases in mean curvature flow.
problem Understanding the evolution of surfaces under mean curvature flow.
method Analyzes mean curvature flow of compact surfaces in 3-manifolds with specific curvature conditions.
result Genus of the regular set decreases over time.
GD monotonically decreases GFS sharpness in neural networks and scalar models.
problem Oscillatory behavior of loss in GD training.
method Analysis of GFS sharpness and empirical validation.
result GFS sharpness decreases monotonically during GD training.
In this paper, we study the problem of learning a mixture of Gaussians with streaming data: given a stream of points in dimensions generated by an unknown mixture of spherical Gaussians, the goal is to estimate the model parameters using a single pass over the data stream. We analyze a streaming version of …