New method avoids saddle points without gradients.
problem Optimizing non-convex functions efficiently.
method Zero-order derivative-free algorithm using only function evaluations.
result Converges to second-order stationary points efficiently.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “zero order methods”
The paper analyzes the efficiency of gradient estimation methods in noisy function evaluations.
problem Estimating gradients of smooth functions using noisy function evaluations.
method Information-theoretic lower bounds and finite difference method analysis.
result The finite difference method is not minimax optimal, suggesting room for improvement in gradient estimation.
Study optimizes zero-order strongly convex function minimization with higher order smoothness.
problem Optimizing a strongly convex function with noisy evaluations.
method Randomized approximation of projected gradient descent with smoothing kernel.
result Upper bounds and minimax lower bounds for the algorithm, showing near-optimality.
In a previous analysis the problem of "zero-inflated" time data (caused by high frequency trading in the electronic order book) was handled by left-truncating the inter-arrival times. We demonstrated, using rigorous statistical methods, that the Weibull distribution describes the corresponding stochastic dynamics for a…
A new algorithm optimizes softmax units in large language models.
problem Efficiently computing gradients for large-scale language models.
method Zero-th Order method for approximating gradients.
result The algorithm converges and efficiently computes gradients.
CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.
problem Byzantine attacks and communication inefficiency in federated learning.
method Transformed robust aggregation for zero-order optimization under client heterogeneity.
result CyBeR-0 achieves stable performance with minimal communication costs and reduced memory usage.
Recently, we proposed to transform the outputs of each hidden neuron in a multi-layer perceptron network to have zero output and zero slope on average, and use separate shortcut connections to model the linear dependencies instead. We continue the work by firstly introducing a third transformation to normalize the scal…
The class of second order ODE's cubic with respect to the first order derivative is considered. Using geometric structures associated with these equations, the subclasses of umbilical equations, zero mean curvature equations, and zero Gaussian curvature equations are defined. Zero mean curvature equations are studied w…
We extend the Heegaard Floer homological definition of spectral order for closed contact 3-manifolds due to Kutluhan, Matić, Van Horn-Morris, and Wand to contact 3-manifolds with convex boundary. We show that the order of a codimension zero contact submanifold bounds the order of the ambient manifold from above. As the…
A gradient estimator via L1-randomization for online zero-order optimization with two point feedbackmath.ST
A new gradient estimator for online optimization with two function evaluations.
problem Online optimization of convex and Lipschitz functions with noisy data.
method L1-randomization approach for gradient estimation.
result Compared or better guarantees than previous methods for canceling noise.
Study meromorphic k-differentials with prescribed singularities on Riemann surfaces.
problem Understanding local invariants of meromorphic k-differentials on Riemann surfaces.
method Analyzing orders of zeros and poles, and k-residues at poles.
result For a given pattern of zeros, there exists a primitive holomorphic k-differential with these zeros.
Paper proposes copula-based models for analyzing multivariate zero-inflated continuous data.
problem Challenges in analyzing multivariate zero-inflated continuous data with mixed discreteness and continuity.
method Proposes two copula-based density estimation models and rectified Gaussian copula.
result Demonstrates superior performance compared to conventional methods.
ZeroS improves Transformers by adding negative weights, matching or beating softmax attention.
problem Limited performance of linear attention methods, especially in long context sequences.
method Proposes Zero-Sum Linear Attention (ZeroS) that removes the zero-order term and reweights zero-sum softmax residuals.
result ZeroS matches or exceeds standard softmax attention across various benchmarks, theoretically expanding representable functions.
Gradient descent struggles to achieve zero loss in deep learning models due to non-generic data distributions.
problem Achieving zero loss minimizers in deep learning networks.
method Analysis of gradient descent algorithm in deep learning, focusing on underparametrized networks.
result Zero loss minimization cannot be achieved generically in deep learning networks.
The paper explores zero-divisors and idempotents in quandle rings, proving their absence in certain cases.
problem Understanding zero-divisors and idempotents in quandle rings.
method Development of quandle rings theory, definition of orderability, computation of idempotents, and analysis of automorphism groups.
result Quandle rings of left or right orderable quandles with semi-latin structure have no zero-divisors.
Enhanced Tweedie model for insurance claims using CatBoost.
problem Accurately modeling aggregate claims with zero-inflated data.
method Refined Tweedie model with boosting methods in CatBoost.
result Marked improvement in model performance for insurance analytics.
In this paper we prove that, given a compact four dimensional smooth Riemannian manifold (M,g) with smooth boundary there exists a metric conformal to g with constant T-curvature, zero Q-curvature and zero mean curvature under generic and conformally invariant assumptions. The problem amounts to solving a fourth order …
Introduces SM-games to analyze machine learning interactions.
problem Lack of understanding and control in n-player games.
method Introduces SM-games with pairwise zero-sum interactions.
result SM-games are amenable to first-order optimization methods.
We obtain an ordering of closed aspherical 4-manifolds that carry a non-hyperbolic Thurston geometry. As application, we derive that the Kodaira dimension of geometric 4-manifolds is monotone with respect to the existence of maps of non-zero degree.
The paper calculates area Siegel--Veech constants for specific submanifolds of REL zero.
problem Calculating area Siegel--Veech constants for affine invariant submanifolds of REL zero.
method Using volumes of the principal boundary strata and intersection theory.
result Proves a conjectural formula for the area Siegel--Veech constant in the case of REL zero.
Renormalized pruning improves neural network accuracy.
problem Over-parameterized neural networks waste many parameters.
method Propose renormalizing sparse neural networks.
result Renormalized pruning converges to zero error.
We study derivative-free methods for policy optimization over the class of linear policies. We focus on characterizing the convergence rate of these methods when applied to linear-quadratic systems, and study various settings of driving noise and reward feedback. We show that these methods provably converge to within a…
In this paper we provide a family of algebraic space-like surfaces in the three dimensional anti de Sitter space that shows that this Lorentzian manifold admits algebraic maximal examples of any order. Then, we classify all the space-like order two algebraic maximal hypersurfaces in the anti de Sitter -dimensional s…
Defines a new knot invariant and studies its properties.
problem Understanding the structure of knot invariants.
method Defining and analyzing the twisted Alexander vanishing order.
result Characterizes knots with a zero-twisted Alexander polynomial.
The problem of resource allocation of nonlinear networked control systems is investigated, where, unlike the well discussed case of triggering for stability, the objective is optimal triggering. An approximate dynamic programming approach is developed for solving problems with fixed final times initially and then it is…
We study the local invariants that a meromorphic -differential on a Riemann surface of genus can have. These local invariants are the orders of zeros and poles, and the -residues at the poles. We show that for a given pattern of orders of zeroes, there exists, up to a few exceptions, a primitive -diff…
In this paper we study some fourth order elliptic equation involving the critical Sobolev exponent, related to the prescription of a fourth order conformal invariant on the standard sphere. We use a topological method to prove the existence of at least a solution when the function to be prescribed is close to a constan…
New algorithm optimizes convex functions with noisy evaluations in one dimension.
problem Optimizing convex functions with noisy zero-order evaluations in one dimension.
method Proposed a computationally efficient algorithm achieving convergence rate.
result Achieved the optimal convergence rate, closing the gap in one dimension.
A new method for distributed optimization with noisy function evaluations.
problem Distributed optimization with noisy function evaluations.
method Zero-order one-point estimate with distributed stochastic gradient-tracking technique.
result The method converges almost surely to the optimum with a rate of .
Homology of abelian differentials stabilizes with more zeros.
problem Understanding the homology of abelian differentials with many simple zeros.
method Developed an h-principle for these strata, valid in a range of homological degrees.
result Homology stabilizes in a range where the number of simple zeros is large.
We study the market impact of a meta-order in the framework of the Minority Game. This amounts to studying the response of the market when introducing a trader who buys or sells a fixed amount h for a finite time T. This perturbation introduces statistical arbitrages that traders exploit by adapting their trading strat…
Study improves variance calculation for random zero sets on complex manifolds.
problem Improving the variance calculation for random zero sets on complex manifolds.
method Deriving an asymptotic expansion for the variance of linear statistics of zero divisors of random holomorphic sections.
result Sharpens leading-order asymptotics for the variance of random zero sets.
New approach to handle ranking function variation in zero-shot NAS.
problem Variation in ranking function outputs due to randomness.
method Viewing ranking function output as a random variable and constructing a stochastic ordering.
result Stochastic ordering boosts performance in neural architecture search.
We define a set of "second-order" L^(2)-signature invariants for any algebraically slice knot. These obstruct a knot's being a slice knot and generalize Casson-Gordon invariants, which we consider to be "first-order signatures". As one application we prove: If K is a genus one slice knot then, on any genus one Seifert …
The paper diagnoses factor models using characteristic axes and zero-curve restrictions.
problem Tackles systematic sign reversals and overcorrections in factor model pricing errors.
method Extends cap-axis integral diagnostic to general characteristic axes, measuring pricing errors as bridge-alpha curves.
result Axis-level pricing errors are nearly orthogonal to maximum-Sharpe gains, showing systematic sign reversals and overcorrections.
Mechanical vs. informational components of price impactphysics.soc-ph
We study the problem of what causes prices to change. We define the mechanical impact of a trading order as the change in future prices in the absence of any future changes in decision making, and its it informational impact as the remainder of the total impact once mechanical impact is removed. We introduce a method o…
Third-order symmetric Lorentzian manifolds, i.e. Lorentzian manifold with zero third derivative of the curvature tensor, are classified. These manifolds are exhausted by a special type of pp-waves, they generalize Cahen-Wallach spaces and second-order symmetric Lorentzian spaces.
Derivative-Free Sequential Quadratic Programming for Equality-Constrained Stochastic Optimizationmath.OC
Derivative-free method solves stochastic optimization problems with noisy objectives and constraints.
problem Solving nonlinear optimization problems with stochastic objectives and deterministic constraints using only zero-order information.
method Derivative-Free Stochastic Sequential Quadratic Programming (DF-SSQP) method using simultaneous perturbation stochastic approximation (SPSA) for gradient and Hessian estimation.
result Global almost-sure convergence of the DF-SSQP method under standard assumptions, with local asymptotic normality and statistical inference.
We analyze analytic approximation formulae for pricing zero-coupon bonds in the case when the short-term interest rate is driven by a one-factor mean-reverting process with a volatility nonlinearly depending on the interest rate itself. We derive the order of accuracy of the analytical approximation due to Choi and Wir…
Zero-inflated datasets, which have an excess of zero outputs, are commonly encountered in problems such as climate or rare event modelling. Conventional machine learning approaches tend to overestimate the non-zeros leading to poor performance. We propose a novel model family of zero-inflated Gaussian processes (ZiGP) …
Study describes how to realize periods of meromorphic differentials with specific properties.
problem Realizing meromorphic differentials with given zeros, poles, and topological constraints.
method Complete description of period representations for specified conditions on Riemann surfaces.
result A comprehensive method for realizing meromorphic differentials with prescribed characteristics.
Bayesian methods suffer from the problem of how to specify prior beliefs. One interesting idea is to consider worst-case priors. This requires solving a stochastic zero-sum game. In this paper, we extend well-known results from bandit theory in order to discover minimax-Bayes policies and discuss when they are practica…
Equity auctions show linear price impact up to a large volume, then non-linear.
problem Understanding price impact in equity auctions.
method High-quality data analysis of Paris stock exchange auctions.
result Price impact is linear up to a large volume, then becomes non-linear.
A new method infers causal structures and generates data without DAGs.
problem Modeling causal relationships without DAGs.
method Fixed-point approach on causally ordered variables, amortized TO inference, transformer-based SCM learning.
result The model learns TOs and SCMs from data, outperforming baselines.
Deep ResNets can achieve zero loss with enough layers and weights.
problem Finding parameters in deep ResNets that fit training data perfectly.
method Mean-field analysis and gradient descent flow to a PDE.
result Gradient descent for ResNet training converges to a zero-loss solution in the large-NN limit.
Formula for Masur-Veech volumes in quadratic differentials with odd zeros.
problem Calculating volumes of specific quadratic differential strata.
method Intersection theory, topological recursion, Hodge integrals.
result Conjectural formula for volumes proved for odd zero orders.
We consider the closely related problems of bandit convex optimization with two-point feedback, and zero-order stochastic convex optimization with two function evaluations per round. We provide a simple algorithm and analysis which is optimal for convex Lipschitz functions. This improves on \cite{dujww13}, which only p…
A classical result of Aubin states that the constant in Moser-Trudinger-Onofri inequality on can be imporved for furnctions with zero first order moments of the area element. We generalize it to higher order moments case. These new inequalities bear similarity to a sequence of Lebedev-Milin type inequa…