FAST optimizes additive segmentation for faster, more interpretable models.
problem Efficiently segmenting and interpreting complex datasets.
method Optimization framework for fast piecewise constant shape functions.
result 2 orders of magnitude faster than state-of-the-art methods.
New framework explains fast transfer of hyperparameters across model scales.
problem Understanding and optimizing hyperparameters for large-scale models.
method Developed a conceptual framework for HP transfer across scale, showing fast transfer is equivalent to useful transfer for compute-optimal grid search.
result Fast transfer of hyperparameters is equivalent to useful transfer for compute-optimal grid search, offering asymptotic computational advantage.
This paper shows how to learn variational inequalities fast with strong monotonicity.
problem Learning variational inequalities efficiently.
method Extending convex optimization techniques to variational inequalities with strong monotonicity.
result Fast generalization rates of Θ(1/ε) for learning variational inequalities. This paper studies the portfolio optimization problem when the investor's utility is general and the return and volatility of the risky asset are fast mean-reverting, which are important to capture the fast-time scale in the modeling of stock price volatility. Motivated by the heuristic derivation in [J.-P. Fouque, R. …
Bayesian optimization for long-term outcomes using fast and slow experiments.
problem Optimizing long-term system effects with short-term misleading results.
method Combining fast and slow experiments for Bayesian optimization.
result Sequential optimization over large action spaces in a short time.
PPGD solves nonconvex nonsmooth optimization problems without KL property.
problem Nonconvex and nonsmooth optimization problems in statistics and machine learning.
method Projective Proximal Gradient Descent (PPGD) for solving a class of nonconvex and nonsmooth problems.
result PPGD achieves a fast convergence rate of O(1/k^2) for k ≥ k_0.
Paper optimizes multi-fidelity function with fast learning rates.
problem Optimizing a locally smooth function with limited budget and varying fidelity approximations.
method Kometo algorithm that achieves simple regret rates without knowing function smoothness or fidelity assumptions.
result Kometo algorithm outperforms previous methods empirically.
Optimizes trading large volumes of volatile assets with fast mean-reverting volatility.
problem Challenges of executing large volumes of illiquid or volatile assets.
method Modeling uncertain volatility and liquidity with fast mean-reverting dynamics, using singular perturbation arguments and high-frequency data.
result Approximately optimal trade execution strategies under fast mean-reversion.
Paper introduces a new gradient statistic to improve deep learning convergence.
problem Fluctuation effect of gradient updates between iterations.
method Introduces an unbiased stratified statistic \(\bar{G}_{mst}\) and a new algorithm MSSG.
result MSSG algorithm outperforms other sgd-like algorithms in training deep models.
Limbo is an open-source C++11 library for Bayesian optimization which is designed to be both highly flexible and very fast. It can be used to optimize functions for which the gradient is unknown, evaluations are expensive, and runtime cost matters (e.g., on embedded systems or robots). Benchmarks on standard functions …
Fast algorithm for online optimization on transport polytopes.
problem Optimizing convex objectives on transport polytopes.
method Mirror Sinkhorn algorithm combining Sinkhorn scaling and mirror descent.
result Robust and efficient online optimization for convex objectives.
We describe a new optimization scheme for finding high-quality correlation clusterings in planar graphs that uses weighted perfect matching as a subroutine. Our method provides lower-bounds on the energy of the optimal correlation clustering that are typically fast to compute and tight in practice. We demonstrate our a…
SFPO optimizes LLM reasoning by repositioning before updating, improving stability and efficiency.
problem Noisy gradients from low-quality rollouts cause instability and inefficient exploration in on-policy RL algorithms.
method Decomposes each step into three stages: a short fast trajectory, repositioning, and slow correction, preserving the objective and rollout process unchanged.
result SFPO consistently improves stability, reduces rollouts, and accelerates convergence, outperforming GRPO on math reasoning benchmarks.
Error bound conditions (EBC) are properties that characterize the growth of an objective function when a point is moved away from the optimal set. They have recently received increasing attention in the field of optimization for developing optimization algorithms with fast convergence. However, the studies of EBC in st…
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form minx∑ifi(x). Specifically, we analyze the SAGA algorithm within an Incremental First-order Oracle framework, and show that it converges to a stationary point provably faster than both gradient descent and s…
We provide a way to infer about existence of topological circularity in high-dimensional data sets in Rd from its projection in R2 obtained through a fast manifold learning map as a function of the high-dimensional dataset X and a particular choice of a positive real σ known as band…
This paper advances FL algorithms for composite optimization and statistical recovery.
problem Federated learning optimization and statistical recovery in composite settings.
method Proposes Fast Federated Dual Averaging for strongly convex and smooth loss, and Multi-stage Federated Dual Averaging for restricted strongly convex and smooth loss.
result Establishes state-of-the-art iteration and communication complexity, and high probability complexity bound with linear speedup.
Empirical studies indicate the presence of multi-scales in the volatility of underlying assets: a fast-scale on the order of days and a slow-scale on the order of months. In our previous works, we have studied the portfolio optimization problem in a Markovian setting under each single scale, the slow one in [Fouque and…
Paper improves a method for fast global and local convergence in optimization.
problem Slow global convergence in optimization methods with noisy Hessian estimates.
method Stochastic Newton Proximal Extragradient method using HPE framework.
result Faster global linear rate and superlinear convergence in fewer iterations.
Quantized Variational Inference improves ELBO optimization with fast convergence.
problem Maximizing Evidence Lower Bound (ELBO) for variational inference.
method Optimal Voronoi Tesselation for variance-free gradients, Richardson extrapolation for asymptotic improvement.
result Quantized Variational Inference leads to fast convergence with comparable computational cost.
Kernel density estimation (KDE) is a popular statistical technique for estimating the underlying density distribution with minimal assumptions. Although they can be shown to achieve asymptotic estimation optimality for any input distribution, cross-validating for an optimal parameter requires significant computation do…
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limited. For example, it is not known whether the proximal stochastic gra…
A new estimator for asynchronous tick data shows better correlation estimates.
problem Estimating correlation from asynchronous tick data.
method Derive a minimum-variance estimator and a fast linear-time estimator.
result The fast tickwise estimator has smaller estimation errors than the usual method.
By reducing optimization to a sequence of smaller subproblems, working set algorithms achieve fast convergence times for many machine learning problems. Despite such performance, working set implementations often resort to heuristics to determine subproblem size, makeup, and stopping criteria. We propose BlitzWS, a wor…
A fast, robust AMP algorithm for quadratic optimization problems.
problem Implementing robust approximate-message passing algorithms for quadratic optimization problems.
method Spectral pre-processing and mild modification of AMP algorithm iterates.
result Output solution close to AMP algorithm output for perturbed inputs.
In this paper, we develop an online method that leverages machine learning to obtain feasible solutions to the AC optimal power flow (OPF) problem with negligible optimality gaps on extremely fast timescales (e.g., milliseconds), bypassing solving an AC OPF altogether. This is motivated by the fact that as the power gr…
Fractional stochastic volatility models have been widely used to capture the non-Markovian structure revealed from financial time series of realized volatility. On the other hand, empirical studies have identified scales in stock price volatility: both fast-time scale on the order of days and slow-scale on the order of…
New method speeds up training of deep networks robust to adversarial attacks.
problem Deep networks are sensitive to adversarial perturbations, compromising security and interpretability.
method Fast adversarial training using Euclidean norm approximation and distributed computing.
result Robust feature representations and reduced training time achieved.
M-L2O adapts fast to new tasks by self-adapting during test-time.
problem L2O optimizers struggle with out-of-distribution tasks.
method Meta-training an L2O optimizer to adapt quickly to new tasks.
result M-L2O converges significantly faster than vanilla L2O with only 5 steps of adaptation.
Paper explores fast adversarial training to improve robustness with less computation.
problem Efficiently defending against adversarial examples.
method Integrates simple self-attacks for faster training, focusing on overfitting recovery.
result Shows superior robust accuracy with reduced training time compared to strong adversarial training.
A faster method for optimizing DNA and protein sequences using machine learning.
problem Designing DNA and protein sequences with improved function.
method Activation maximization with a straight-through approximation and adaptive entropy variable.
result Fast SeqProp achieves up to 100-fold faster convergence and improved fitness optima.
Paper proposes efficient optimizers for large language models with fast convergence and low memory usage.
problem Designing efficient optimizers for large language models with low-memory requirements and fast convergence.
method Structured Fisher information matrix approximation and low-rank extension framework.
result New optimizers (RACS and Alice) achieve better convergence and lower memory usage than existing methods.
A new method for fast optimal transport using sliced Wasserstein generalized geodesics.
problem Computing optimal transport distances efficiently and accurately.
method Proposes a new proxy of squared Wasserstein distance based on one-dimensional projections.
result min-SWGG is an upper bound of Wasserstein distance with similar computational complexity.
Study shows how to learn optimal policies quickly in stochastic control problems.
problem Learning optimal policies in large, continuous state and action spaces with limited data.
method Analyzes three geometric exponents to quantify fast policy regret convergence.
result Shows that fast policy regret convergence is induced by specific geometric structures.
This paper deals with two related problems, namely distance-preserving binary embeddings and quantization for compressed sensing . First, we propose fast methods to replace points from a subset X⊂Rn, associated with the Euclidean metric, with points in the cube {±1}m and we associa…
Paper develops fast method for computing optimal transport.
problem Efficient computation of optimal transport distance between distributions.
method Entropy-regularized extragradient method for first-order optimization.
result Achieves state-of-the-art runtime guarantees and good numerical performance.
PyBADS optimizes complex functions quickly and reliably.
problem Optimizing rough, noisy, and expensive functions with unknown gradients.
method Bayesian Adaptive Direct Search (BADS) algorithm.
result PyBADS performs well on both artificial and real-world problems.
A new fast algorithm solves SLOPE optimization problem.
problem Inefficient algorithms for SLOPE optimization in high dimensions.
method Combines proximal gradient descent and proximal coordinate descent steps.
result Our method outperforms competing algorithms in benchmarks.
What is a good exploration strategy for an agent that interacts with an environment in the absence of external rewards? Ideally, we would like to get a policy driving towards a uniform state-action visitation (highly exploring) in a minimum number of steps (fast mixing), in order to ease efficient learning of any goal-…
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
GraPhyR uses GNNs to optimize power grid reconfiguration in real-time.
problem Optimizing power grid reconfiguration for reliability and efficiency.
method Physics-informed Graph Neural Network (GNN) framework.
result GraPhyR learns to optimize DyR tasks efficiently.
This paper speeds up OCSSVM training using SMO.
problem Training One-Class Slab SVMs is slow.
method Uses updated SMO to divide large problems into smaller, analytically solvable subproblems.
result Training OCSSVMs scales better with large datasets.
Study shows convergence rate for empirical minimizer of unbounded functions with fast growth.
problem Convergence rate of empirical minimizer for unbounded functions with fast growth.
method Analyzes L1-distance convergence rate of the empiric minimizer for coercive functions sampled with noise. result Convergence rate is bounded above by ann−1/q, where q is the dimension and an=o(nε) for every ε>0. While optimizing convex objective (loss) functions has been a powerhouse for machine learning for at least two decades, non-convex loss functions have attracted fast growing interests recently, due to many desirable properties such as superior robustness and classification accuracy, compared with their convex counterpa…
Empirical studies indicate the existence of long range dependence in the volatility of the underlying asset. This feature can be captured by modeling its return and volatility using functions of a stationary fractional Ornstein--Uhlenbeck (fOU) process with Hurst index H∈(21,1). In this paper, we analyz…
Paper tackles fast convergence for non-convex strongly-concave min-max problems.
problem Non-convex strongly-concave min-max problems in deep learning.
method Proximal stage-based method with PL condition for faster convergence.
result Established fast convergence in primal objective gap and duality gap.
We consider a single security market based on a limit order book and two investors, with different speeds of trade execution. If the fast investor can front-run the slower investor, we show that this allows the fast trader to obtain risk free profits, but that these profits cannot be scaled. We derive the fast trader's…
A fast method for decentralized non-convex optimization over networks.
problem Decentralized non-convex optimization problems over a network of nodes.
method GT-SAGA, a randomized incremental gradient method that evaluates one component gradient per node per iteration.
result GT-SAGA achieves almost sure and mean-squared convergence to a first-order stationary point for general smooth non-convex problems.