Accelerates machine learning algorithms for sparse data.
problem Efficiently solving composite convex minimization problems.
method Accelerated dual-averaging primal-dual method for composite convex minimization.
result Demonstrates advantages in handling sparse data both theoretically and empirically.
New algorithm for federated learning with non-smooth regularizers.
problem Federated Learning with non-smooth composite optimization problems.
method Proposed Federated Dual Averaging (FedDualAvg) algorithm to overcome convergence issues.
result FedDualAvg outperforms other algorithms in federated composite optimization.
New methods reduce variance in stochastic dual averaging for sparse solutions.
problem Regularized empirical risk minimization problems in machine learning.
method Stochastic dual averaging with variance reduction for sparser solutions.
result Achieve best known convergence rates for both strongly and non-strongly convex regularizers.
DSPI connects natural policy gradient to policy iteration, proving global convergence.
problem Optimizing policies in reinforcement learning.
method DSPI framework, combining smoothed policy iteration and natural policy gradient.
result DSPI achieves geometric convergence and optimal complexity for policy optimization.
A new algorithm speeds up multi-agent reinforcement learning.
problem Complex interactions between agents in multi-agent reinforcement learning.
method Double averaging scheme for decentralized convex-concave saddle-point problems.
result The algorithm converges to the optimal solution at a global geometric rate.
New gossip algorithms solve decentralized optimization of pairwise functions.
problem Efficiently optimize global cost functions in decentralized networks.
method Gossip dual averaging algorithms for synchronous and asynchronous settings.
result Preserves convergence rate of centralized dual averaging with an additive bias.
PDA method optimizes neural networks with global convergence rate analysis.
problem Quantitative convergence rate for neural network optimization in mean field regime.
method Particle dual averaging (PDA) method, combining Langevin algorithm and outer loop optimization.
result Established quantitative global convergence for two-layer mean field neural networks.
New method tackles composite optimization with error feedback.
problem Challenges in distributed machine learning training and message compression.
method Combines Dual Averaging with EControl for composite optimization.
result First strong convergence analysis for composite optimization with error feedback.
This dissertation advances the theoretical foundation of local optimization methods in Federated Learning.
problem Theoretical understanding of local optimization methods in Federated Learning is lacking.
method The dissertation proposes and analyzes new methods to improve convergence rates and communication efficiency in Federated Learning.
result Sharp bounds and convergence rates for FedAvg are established, and new methods like FedAc and Federated Dual Averaging are proposed.
MDA optimizer performs similarly to SGD+M in CV and Adam in NLP.
problem Performance degradation due to choosing the wrong optimizer.
method Modernized Dual Averaging (MDA) optimizer, inspired by dual averaging.
result MDA performs as well as SGD+M in CV and as Adam in NLP.
OMD and DA perform similarly in static settings but OMD is inferior under dynamic learning rates.
problem Proving and understanding the performance difference between OMD and DA under dynamic learning rates.
method Introducing stabilization to OMD and modifying its convergence analysis.
result OMD with stabilization and DA have the same performance guarantees under dynamic learning rates.
Paper develops Byzantine-resilient algorithms for decentralized learning.
problem Vulnerability of distributed learning to Byzantine attacks.
method Dual approach for decentralized optimization.
result Convergence guarantees and experimental validation of the proposed algorithm.
Unified framework for entropy-regularized reinforcement learning in MDPs.
problem Entropy-regularized reinforcement learning in Markov decision processes.
method Extending linear programming to accommodate convex regularization functions.
result Using conditional entropy as regularization yields a dual problem similar to Bellman equations.
Discrete normal surfaces are normal surfaces whose intersection with each tetrahedron of a triangulation has at most one component. They are also natural Poincaré duals to 1-cocycles with $\ZZ/2\ZZ$-coefficients. For a fixed cohomology class in a simplicial poset the average Euler characteristic of the associated discr…
We propose a voted dual averaging method for online classification problems with explicit regularization. This method employs the update rule of the regularized dual averaging (RDA) method, but only on the subsequence of training examples where a classification error is made. We derive a bound on the number of mistakes…
The paper develops methods for welfare analysis in dynamic models.
problem Estimating welfare metrics in complex, high-dimensional models.
method Dual and doubly robust representations, Lasso and Neural Network estimators.
result Automatic debiasing of welfare metrics without needing bias correction.
Study on Santaló point for convex bodies in normed spaces.
problem Exploring Santaló point for convex bodies in normed spaces.
method Existence and uniqueness proof for C1 norms, dual Santaló point for smooth curved unit balls. result Existence and uniqueness of Santaló point for convex bodies in normed spaces.
A new online learning algorithm for graph-structured sparsity.
problem Efficiently handling graph-structured sparsity constraints in online learning settings.
method Proposes extsc{GraphDA} algorithm that projects gradients and variables onto subspaces.
result Improves classification performance and captures graph-structured features effectively.
New approach shapes error distribution in long-term forecasting.
problem Disparate error distributions in recent transformer models.
method Loss shaping constraints to respect upper bounds on loss at each time-step.
result Competitive average performance with shaped error distribution.
Develops a new method for robust risk measurement by averaging nearby payoffs.
problem Measuring risk under uncertainty with a focus on robustness.
method Averaging nearby payoffs weighted by a chosen metric.
result The method leads to a convex risk measure and provides stability under large neighborhoods.
New algorithms optimize spectral risk measures, improving interpolation between average and worst-case performance.
problem Optimizing spectral risk measures for learning systems.
method Developed stochastic algorithms to optimize spectral risk measures by characterizing their subdifferential and addressing challenges like biasedness of subgradient estimates and non-smoothness.
result Our approach outperforms out-of-the-box stochastic subgradient and dual averaging methods in optimizing spectral risk measures.
Proposes a computational framework for real-time risk assessment and prioritization.
problem Real-time risk assessment and prioritization for uncertain outcomes.
method Develops a computational framework based on satisficing measure for real-time risk assessment and prioritization. Applies sample average approximation and primal-dual stochastic approximation algorithms.
result Demonstrates the effectiveness of the proposed framework in real-time risk assessment and prioritization.
The Ricci tensor (Ric) is fundamental to Einstein's geometric theory of gravitation. The 3-dimensional Ric of a spacelike surface vanishes at the moment of time symmetry for vacuum spacetimes. The 4-dimensional Ric is the Einstein tensor for such spacetimes. More recently the Ric was used by Hamilton to define a non-li…
In this work we develop a new algorithm for regularized empirical risk minimization. Our method extends recent techniques of Shalev-Shwartz [02/2015], which enable a dual-free analysis of SDCA, to arbitrary mini-batching schemes. Moreover, our method is able to better utilize the information in the data defining the ER…
This paper advances FL algorithms for composite optimization and statistical recovery.
problem Federated learning optimization and statistical recovery in composite settings.
method Proposes Fast Federated Dual Averaging for strongly convex and smooth loss, and Multi-stage Federated Dual Averaging for restricted strongly convex and smooth loss.
result Establishes state-of-the-art iteration and communication complexity, and high probability complexity bound with linear speedup.
Optimizes stochastic convex optimization with local minimax theory and adaptive methods.
problem Optimizing stochastic convex optimization problems with local complexity measures.
method Local minimax theory, adaptive fully online methods, Nesterov's dual averaging, Riemannian stochastic gradient methods.
result Achieves optimal convergence guarantees for stochastic convex optimization problems.
Optimizes wireless network resource management with state-augmented policies.
problem Optimizing network-wide utility with user performance constraints.
method State-augmented parameterization of RRM policy, using dual variables.
result Superior trade-off between mean, minimum, and 5th percentile rates.
New method uses LP to achieve optimal sample complexity in multi-agent reinforcement learning.
problem Achieving global optimality in multi-agent reinforcement learning with average-cost criterion.
method Randomized Linear Programming and Stochastic Primal-Dual Methods for multi-agent saddle point problems.
result Sample complexity matches tight dependencies on state and action spaces, and scales with network size.
Communication remains the most significant bottleneck in the performance of distributed optimization algorithms for large-scale machine learning. In this paper, we propose a communication-efficient framework, CoCoA, that uses local computation in a primal-dual setting to dramatically reduce the amount of necessary comm…
A new GAN algorithm using primal-dual formulations of optimal transport.
problem Building latent variable models of data distributions.
method Primal formulation for inference and dual formulation for adversarial training.
result Improves mode coverage and avoids averaging properties of auto-encoding models.
The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. It arises in various application domains, including distributed tracking and localization, multi-agent co-ordination, est…
We construct a discrete form of Hamilton's Ricci flow (RF) equations for a d-dimensional piecewise flat simplicial geometry, S. These new algebraic equations are derived using the discrete formulation of Einstein's theory of general relativity known as Regge calculus. A Regge-Ricci flow (RRF) equation is naturally asso…
This paper shows how to use ℓ1 regularization effectively in training sparse CNNs.
problem Why ℓ1 regularization hasn't been used in sparse deep learning models like CNNs. method Demonstrated that SGD is not suitable for ℓ1 regularization and replaced it with a new training algorithm based on regularized dual averaging (RDA). result Achieved state-of-the-art sparsity for CNNs using RDA with ℓ1 regularization, achieving 95% sparsity for ResNet18 on CIFAR-10. Researchers calculate the precise boundary operator for interacting bulk scalar fields in AdS/CFT.
problem Understanding the precise form of boundary operators dual to interacting bulk scalar fields.
method Holographic renormalization coupled with the Caffarelli/Silvestre extension theorem.
result Boundary operator dual to a bulk scalar field is an anti-local operator, the fractional Laplacian.
Paper tackles class-incremental time series classification with dual-stream feature extraction.
problem Class-incremental continual learning for multivariate time series data.
method Dual-stream feature extraction pipeline combining deep temporal embedding features and statistical features.
result Competitive average accuracy across multiple datasets with low forgetting rates.
In this paper we present an optimization-based view of distributed parameter estimation and observational social learning in networks. Agents receive a sequence of random, independent and identically distributed (i.i.d.) signals, each of which individually may not be informative about the underlying true state, but the…
Optimizes power allocation for WDM in RoFSO systems.
problem Maximizing total capacity with power and eye safety constraints.
method Model-based Stochastic Dual Gradient algorithm and model-free Primal-Dual Deep Learning algorithm.
result Deep Learning algorithm outperforms average equal power allocation.
This research develops a dual-level reinforcement learning strategy to track daily VWAP accurately.
problem Inaccurate tracking of daily VWAP due to short trading horizons.
method Dual-level architecture using Transformer and LSTM models.
result Improves accuracy in approximating daily VWAP compared to previous models.
New algorithms optimize without knowing problem parameters.
problem Optimizing large-scale problems without knowing key parameters.
method Combining mirror descent with dual averaging techniques.
result Converges without prior knowledge of problem parameters.
Framework explains how dual deep networks learn features from unlabeled data.
problem Understanding self-supervised learning with dual deep networks.
method Theoretical framework and hierarchical latent tree model.
result Deep ReLU networks learn latent variables through contrastive SSL.
New algorithm optimizes stochastic optimization with circular dependency.
problem Circular dependency between decision variable and importance sampling.
method Single-loop stochastic approximation algorithm based on Nesterov's dual averaging.
result Achieves minimal asymptotic variance and resolves circular optimization challenge.
Two signature-based methods solve optimal stopping in non-Markovian frameworks.
problem Optimal stopping in non-Markovian frameworks, particularly pricing American options.
method Primal and dual formulations using linear functionals of rough path signatures.
result Both primal and dual methods converge and provide numerical examples.
New algorithm learns optimal resource allocation in wireless systems without models.
problem Learning optimal resource allocation in wireless systems without system models.
method Developed a model-free primal-dual algorithm using smoothed surrogates of constrained problems.
result The algorithm can make the gap between optimal values and dual values arbitrarily small.
Algorithm for pricing American options using martingale approximations.
problem Pricing American options efficiently and accurately.
method Approximating uniformly square integrable martingales with Wiener chaos expansion, solving the dual minimization problem via sample average approximation.
result Scalable parallel implementation for multi-dimensional path-dependent options.
In this work we consider optimal stopping problems with conditional convex risk measures called optimised certainty equivalents. Without assuming any kind of time-consistency for the underlying family of risk measures, we derive a novel representation for the solution of the optimal stopping problem. In particular, we …
ProxQuant improves quantized neural networks using proximal operators.
problem Making neural networks work on devices with limited resources.
method Formulates quantized network training as a regularized learning problem and optimizes it via the prox-gradient method.
result ProxQuant outperforms state-of-the-art results on binary quantization and is on par with state-of-the-art on multi-bit quantization.
A new parallel algorithm for learning optimal policies in MDPs with low communication costs.
problem Learning optimal policies for infinite-horizon MDPs.
method Primal-Dual Stochastic Mirror Descent for convex programming problems with inexact constraints.
result First parallel algorithm for average-reward MDPs with generative model and low communication costs.
Paper accelerates distributed optimization in growing networks.
problem Improving convergence rate of distributed optimization in evolving networks.
method Extending DDA to growing networks, optimizing edge selection and scheduling.
result DDA convergence rate improves with growing network connectivity.