Federated learning algorithm improves with intermittent client availability.
problem Performance degradation in Federated Averaging due to client availability changes.
method Federated Latest Averaging (FedLaAvg) uses latest gradients from all clients, even when unavailable.
result FedLaAvg achieves sublinear speedup compared to classical Federated Averaging.
In distributed statistical learning, N samples are split across m machines and a learner wishes to use minimal communication to learn as well as if the examples were on a single machine. This model has received substantial interest in machine learning due to its scalability and potential for parallel speedup. Howev…
Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to perform inference in log-linear models with sublinear amor…
Quantum algorithms reduce clustering input size, achieving near-linear approximation.
problem Efficiently clustering large datasets in quantum computing.
method Quantum coresets for k-clustering with sublinear query complexity. result Achieves near-linear approximation for k-clustering with coresets. pFedMe uses Moreau envelopes to improve personalized FL performance.
problem Statistical diversity among clients limits FL performance.
method pFedMe employs Moreau envelopes as clients' regularized loss functions in a bi-level optimization problem.
result pFedMe achieves state-of-the-art convergence rates and superior empirical performance.
A wide range of fundamental machine learning tasks that are addressed by the maximum a posteriori estimation can be reduced to a general minimum conical hull problem. The best-known solution to tackle general minimum conical hull problems is the divide-and-conquer anchoring learning scheme (DCA), whose runtime complexi…
This paper represents a preliminary (pre-reviewing) version of a sublinear variational algorithm for isotropic Gaussian mixture models (GMMs). Further developments of the algorithm for GMMs with diagonal covariance matrices (instead of isotropic clusters) and their corresponding benchmarking results have been published…
The paper shows how sublinear biLipschitz equivalences affect Morse boundaries of metric spaces.
problem Understanding how sublinear biLipschitz equivalences affect Morse boundaries of metric spaces.
method Defining sublinear biLipschitz equivalence and Morse boundaries, proving invariance under SBEs, using sublinear rays.
result κ-Morse boundaries of proper geodesic metric spaces are invariant under suitable sublinear biLipschitz equivalences.
Efficient Maximum Inner Product Search (MIPS) is an important task that has a wide applicability in recommendation systems and classification with a large number of classes. Solutions based on locality-sensitive hashing (LSH) as well as tree-based solutions have been investigated in the recent literature, to perform ap…
Federated learning for combinatorial multi-agent bandits reduces regret and speeds up with fewer communications.
problem Online combinatorial optimization with noisy feedback and cooperation.
method Transforms offline algorithms into online multi-agent algorithms with sublinear regret and communication efficiency.
result Achieves sublinear regret and linear speedup with more agents, communication-efficient.
Sublinear LSVI via LSH reduces runtime to sublinear in actions.
problem Efficiently estimating value functions in reinforcement learning with sublinear runtime.
method Formulated as approximate maximum inner product search, used LSH to solve with sublinear time complexity.
result Sublinear runtime while maintaining LSVI's regret.
The aim of this paper is to introduce the sublinear Higson corona and show that the sublinear Higson corona of Euclidean cone of P and X is decomposed into the product of P and that of X. Here P is a compact metric space and X is unbounded proper metric space. For example, the sublinear Higson corona of n-dimensional E…
New model for Knightian uncertainty with jumps.
problem Knightian uncertainty and non-linear jumps.
method Probabilistic construction of non-linear affine processes with jumps.
result Tractable model for Knightian uncertainty with sublinear expectations.
We give a proof of the sublinear tracking property for sample paths of random walks on various groups acting on spaces with hyperbolic-like properties. As an application, we prove sublinear tracking in Teichmueller distance for random walks on mapping class groups, and on Cayley graphs of a large class of finitely gene…
We provide a general construction of time-consistent sublinear expectations on the space of continuous paths. It yields the existence of the conditional G-expectation of a Borel-measurable (rather than quasi-continuous) random variable, a generalization of the random G-expectation, and an optional sampling theorem that…
ProxSkip achieves linear speedup in distributed non-convex optimization.
problem Achieving linear speedup in distributed non-convex optimization.
method Unified convergence analysis for stochastic non-convex, convex, and strongly convex problems.
result ProxSkip achieves linear speedup in the number of nodes under stochastic gradients.
New sublinear sketches improve ANN and KDE for massive data streams.
problem Efficiently approximate nearest neighbors and kernel density estimation in large datasets.
method Developed sublinear space and query time algorithms for ANN and A-KDE in streaming and sliding-window models.
result Achieved near-optimal trade-offs between memory size and approximation error for ANN.
Sublinear functionals of random variables are known as sublinear expectations; they are convex homogeneous functionals on infinite-dimensional linear spaces. We extend this concept for set-valued functionals defined on measurable set-valued functions (which form a nonlinear space), equivalently, on random closed sets. …
The paper develops methods for time-varying constrained online convex optimization.
problem Time-varying loss and constraint functions in online convex optimization.
method Model-based augmented Lagrangian methods (MALM) for time-varying and delayed feedback.
result Sublinear regret and constraint violation for both time-varying and delayed feedback scenarios.
For α∈(1,2), we present a generalized central limit theorem for α-stable random variables under sublinear expectation. The foundation of our proof is an interior regularity estimate for partial integro-differential equations (PIDEs). A classical generalized central limit theorem is recovered as a special case, p…
New algorithms for constrained online optimization with memory and predictions.
problem Control of constrained dynamical systems and scheduling with reconfiguration budgets.
method Proposed algorithms achieving sublinear regret and constraint violation under time-varying constraints, both with and without predictions.
result First algorithms achieving sublinear regret and constraint violation in constrained online optimization with memory.
Memory-limited learning tackles adversarial bandits with reduced storage.
problem Adversarial bandit problem with limited memory storage.
method Hierarchical learning policy with sublinear memory requirement.
result Established sublinear regret bounds for weak and shifting regrets.
Quantum computing promises faster finance algorithms.
problem Solving finance problems faster than classical methods.
method Quantum computing applications to finance, including Monte Carlo, portfolio optimization, and machine learning.
result Quantum speedups for finance problems, especially Monte Carlo and portfolio optimization.
Sharp Liouville theorem for minimal graphs on manifolds with nonnegative Ricci curvature.
problem Characterizing smooth solutions to minimal hypersurface equations on manifolds with nonnegative Ricci curvature.
method Gradient estimate for minimal graphs over Σ with small linear growth of the negative parts of graphic functions via iteration. result Every smooth solution u to minimal hypersurface equation on Σ is a constant provided u has sublinear growth for its negative part. A fast method for Lasso and Logistic Lasso problems.
problem Solving Lasso and Logistic Lasso regression problems efficiently.
method Iterative active set approach using solver updates.
result 31.41 times faster on average for compressed sensing.
New method connects CAT(0) spaces to hyperbolic spaces.
problem Injecting sublinear Morse boundaries into Gromov boundaries.
method Developed curtain machinery to characterize sublinear Morse properties.
result Continuous injection of sublinear Morse boundaries into Gromov boundaries.
New insights explain speedup saturation in distributed learning with large batches and delays.
problem Understanding and optimizing speedup in distributed learning with large batches and delays.
method Theoretical analysis of strongly convex, convex, and non-convex settings, considering data sparsity.
result Identification of a data-dependent parameter explaining speedup saturation in both batch size and gradient staleness.
Quantum computing offers a quadratic speedup for estimating non-linear functionals.
problem Estimating non-linear functionals of probability distributions.
method Proposes a quantum-inside-quantum Monte Carlo algorithm for a broad class of non-linear estimation problems.
result Achieves a quadratic speedup for non-linear estimation problems, including nested conditional expectations and stochastic optimization.
Study online learning in unknown Markov games with sublinear regret.
problem Online learning in unknown Markov games with unobservable opponents.
method Introduced an algorithm achieving sublinear regret against the minimax value.
result First sublinear regret bound for unknown Markov games, independent of action spaces size.
Predicting structured outputs can be computationally onerous due to the combinatorially large output spaces. In this paper, we focus on reducing the prediction time of a trained black-box structured classifier without losing accuracy. To do so, we train a speedup classifier that learns to mimic a black-box classifier u…
Quantum algorithm speeds up nested expectation estimation by nearly quadratically.
problem Estimating repeatedly nested expectations with quantum computing.
method Proposes a quantum algorithm achieving nearly quadratic speedup over classical methods.
result Achieves nearly quadratic speedup for RNEs, up to logarithmic factors.
Federated Q-Learning achieves linear regret speedup with low communication cost.
problem Achieving linear regret speedup in federated reinforcement learning without high communication costs.
method Proposed two federated Q-Learning algorithms: FedQ-Hoeffding and FedQ-Bernstein, using event-triggered synchronization, novel step size selection, and concentration inequalities.
result Total regrets achieve linear speedup compared to single-agent counterparts with logarithmic communication cost.
Open problem seeks an online learning algorithm for binary classification.
problem Existence of an online learning algorithm for binary classification with sublinear mistakes.
method Assumption of sequence allowing learning algorithm's existence.
result Specific condition determines sequence's learnability.
Efficiently trains large GMMs with millions to billions of parameters.
problem Training large Gaussian Mixture Models (GMMs) is computationally expensive.
method Derives a variational approximation integrated with mixtures of factor analyzers (MFAs) to reduce complexity.
result Sublinear scaling in training GMMs, achieving significant speed-ups.
GPU optimization speeds up large-scale classification tasks.
problem Efficiently training large-scale classification models on GPUs.
method Judecious GPU-optimization principles applied to TRON algorithm.
result Significant speedups for logistic regression and SVM classification.
Average signature of 2-bridge knots approximates sqrt(2c/π).
problem Estimating the average signature and 4-genus of 2-bridge knots.
method Developed a model for 2-bridge knot diagrams indexed by crossing number, and used it to derive upper bounds for the average 4-genus.
result Upper bound for the average 4-genus of a 2-bridge knot is 9.75c/log c.
Online learning is a powerful tool for analyzing iterative algorithms. However, the classic adversarial setup sometimes fails to capture certain regularity in online problems in practice. Motivated by this, we establish a new setup, called Continuous Online Learning (COL), where the gradient of online loss function cha…
New algorithm achieves sublinear regret in CMDPs without error cancellations.
problem Safety constraints in reinforcement learning with error cancellations.
method Model-based primal-dual algorithm for CMDPs with multiple constraints.
result Achieves sublinear regret without error cancellations.
Extends tracking guarantees for time-varying variational inequalities.
problem Tracking solutions of time-varying variational inequalities.
method Extends existing results to sublinear solution paths and periodic problems.
result Discrete dynamical systems of periodic time-varying VI can exhibit chaotic behavior or converge to the solution.
Geodesic loops escape from balls at a sublinear rate imply virtually abelian fundamental group.
problem Understanding fundamental groups of open manifolds with nonnegative Ricci curvature.
method Generalizing the Cheeger-Gromoll splitting theorem to sublinear escape rates.
result Fundamental groups of open manifolds with nonnegative Ricci curvature are virtually abelian if geodesic loops escape sublinearly.
Develops geometric foundations for sublinear Morse boundaries in mapping class groups and Teichmüller spaces.
problem Capturing generic directions in mapping class groups and Teichmüller spaces.
method Develops tools for modeling hulls of median rays in hierarchically hyperbolic spaces via CAT(0) cube complexes.
result Sublinear Morse boundaries are visibility spaces and admit continuous equivariant injections into the boundary of the curve graph.
LaPSRL achieves optimal regret for isoperimetric RL distributions.
problem Designing RL algorithms with sublinear regret for non-log-concave distributions.
method Posterior Sampling (PSRL) and Langevin sampling (LaPSRL) for isoperimetric distributions.
result LaPSRL achieves order-optimal regret and subquadratic complexity.
New approach for distributed online optimization of non-convex losses with sublinear regret.
problem Regret evaluation and consensus in distributed, multi-agent systems with non-convex losses.
method Composite regret metric and consensus-based online normalized gradient (CONGD) approach for pseudo-convex losses; offline optimization oracle for general non-convex losses.
result First sublinear regret bound for general distributed online non-convex learning.
New algorithm achieves linear speedup in non-i.i.d. federated bilevel learning.
problem Linear speedup in convergence for non-i.i.d. datasets in federated bilevel optimization.
method Proposes FedMBO with a novel client sampling scheme for non-i.i.d. datasets.
result Achieves a convergence rate of O(1/√(nK) + 1/K + √n/K³/²).
Under appropriate cooperation protocols and parameter choices, fully decentralized solutions for stochastic optimization have been shown to match the performance of centralized solutions and result in linear speedup (in the number of agents) relative to non-cooperative approaches in the strongly-convex setting. More re…
LIBO optimizes repeated bandit tasks without prior knowledge or regret.
problem Optimizing repeated bandit tasks without prior knowledge or regret.
method LIBO sequentially meta-learns a kernel to adapt to the environment and solve tasks with the latest estimate.
result LIBO achieves sublinear lifelong regret, converging to oracle performance as more tasks are solved.
System learns optimizer hyperparameters to generalize across tasks.
problem Lack of generalization in learning optimizers for neural networks.
method Generalization-first approach, learning optimizer hyperparameters.
result System outperforms Adam on various tasks, including unseen modalities.
GP-UCB resolves sublinear regret for kernelized bandits.
problem Minimizing regret in kernelized bandit problems.
method Using a new regularization technique for kernel ridge estimators, improving GP-UCB's sublinear regret rate.
result GP-UCB achieves nearly optimal sublinear regret for the Matérn kernel.