New definition of regret for nonconvex online learning models.
problem Intractability of standard regret measures for nonconvex models.
method Introduced a local gradient based regret definition.
result Our definition provides more interpretable bounds for forecasting.
Adversarial online nonparametric regression achieves optimal rates with locally adaptive learning.
problem Adversarial online nonparametric regression with general convex losses.
method Parameter-free learning algorithm leveraging chaining trees to compete against H{ö}lder functions, dynamically tracking and adapting to local smoothness variations.
result First computationally efficient algorithm with locally adaptive optimal rates for online regression in an adversarial setting.
A new online local metric learning framework reduces overfitting and scales with data dimensions.
problem Global metrics fail to capture the complexity of real-world datasets.
method Proposes an online multiple metric learning framework with a global and local component.
result Reduces overfitting and scales with data dimensions.
Locally private algorithm improves online federated learning with correlated noise.
problem Privacy-preserving online federated learning with non-IID data.
method Locally differentially private algorithm using temporally correlated noise.
result Established dynamic regret bound for nonconvex loss functions.
Paper extends meta-learning framework to non-convex settings with improved performance.
problem Learning from past tasks for faster future tasks in a sequential setting.
method Generalized online meta-learning framework to non-convex settings, introduced local regret as performance measure.
result The framework achieves logarithmic local regret and robustness to hyperparameter initialization.
Proves local convergence of various online and recurrent optimization algorithms.
problem Proves local convergence of online and recurrent optimization algorithms not covered by standard stochastic gradient descent theory.
method Uses a general set of assumptions for learning dynamical systems online, adopting an 'ergodic' viewpoint.
result Local convergence results for online and recurrent optimization algorithms, including RMSProp, NoBackTrack, UORO, Adam, and RTRL.
The study sets criteria for efficient communication in distributed online learning.
problem Achieving optimal learning performance while minimizing communication in distributed online learning.
method Formal criteria based on the intuition that in the worst case, every input is essential for learning performance and must be exchanged.
result The criteria hold for a simplified version of a previously published protocol, providing a communication bound that scales with the serialized prediction problem's hardness.
Proposes time-smoothed gradients for more stable online forecasting.
problem Stability and efficiency in online forecasting with SGD.
method Introduces time-smoothed gradients within SGD update rules.
result Time-smoothed gradients yield more stable results than existing methods.
Locally private online quantile regression method addresses privacy constraints.
problem Estimating and inferring quantile regression under local differential privacy constraints.
method Developed a finite-alphabet channel where users compute local contributions, apply randomized response, and send reports. A public decoder corrects distortion and reconstructs inputs for averaging.
result Established local privacy, decoder unbiasedness, consistency, asymptotic normality, and inference for scalar contrasts.
Central to robot exploration and mapping is the task of persistent localization in environmental fields characterized by spatially correlated measurements. This paper presents a Gaussian process localization (GP-Localize) algorithm that, in contrast to existing works, can exploit the spatially correlated field measurem…
Kolmogorov-Arnold Networks enable ultrafast online learning with fixed-point quantization.
problem Efficient online learning for high-frequency systems with strict memory constraints.
method Fixed-point online training on FPGAs exploiting B-spline locality in KANs.
result Kolmogorov-Arnold Networks are more efficient and expressive than MLPs for low-latency tasks.
New algorithms adaptively compete against complex environments with local regularities.
problem Efficiently competing against complex, locally regular comparator functions in nonparametric settings.
method Locally-adaptive online algorithms using hierarchical ε-nets and tree experts. result Proved regret bounds scaling with different types of local regularities, delivering better performance for simple profiles.
AIHT improves online high-dimensional quantile regression by separating support discovery and refinement.
problem Online high-dimensional quantile regression with structural sparsity.
method Adaptive Iterative Hard Thresholding (AIHT) alternates stochastic updates with adaptive hard-thresholding steps.
result AIHT achieves logarithmic regret for the sliding-window objective in high-dimensional settings.
Paper tackles dynamic open world recognition in online settings.
problem Dynamic open world recognition in online settings.
method Incremental learning of the underlying metric, incremental estimate of confidence thresholds, local learning.
result Proposed methods outperform non-online counterparts in various scenarios.
Improves multi-objective learning by adapting to local subintervals.
problem Learning a predictor satisfying multiple objectives in an online, changing data setting.
method Adapting an existing multi-objective learning method with an adaptive online algorithm.
result Improves predictions over subgroups and remains robust under distribution shift.
OLBoost improves online decision tree performance without increasing memory or time costs.
problem Improving predictive performance in online decision trees without high memory or time costs.
method OLBoost applies boosting to small regions of the instances space within online decision tree algorithms.
result OLBoost can significantly improve online learning decision tree performance without increasing tree size.
New RL approach uses resets to learn complex MDPs efficiently.
problem Learning complex MDPs with high-dimensional states and function approximation.
method Local simulator access and recursive value function search.
result Proven sample-efficient learning for MDPs with low coverability.
Online PaLD extends PaLD for semi-supervised online applications.
problem Scalability of unsupervised clustering algorithms for large datasets.
method Adapted partitioned local depth algorithm for online semi-supervised prediction.
result Online PaLD extends cohesion network to new data points efficiently.
CDLEEDS detects local changes in evolving data streams for accurate feature attributions.
problem Local feature attributions become obsolete in evolving data streams.
method CDLEEDS, a flexible framework for detecting local change and concept drift.
result CDLEEDS reliably detects both local and global concept drift.
This work analyzes DP-SGD for online LDP problems with practical convergence rates.
problem Analyzing DP-SGD for online LDP problems with practical convergence rates.
method Developed a general framework for online LDP model in stochastic optimization problems, conducted non-asymptotic convergence analysis.
result Comprehensive non-asymptotic convergence analysis of the proposed estimators in finite-sample situations.
Biological neural network mimics CCA for multi-channel data.
problem Implementing CCA in a biologically plausible neural network.
method Derive an online CCA algorithm with local synaptic updates for multi-compartmental neurons.
result The derived neural network architecture and synaptic updates resemble cortical pyramidal neuron behavior.
Develops an online learning method for large/streaming data.
problem Prediction in large/streaming data sets.
method Covariance-fitting methodology for online learning.
result Predictor with desirable properties: linear runtime, constant memory, no local minima, prunes redundant dimensions.
Neural network tackles continual learning with neuromodulation and local error signals.
problem Catastrophic forgetting in continuous learning.
method Biologically-inspired neural architecture with local learning and neuromodulation, combined with transfer metalearning.
result Superior performance in continual learning tasks compared to other approaches.
SpectralLeader learns latent variables from streaming data efficiently and globally.
problem Learning latent variable models from a stream of data efficiently and globally.
method SpectralLeader, an online learning algorithm for latent variable models that converges to the global optimum.
result SpectralLeader achieves a sublinear upper bound on n-step regret in the bag-of-words model. This paper improves federated learning efficiency by adaptively sparsifying gradients.
problem Efficiently training machine learning models with geographically dispersed data.
method Adaptive gradient sparsification for non-i.i.d. local datasets, fairness-aware, online learning approach.
result Up to 40% improvement in model accuracy for a finite training time.
Agents learn locally, converge globally in online learning with kernels.
problem Multi-agent learning with limited data and communication.
method Local regression functions with consensus constraints, functional stochastic gradient descent, and greedy subspace projections.
result Agents' functions converge to a neighborhood of the globally optimal one as the penalty parameter increases.
NeuRewriter learns to choose and rewrite heuristics in combinatorial problems.
problem Time-consuming tuning of heuristics in combinatorial optimization.
method NeuRewriter uses reinforcement learning to learn a policy for picking heuristics and rewriting solutions.
result NeuRewriter outperforms existing methods in various combinatorial tasks.
OSGM uses online learning to adapt stepsize for faster convergence.
problem Improving convergence rates of first-order methods.
method OSGM combines online learning and feedback functions to adjust stepsize.
result OSGM achieves convergence rates asymptotically no worse than optimal.
Neuromorphic column performs online unsupervised clustering.
problem Real-time clustering of streaming data.
method Localized, spike timing-dependent plasticity (STDP) neural column.
result Prototype column performs similarly to k-means clustering.
We introduce a new local regret framework for non-convex models in dynamic environments.
problem Challenges in online forecasting for non-convex models with frequent updates and concept drift.
method We propose a novel local regret framework and a time-smoothed gradient update rule.
result Our approach yields more stable, robust, and computationally efficient forecasting compared to state-of-the-art methods.
Local volatility model for commodity futures options with online calibration.
problem Valuation and calibration of options on commodity futures.
method Local volatility model, online calibration, model-based price adjustment, Tikhonov regularization.
result Improved option valuation and smile adherence through online calibration and regularization.
A neural network learns to control a two-link arm with non-linear dynamics.
problem Training spiking neural networks to control complex, non-linear systems.
method Feedback-based Online Local Learning Of Weights (FOLLOW) to train a network of spiking neurons with hidden layers.
result The network learns an inverse model of the arm's dynamics and uses it to generate a motor command for control.
This paper explores estimating chaotic dynamics and parameters using local ensemble Kalman filters.
problem Estimating chaotic dynamics and parameters from observations.
method Local ensemble Kalman filters with covariance and local domain localisation.
result Rigorously updating global parameters using a local domain ensemble Kalman filter.
Federated learning improves with adaptive hyper-parameters and representation matching.
problem Heterogeneous client data leads to divergent local models in federated learning.
method Representation matching and adaptive hyper-parameters.
result Significant performance and robustness improvements in federated learning.
Extracting actionable intelligence from distributed, heterogeneous, correlated and high-dimensional data sources requires run-time processing and learning both locally and globally. In the last decade, a large number of meta-learning techniques have been proposed in which local learners make online predictions based on…
POLO framework enables efficient learning and exploration in model-based control.
problem Efficient learning and exploration in model-based control settings.
method Combines local model-based control, global value function learning, and exploration.
result POLO framework accelerates value function learning and enables better policies.
This work tackles online adaptation for reinforcement learning in dynamic real-world environments.
problem Expensive sample generation and failure of specialized policies in the real world.
method Meta-learning to train a dynamics model prior that can be rapidly adapted to new contexts.
result Demonstrated online adaptation for continuous control tasks in both simulated and real-world agents.
An online algorithm improves cPCA for efficient, interpretable data analysis.
problem Efficiently finding informative low-dimensional representations in large datasets.
method Developed an online algorithm for a modified cPCA* method.
result The online algorithm for cPCA* shows improved performance and interpretability.
This paper introduces a new learning rule for probabilistic SNNs that improves log-likelihood, accuracy, and calibration.
problem Training and inference of deterministic SNNs are constrained by their inability to generate multiple independent outputs.
method Introduces a generalized expectation-maximization (GEM) learning rule for probabilistic SNNs.
result The GEM-SNN learning rule leads to significant improvements in log-likelihood, accuracy, and calibration.
New online adaptive SVR model for IVS with hardware acceleration.
problem Modeling implied volatility surface (IVS) in real-time.
method Online adaptive primal support vector regression (SVR) with hardware acceleration.
result Gaussian kernel outperforms linear kernel in support vector size regulation.
GALA adapts learning rates online by aligning gradients, improving deep learning model performance.
problem Fine-tuning learning rates for deep learning models requires extensive grid search.
method GALA dynamically adjusts learning rates by tracking gradient alignment and local curvature.
result GALA produces a flexible, adaptive learning rate schedule that increases when gradients align.
Paper proposes federated learning for SNNs to enable low-power, online training.
problem Limited data at each device for on-device SNN training.
method Federated Learning (FL) for cooperative SNN training, leveraging local and global feedback.
result FL-SNN achieves significant advantages over separate training and offers a flexible trade-off between accuracy and communication load.
This work improves Gaussian process regression for large, non-stationary data.
problem Scalability issues and performance degradation for non-stationary data.
method Combines variational free energy approximations with online expectation propagation and local splitting steps.
result Incremental adaptation to locality, heterogeneity, and non-stationarity in training data.
FLeet improves online FL for mobile apps with better performance and privacy.
problem Federated Learning's offline nature limits its applicability for online updates.
method Combines staleness awareness and performance prediction with adaptive learning.
result 2.3x quality boost with minimal battery consumption.
Paper introduces UL layers for unsupervised video analysis.
problem Label-free video analysis.
method Two unsupervised learning layers for fully connected and convolutional neural networks.
result Neural networks with UL layers can extract shape and motion information from unlabeled videos.
Despite our extensive knowledge of biophysical properties of neurons, there is no commonly accepted algorithmic theory of neuronal function. Here we explore the hypothesis that single-layer neuronal networks perform online symmetric nonnegative matrix factorization (SNMF) of the similarity matrix of the streamed data. …
Gated Linear Networks bypass feature learning for fast online learning.
problem Fast online learning and feature learning trade-offs in neural networks.
method Distributed and local credit assignment mechanism, data-dependent gating, online convex optimization.
result GLNs achieve universal learning capabilities and resilience to catastrophic forgetting.
This work examines a semi-blind single-channel source separation problem. Our specific aim is to separate one source whose local structure is approximately known, from another a priori unspecified background source, given only a single linear combination of the two sources. We propose a separation technique based on lo…