Paper investigates robustness to interference as a new training signal for meta-learning.
problem Improving incremental learning through robust representations.
method Directly minimizing catastrophic interference as a training signal.
result Representations learned to minimize interference lead to better incremental learning.
FAST improves fast and stable task adaptation in DNNs.
problem Catastrophic forgetting in fine-tuned pretrained models.
method Introducing FAST, an easy-to-implement fine-tuning algorithm.
result FAST learns target tasks faster and retains source knowledge longer.
A fast algorithm for generalized matrix regression improves machine learning performance.
problem Efficiently solving generalized matrix regression problems in machine learning.
method Utilizes sketching technique to achieve (1+ε) relative error with sketching sizes of order $\cO(ε^{-1/2})$. result The Fast GMR algorithm achieves better performance in symmetric positive definite matrix approximation and single pass singular value decomposition.
FAST optimizes additive segmentation for faster, more interpretable models.
problem Efficiently segmenting and interpreting complex datasets.
method Optimization framework for fast piecewise constant shape functions.
result 2 orders of magnitude faster than state-of-the-art methods.
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
Anomaly detection aids in labeling fast-running processes for machine learning.
problem Manual labeling of fast-running processes for machine learning models.
method Anomaly detection to assist in labeling data, specific metrics for model validation.
result Possibility to manually classify data for training machine learning models.
A concise review of recent few-shot meta-learning methods.
problem Mimicking human fast adaptation to new concepts based on prior knowledge.
method Categorized into four branches based on technical characteristics.
result Current challenges and future prospects identified.
This paper shows how to learn variational inequalities fast with strong monotonicity.
problem Learning variational inequalities efficiently.
method Extending convex optimization techniques to variational inequalities with strong monotonicity.
result Fast generalization rates of Θ(1/ε) for learning variational inequalities. FSNet improves online time series forecasting by balancing fast adaptation and old knowledge.
problem Online time series forecasting challenges in handling abrupt and recurring patterns.
method Inspired by CLS theory, FSNet uses a dynamic balance between fast adaptation and old knowledge retrieval.
result FSNet achieves robustness to both new and recurring patterns through dynamic balancing and associative memory.
The speed with which a learning algorithm converges as it is presented with more data is a central problem in machine learning --- a fast rate of convergence means less data is needed for the same level of performance. The pursuit of fast rates in online and statistical learning has led to the discovery of many conditi…
Paper achieves fast learning rates without margin assumptions in agnostic settings.
problem Learning fast rates in the agnostic setting without margin assumptions.
method Proposes a specific learning algorithm for classification with a reject option.
result Achieves a learning rate of $O\left(\frac{d}{n}\log \frac{n}{d}
ight)$ in the agnostic setting.
The paper analyzes reinforcement learning methods for estimating weights and quality functions with fast convergence rates.
problem Estimating weights and quality functions in reinforcement learning with function approximation.
method The paper uses minimax methods for estimating marginal importance weights and q-functions.
result The minimax approach enables fast rates of convergence for weights and quality functions, achieving first-order efficiency.
Empirical risk minimization (ERM) is a fundamental learning rule for statistical learning problems where the data is generated according to some unknown distribution P and returns a hypothesis f chosen from a fixed class F with small loss ℓ. In the parametric setting, depending upon $(\ell…
FAWMF adapts weights for implicit feedback recommendation efficiently.
problem Challenges in treating unobserved data as negative in implicit feedback recommendation.
method FAWMF uses a variational auto-encoder with a parameterized neural network to adaptively assign personalized data confidence weights, and fBGD for efficient learning.
result FAWMF and fBGD outperform existing methods in real-world datasets.
A new supervised tree-Wasserstein distance improves document classification.
problem Measuring document similarity efficiently and accurately.
method Rewriting Wasserstein distance on tree metric, using contrastive loss for optimization.
result The Supervised Tree-Wasserstein (STW) distance improves document classification accuracy.
We provide a way to infer about existence of topological circularity in high-dimensional data sets in Rd from its projection in R2 obtained through a fast manifold learning map as a function of the high-dimensional dataset X and a particular choice of a positive real σ known as band…
Develops a privacy-preserving algorithm for sparse robust regression.
problem Privacy-preserving machine learning for sparse robust regression.
method Develops FRAPPE algorithm for non-smooth loss under differential privacy.
result Achieves better privacy and statistical accuracy trade-off.
Paper proposes a new approach for agents to explore environments efficiently.
problem Learning policies that explore uniformly and mix quickly in environments without external rewards.
method Introduces a surrogate objective to maximize entropy and develops a model-based reinforcement learning algorithm, IDE3AL. result Demonstrates improved exploration and mixing in hard-exploration tasks.
A fast method learns plasma collision kernels from simulations, improving kinetic models.
problem Improving kinetic models for plasma dynamics beyond the weakly coupled regime.
method Data-driven collisional operator, fast spectral separation method.
result Accurately captures plasma dynamics in moderately coupled regime.
MAML adapts faster with deeper architectures, especially in shallow tasks.
problem Understanding and improving MAML's fast adaptation.
method Empirical and theoretical studies on MAML's properties and optimization.
result MAML adapts better with deep architectures even for shallow tasks.
Study shows fast rates for inverse reinforcement learning with linear rewards.
problem Entropy-regularized min-max inverse reinforcement learning in finite-horizon MDPs.
method Structural and statistical analysis of Min-Max-IRL with pseudo-self-concordance.
result Both trajectory-level KL divergence and parameter error decay at O(n−1). Study shows convergence rate for empirical minimizer of unbounded functions with fast growth.
problem Convergence rate of empirical minimizer for unbounded functions with fast growth.
method Analyzes L1-distance convergence rate of the empiric minimizer for coercive functions sampled with noise. result Convergence rate is bounded above by ann−1/q, where q is the dimension and an=o(nε) for every ε>0. Deep neural nets estimate operators between infinite-dimensional spaces with fast rates.
problem Estimating operators between infinite-dimensional spaces.
method Deep neural networks for nonparametric estimation of Lipschitz operators.
result Error bounds decay with fast rates depending on intrinsic dimension.
Two new algorithms speed up TreeSHAP computation for tree-based models.
problem Slow computation of SHAP values on tree-based models.
method Two new algorithms, Fast TreeSHAP v1 and v2, designed to improve computational efficiency.
result Fast TreeSHAP v2 is 2.5x faster than TreeSHAP, with slightly higher memory usage.
Paper develops a fast method to find near-optimal power solutions.
problem Solving AC OPF on fast timescales for large networks.
method Leverages machine learning to map system loading to optimal generation values.
result Near-optimal and feasible solutions found on milliseconds timescales.
Error bound conditions (EBC) are properties that characterize the growth of an objective function when a point is moved away from the optimal set. They have recently received increasing attention in the field of optimization for developing optimization algorithms with fast convergence. However, the studies of EBC in st…
Paper offers a fast convergence theory for offline decision making.
problem Offline decision making problems, including reinforcement learning and off-policy evaluation.
method Introduces a framework (DMOF) and algorithm (EDD) with a fast convergence guarantee.
result Demonstrates a fast convergence guarantee with a lower bound complement.
MACE uses higher-order messages to create fast, accurate force fields.
problem Creating fast and accurate force fields in computational chemistry and materials science.
method Introducing MACE, an equivariant MPNN model that uses four-body messages.
result MACE reduces the required number of message passing iterations to just two, achieving state-of-the-art accuracy.
We present a method for fast resting-state fMRI spatial decomposi-tions of very large datasets, based on the reduction of the temporal dimension before applying dictionary learning on concatenated individual records from groups of subjects. Introducing a measure of correspondence between spatial decompositions of rest …
We unify recent neural approaches to one-shot learning with older ideas of associative memory in a model for metalearning. Our model learns jointly to represent data and to bind class labels to representations in a single shot. It builds representations via slow weights, learned across tasks through SGD, while fast wei…
Weak labels can significantly speed up learning for strong tasks.
problem Learning with limited strong labels.
method Using weak labels to accelerate learning of strong tasks.
result Weak labels can accelerate learning to O(icefrac1n) rate. Paper derives a fast learning rate for deep neural networks without scale invariant activation functions.
problem Analyzing the impact of non-scale invariant activation functions on deep learning performance.
method Using Suzuki (2018) framework, derived a tight generalization error bound for deep neural networks with non-scale invariant activations.
result Without scale invariance of activation functions, deep learning can still achieve a fast learning rate.
Hierarchical pretraining with slow-fast ODEs
problem Causal self-attention vs. slow-fast ODEs
method Instantiating fast-slow ODE formalism as a concrete neural network
result Equilibrium manifold x=φ(y) is exactly the master-equation (ME) stationary distribution New method speeds up training of deep networks robust to adversarial attacks.
problem Deep networks are sensitive to adversarial perturbations, compromising security and interpretability.
method Fast adversarial training using Euclidean norm approximation and distributed computing.
result Robust feature representations and reduced training time achieved.
Novel framework proves fast RL convergence in continuous spaces.
problem Analyzing stability in continuous state-action RL.
method Introduces a novel framework to analyze stability properties of RL.
result Highlights two key stability properties and demonstrates their satisfaction in RL.
DIP-FAT improves adversarial training by diversifying perturbations.
problem Adversarial examples fool deep neural networks, leading to overfitting and poor performance.
method DIP-FAT uses random directions to diversify perturbations in adversarial training.
result DIP-FAT reduces overfitting and improves clean data accuracy.
When applied to training deep neural networks, stochastic gradient descent (SGD) often incurs steady progression phases, interrupted by catastrophic episodes in which loss and gradient norm explode. A possible mitigation of such events is to slow down the learning process. This paper presents a novel approach to contro…
Abstract MDPs enable strategic exploration and fast reward transfer in complex environments.
problem Challenging to learn accurate MDPs for high-dimensional states.
method Learn an abstract MDP over low-dimensional coarse states, using an abstraction function.
result Achieves superhuman performance on Pitfall! and higher reward with fewer samples.
Paper introduces a new gradient statistic to improve deep learning convergence.
problem Fluctuation effect of gradient updates between iterations.
method Introduces an unbiased stratified statistic \(\bar{G}_{mst}\) and a new algorithm MSSG.
result MSSG algorithm outperforms other sgd-like algorithms in training deep models.
This work develops scalable model selection methods with fast update and selection.
problem Efficient model selection for large pools of candidate models.
method Isolated model embedding, which supports asymptotically fast update and selection.
result Standardized Embedder achieves competitive model selection performances.
This work learns effective dynamics from short-term data of stochastic systems.
problem Learning effective dynamics from short-term data of stochastic systems.
method Proposes a novel algorithm using a neural network (Auto-SDE) to learn invariant slow manifold from data.
result Validated through numerical experiments to be accurate, stable, and effective.
New learning dynamics achieve fast convergence in games without needing to know utility scales.
problem Fast convergence guarantees in learning games require prior knowledge of utility scales.
method Developed scale-free and scale-invariant learning dynamics using optimistic follow-the-regularized-leader with adaptive learning rates and clipping techniques.
result Achieved fast convergence rates to Nash and correlated equilibria without prior utility scale knowledge.
Fast linear transforms are ubiquitous in machine learning, including the discrete Fourier transform, discrete cosine transform, and other structured transformations such as convolutions. All of these transforms can be represented by dense matrix-vector multiplication, yet each has a specialized and highly efficient (su…
The paper introduces a fast algorithm for learning and forecasting nonlinear dynamics from noisy time series data.
problem Challenges in capturing nonlinear dynamics from noisy time series data.
method A projected nonlinear state-space model with kernel functions applied to projected lines.
result The model effectively learns and forecasts complex nonlinear dynamics with computational efficiency.
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
New classifier combines locally linear kernels for fast and accurate non-linear classification.
problem Developing a fast and accurate non-linear classifier.
method Combines locally linear classifiers using a ℓ1 Multiple Kernel Learning (MKL) problem with scalable MKL training for streaming kernels. result The resulting classifier achieves high accuracy with fast inference time.
Neural network learns fast PDE solvers with proven guarantees.
problem Designing fast iterative solvers for specific PDE problems.
method Learn to modify an existing solver using a deep neural network.
result Achieves 2-3 times speedup compared to state-of-the-art solvers.
Dictionary learning is the task of determining a data-dependent transform that yields a sparse representation of some observed data. The dictionary learning problem is non-convex, and usually solved via computationally complex iterative algorithms. Furthermore, the resulting transforms obtained generally lack structure…