Adaptive neural network adjusts layer depth based on computational needs.
problem Deciding the number of layers in deep recurrent neural networks is challenging.
method Proposes a layer flexible recurrent neural network with adaptive computation time.
result Improves performance by 7\% to 12\% on financial and language data.
ATA optimizes task allocation in distributed machine learning.
problem Greedy task allocation leads to inefficiencies in distributed machine learning.
method Adaptive Task Allocation (ATA) adapts to unknown computation time distributions.
result ATA identifies optimal task allocation without prior knowledge of computation times.
Adaptive compute allocation improves model performance by prioritizing harder queries.
problem Inefficiency in allocating test-time compute uniformly across all queries.
method Formulated as a bandit learning problem, proposed adaptive algorithms that estimate query difficulty and allocate compute accordingly.
result Achieved up to 15.29% relative performance improvement on various benchmarks.
Best-of-∞ improves LLM performance by efficiently allocating inference-time computation.
problem Achieving optimal performance in test-time LLM ensembling with infinite budget.
method Adaptive generation scheme and weighted ensembles of LLMs, formulated as mixed-integer linear program.
result Optimal ensemble weighting improves performance over individual models.
FPGA-based multi-layer equalizer adapts to changing channels.
problem Real-time adaptation to time-varying channel impairments.
method Multi-layer machine learning on FPGA with on-chip gradient backpropagation training.
result Real-time adaptation to changing channel conditions achieved.
Multi-hop inference is necessary for machine learning systems to successfully solve tasks such as Recognising Textual Entailment and Machine Reading. In this work, we demonstrate the effectiveness of adaptive computation for learning the number of inference steps required for examples of different complexity and that l…
New algorithm detects and adapts to changes in real-time data streams.
problem Adapting to fast-changing data in real-time systems.
method Concept drift detection followed by prototype-based adaptation.
result Stable and quick adjustments during model adaptation.
Improves an existing algorithm for computing algebraic set homology groups.
problem Efficiently computing the homology groups of algebraic or semi-algebraic sets.
method Adaptive grid algorithm on the unit sphere.
result Practical improvement of an existing algorithm for homology computation.
In this paper, we present an online adaptive PCA algorithm that is able to compute the full dimensional eigenspace per new time-step of sequential data. The algorithm is based on a one-step update rule that considers all second order correlations between previous samples and the new time-step. Our algorithm has O(n) co…
Real-time scene understanding solved using Approximate Bayesian Computation.
problem Predicting human actions, object poses, and pedestrian crossings from depth images.
method Bayesian error model, neural surrogates, and adaptive discretization.
result Real-time inference on real-world problems is feasible.
We present an approach to adaptively utilize deep neural networks in order to reduce the evaluation time on new examples without loss of accuracy. Rather than attempting to redesign or approximate existing networks, we propose two schemes that adaptively utilize networks. We first pose an adaptive network evaluation sc…
A new algorithm estimates mean adaptively to covariance, faster and more flexible than existing methods.
problem Estimating mean of a distribution with unknown covariance efficiently and privately.
method Adaptive differentially private algorithm with optimal convergence rates and near-linear sample complexity.
result Achieves optimal rates of convergence with respect to the Mahalanobis norm ∣∣⋅∣∣Σ. Adaptive TBPTT controls gradient bias in RNNs for faster convergence.
problem Choosing optimal truncation length in TBPTT for RNNs is difficult.
method Adaptive TBPTT converts lag selection to bias control, estimating optimal truncation length during training.
result Adaptive TBPTT improves convergence rate and computational efficiency in RNNs.
New solver avoids memory issues for long differential equations.
problem Memory constraints in adaptive probabilistic ODE solvers.
method Fixed memory demands adaptive probabilistic solver using robust state estimation.
result Eliminates memory issues for long time series simulations.
ARU adapts deep forecasting models in streaming data with efficient updates.
problem Adapting deep globally trained models for streaming data efficiently.
method ARU combines deep global models with closed-form linear models for per-series adaptation.
result ARU outperforms local adaptation methods on various datasets.
SANAS adapts neural network architecture in real-time for efficient keyword spotting.
problem Efficiently identifying keywords in real-time audio streams with minimal computational cost.
method Stochastic Adaptive Neural Architecture Search (SANAS) that dynamically adjusts network size based on task difficulty.
result SANAS achieves high keyword recognition accuracy with significantly reduced computational cost.
Improves distributed SGD convergence speed with reduced computation load.
problem Mitigating stragglers in distributed SGD to speed up convergence.
method Modeling communication and computation times, adapting number of workers and computation load dynamically.
result Significantly reduces computation load while improving convergence speed.
Study cryptocurrency price dynamics using adaptive EMD and spectral analysis.
problem Analyze the time-varying volatility of cryptocurrency prices.
method Adaptive complementary ensemble empirical mode decomposition (ACE-EMD) and Hilbert spectral analysis.
result Reveal the properties of various timescales in cryptocurrency price dynamics.
Can we effectively learn a nonlinear representation in time comparable to linear learning? We describe a new algorithm that explicitly and adaptively expands higher-order interaction features over base linear representations. The algorithm is designed for extreme computational efficiency, and an extensive experimental …
Adaptive transfer learning improves time series classification.
problem Improving generalization in time series classification.
method Proposed a general adaptive transfer learning framework for multi-view time series data.
result Framework maintains desirable generalization ability.
Manifold Markov chain Monte Carlo algorithms have been introduced to sample more effectively from challenging target densities exhibiting multiple modes or strong correlations. Such algorithms exploit the local geometry of the parameter space, thus enabling chains to achieve a faster convergence rate when measured in n…
We present a general framework for classification of sparse and irregularly-sampled time series. The properties of such time series can result in substantial uncertainty about the values of the underlying temporal processes, while making the data difficult to deal with using standard classification methods that assume …
New method adapts without backprop, faster and better.
problem Efficient domain adaptation without source data.
method Computing class prototypes from pre-trained model.
result Significant accuracy improvements over pre-trained model.
New algorithm adapts to unknown smoothness in stochastic bandits with polynomial cost.
problem Adapting to unknown smoothness in stochastic bandits.
method Reconsidered Locatelli and Carpentier's lower bound, defined admissible rate functions, and developed a new algorithm.
result New algorithm matches minimal rate functions and provides polynomial cost of adaptation.
A neural RNN model adapts time steps for non-stationary time series data.
problem Modeling and forecasting non-stationary time series with sharp changes.
method RNN-ODE-Adap model using neural ODE and adaptive time steps.
result Consistent estimation of intensity function for Hawkes-type data.
Adaptive estimation for nonstationary time series reduces computational cost.
problem Estimating parameters of nonstationary time series with varying parameters over time.
method Moving exponential moving ML estimator for scale parameter estimation.
result Significantly improved log-likelihoods compared to standard estimation.
We study the problem of structured prediction under test-time budget constraints. We propose a novel approach applicable to a wide range of structured prediction problems in computer vision and natural language processing. Our approach seeks to adaptively generate computationally costly features during test-time in ord…
A new adaptive binarization technique using fuzzy integrals improves image quality.
problem Improving image thresholding quality.
method FLAT (Fuzzy Local Adaptive Thresholding) based on fuzzy integrals.
result The proposed FLAT method produces better image quality than traditional algorithms and neural networks.
iDAD uses neural networks to quickly adapt experiments without likelihoods.
problem Performing adaptive experiments in real-time with implicit models.
method iDAD learns a design policy network upfront to make quick design decisions.
result iDAD can make design decisions in milliseconds, unlike traditional BOED approaches.
Paper presents adaptive minimax risk classifiers for multidimensional concept drift.
problem Multidimensional concept drift in supervised classification.
method Adaptive minimax risk classifiers (AMRCs) tracking multivariate and high-order distribution changes.
result AMRCs provide computable tight performance guarantees and improve classification.
SpecGrad improves neural vocoder sound quality by adapting diffusion noise to log-mel spectrogram.
problem Improving neural vocoder sound quality, especially in high-frequency bands.
method Adapting the diffusion noise distribution to the conditioning log-mel spectrogram through time-varying filtering.
result SpecGrad generates higher-fidelity speech waveform than conventional DDPM-based neural vocoders.
New method reduces Gibbs partition function estimation complexity.
problem Estimating partition functions of Gibbs distributions.
method Doubly-adaptive MCMC with adaptive cooling schedule and mean estimator.
result Outperforms state-of-the-art algorithms in computational complexity and robustness.
ALT transforms time series data for better classification.
problem Efficiently classifying time series data with varying temporal scales.
method ALT algorithm using variable-length shifted time windows.
result State-of-the-art performance with minimal computational overhead.
We consider the problem of discrete-time signal denoising, focusing on a specific family of non-linear convolution-type estimators. Each such estimator is associated with a time-invariant filter which is obtained adaptively, by solving a certain convex optimization problem. Adaptive convolution-type estimators were dem…
Framework disentangles deep feature uncertainty for efficient inference.
problem Inference-time uncertainty estimation for reliable decision-making.
method Uncertainty-Guided Inference-Time Selection framework.
result Significantly tighter prediction intervals and 60% compute reduction.
Boosting ridge regression for high-dimensional data classification reduces computational cost and improves learning time.
problem High computational demand of inverting regularised covariance matrix in ridge regression for high-dimensional problems.
method Train an ensemble of ridge regressors in randomly projected subspaces, then combine them using adaptive boosting.
result Effective in terms of learning time and improved predictive performance in some cases.
Adaptive algorithm speeds up k-nearest-neighbor searches.
problem Finding k nearest neighbors with varying efficiency.
method Adaptive estimation of distances to optimize search.
result The algorithm achieves significant speedups compared to naive methods.
Improved elimination strategies for adaptive bandit identification reduce sample complexity and computational burden.
problem Inefficient elimination strategies in bandit identification.
method Adaptive elimination methods that update sampling rules frequently and reduce problem size.
result Adaptive elimination methods achieve better sample complexity and computational efficiency.
Estimates isotonic functions under unknown permutations, achieving optimal statistical and computational efficiency.
problem Estimating isotonic functions with unknown permutations in multiway comparison data.
method Mirsky partition estimator for minimax optimal and adaptive estimation.
result Achieves optimal worst-case statistical performance and computational efficiency.
This research adapts superpixels for Shapley value computation in DNA profile classification.
problem Efficiently computing Shapley values for large, multidimensional time-series data.
method Adapting the concept of superpixels to streamline Shapley value computation for time-series-like data.
result Realistic, accurate, and fast computation of Shapley values for DNA profile classification.
Path signatures adapted for Lie groups improve action recognition in computer vision.
problem Improving action recognition in computer vision with geometric constraints.
method Lifting path signatures to Lie groups and proving universality and characteristic property.
result Path signatures on Lie groups provide comparable performance to shallow learning approaches in action recognition.
Benchmark for UDA in time series classification.
problem Lack of benchmarks for unsupervised domain adaptation in time series.
method Introduces a comprehensive benchmark with new datasets and state-of-the-art neural network backbones.
result Insights into strengths and limitations of UDA methods for time series data.
We analyze the computational limits of LoRA for transformer models using fine-grained complexity theory.
problem Computational efficiency of LoRA fine-tuning for transformer models.
method Fine-grained complexity theory, identifying phase transitions, almost linear algorithms.
result Existence of almost linear algorithms for LoRA adaptation based on specific norms.
New method forecasts time series with changing variances.
problem Real-world processes with changing variances cannot be captured by classical models.
method State-space model with Markov switching variances, using online learning and expert aggregation.
result Proposed method outperforms traditional expert aggregation and is robust to misspecification.
PruneFL reduces FL training time on edge devices by pruning model size.
problem Limited computation and communication resources on edge devices in FL.
method Adaptive and distributed parameter pruning during FL process.
result Pruned model converges to similar accuracy as original model with reduced training time.
Unified online tensor learning algorithm reduces computational and memory costs.
problem Efficiently learning from large tensors with minimal data storage and timely predictions.
method oRGrad algorithm for online tensor learning.
result oRGrad achieves optimal O(T1/2) regret and O(logT) adaptive regret. ACA method improves gradient estimation for neural ODEs, reducing error and training time.
problem Inaccurate gradient estimation methods hinder the performance of neural ODEs on benchmark tasks.
method Adaptive Checkpoint Adjoint (ACA) method that applies trajectory checkpointing, deletes redundant components, and supports adaptive solvers.
result ACA reduces error rate by half and training time by half compared to adjoint and naive methods on image classification tasks.
Paper proposes an adaptive gradient method for federated learning.
problem Communication efficiency in federated learning.
method Adaptive gradient method for federated learning.
result Guaranteed convergence and communication efficiency in federated learning.