Sequence-to-sequence (seq2seq) based ASR systems have shown state-of-the-art performances while having clear advantages in terms of simplicity. However, comparisons are mostly done on speaker independent (SI) ASR systems, though speaker adapted conventional systems are commonly used in practice for improving robustness…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “adaptive model”
Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.
Automation of machine learning model development is increasingly becoming an established research area. While automated model selection and automated data pre-processing have been studied in depth, there is, however, a gap concerning automated model adaptation strategies when multiple strategies are available. Manually…
Paper presents a Transformer model for automatic domain adaptation.
problem Challenges in selecting or designing domain adaptation algorithms.
method Transformer model approximates and selects domain adaptation algorithms.
result Transformers can approximate and automatically select domain adaptation algorithms.
IA-BMA adapts model weights to inputs for better predictions.
problem Predicting with multiple models in heterogeneous settings.
method Input adaptive Bayesian Model Averaging (IA-BMA) with an input adaptive prior and amortized variational inference.
result IA-BMA consistently delivers more accurate and better-calibrated predictions.
We study the performance of the adaptive construction scheme for a Bayesian inference on the Quadratic GARCH model which introduces the asymmetry in time series dynamics. In the adaptive construction scheme a proposal density in the Metropolis-Hastings algorithm is constructed adaptively by changing the parameters of t…
LoRA enhances model adaptability without increasing parameters.
problem Theoretical understanding of LoRA's adaptability.
method Theoretical analysis of LoRA's expressive power.
result LoRA can adapt any model to a smaller target model with a specific rank.
Supporting evidence for adaptive feature program across diverse models.
problem Analyzing feature learning in neural networks.
method Over-parameterized sequence models and feature error measure (FEM).
result FEM is decreasing during training of adaptive feature models.
New analysis shows surprising results on adaptation speed of causal models.
problem Investigate the adaptation speed of causal models under interventions.
method Use convergence rates from stochastic optimization to measure adaptation speed.
result Surprising findings: anticausal model can be faster than causal model under certain conditions.
AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecastingstat.ML
AdaPTS adapts univariate FMs for multivariate time series forecasting.
problem Challenges in managing feature dependencies and uncertainty quantification in multivariate time series forecasting.
method Adapters that transform multivariate inputs into a latent space and apply univariate FMs independently to each dimension.
result AdaPTS enhances forecasting accuracy and uncertainty quantification compared to baseline methods.
New adaptive models improve prediction accuracy with missing data.
problem Improving prediction accuracy with missing data entries.
method Adaptive optimization approach, learning imputation and regression simultaneously.
result 2-10% improvement in out-of-sample accuracy in strongly non-random missing data settings.
AdapTable adapts tabular models to shifts without source data, improving HELOC performance.
problem Distribution shifts in tabular data threaten model performance.
method Shift-aware uncertainty calibrator and label distribution handler.
result Up to 16% improvement on HELOC dataset.
Modular neural causal models outperform other models in generalization and adaptation.
problem Robust out-of-distribution generalization and fast adaptation in machine learning.
method Factorizing data generating process into modules using only causal parents as predictors.
result Modular neural causal models offer robust generalization and fast adaptation, especially in low data regimes.
This paper improves neural network generalization by dynamically learning kernel parameters.
problem Improving neural network generalization and adaptability.
method Diagonal adaptive kernel model that learns kernel eigenvalues and output coefficients during training.
result The diagonal adaptive kernel model significantly improves generalization over fixed-kernel methods.
Most neural-network based speaker-adaptive acoustic models for speech synthesis can be categorized into either layer-based or input-code approaches. Although both approaches have their own pros and cons, most existing works on speaker adaptation focus on improving one or the other. In this paper, after we first systema…
Adaptive networks improve model robustness through conditional normalization.
problem Limited robustness of adversarial-trained networks due to network capacity and training samples.
method Proposes a conditional normalization module to adapt networks during adversarial training.
result Adaptive networks outperform both clean validation accuracy and robustness compared to non-adaptive counterparts.
Paper tackles adaptive deletion of data points from trained models.
problem Adaptive deletion of data points from trained models.
method Reduction from adaptive to non-adaptive deletion guarantees using differential privacy and max information.
result Strong provable deletion guarantees for adaptive deletion sequences.
Generative Adapter adapts LMs with a single forward pass, reducing inference overhead.
problem Efficient adaptation of large language models for new contexts.
method Generative Adapter directly maps new contexts to low-rank LM adapters via self-supervised learning.
result Significant reduction in inference overhead with no need for fine-tuning.
A Bayesian estimation of a GARCH model is performed for US Dollar/Japanese Yen exchange rate by the Metropolis-Hastings algorithm with a proposal density given by the adaptive construction scheme. In the adaptive construction scheme the proposal density is assumed to take a form of a multivariate Student's t-distributi…
Tent adapts models during testing by minimizing entropy of predictions.
problem Adapting models to new data during testing with limited information.
method Test entropy minimization (tent) and online channel-wise affine transformations.
result Reduces generalization error on various datasets and benchmarks.
We present ARU, an Adaptive Recurrent Unit for streaming adaptation of deep globally trained time-series forecasting models. The ARU combines the advantages of learning complex data transformations across multiple time series from deep global models, with per-series localization offered by closed-form linear models. Un…
Meta-SAGE improves deep RL scalability for CO tasks by adapting pre-trained models to larger-scale problems.
problem Improving scalability of deep reinforcement learning models for combinatorial optimization tasks.
method Meta-SAGE combines a scale meta-learner and scheduled adaptation with guided exploration to adjust model parameters for larger-scale problems.
result Meta-SAGE outperforms previous methods and significantly improves scalability in CO tasks.
Bayesian nonparametric LABS model adapts to function smoothness in Besov spaces.
problem Estimating functions with unknown smoothness in Besov spaces.
method Lévy Adaptive B-spline (LABS) regression model with automatic smoothness adaptation.
result LABS posterior contracts around true function in Besov classes at nearly minimax-optimal rates.
We present an adaptive approach to the construction of Gaussian process surrogates for Bayesian inference with expensive-to-evaluate forward models. Our method relies on the fully Bayesian approach to training Gaussian process models and utilizes the expected improvement idea from Bayesian global optimization. We adapt…
K-priors enable quick adaptation with minimal retraining.
problem Machine learning models struggle to adapt to changes efficiently.
method Combines weight and function-space priors to reconstruct past gradients.
result Adaptation with K-priors achieves similar performance to full retraining with less data.
AdAdaGrad optimizes batch sizes for deep learning models, reducing the generalization gap.
problem The generalization gap between large-batch and small-batch training in deep learning.
method AdAdaGrad introduces adaptive batch size strategies derived from adaptive sampling methods.
result AdAdaGradNorm converges to a first-order stationary point with a rate of O(1/K) in K iterations.
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
PLoP optimizes LoRA placement for efficient large model finetuning.
problem Improving efficiency of LoRA adaptation for large models.
method Intuitive theoretical analysis for automatic adapter placement.
result PLoP consistently outperforms existing placement strategies.
New approach shows AI can adapt like toddlers by correcting old knowledge.
problem Lack of adaptability in AI models compared to humans and animals.
method Casting adaptation as posterior correction and using Bayesian Learning Rule.
result AI can learn to adapt quickly by using posterior correction.
Distributed Nonparametric Function Estimation: Optimal Rate of Convergence and Cost of Adaptationmath.ST
Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.
problem Optimal rate of convergence and cost of adaptation in distributed nonparametric function estimation.
method Distributed minimax estimation and adaptive estimation under communication constraints for Gaussian sequence model and white noise model.
result Established minimax rate of convergence and exact communication cost for adaptation.
We propose a novel adversarial speaker adaptation (ASA) scheme, in which adversarial learning is applied to regularize the distribution of deep hidden features in a speaker-dependent (SD) deep neural network (DNN) acoustic model to be close to that of a fixed speaker-independent (SI) DNN acoustic model during adaptatio…
Improved LoRA+ adapts large models more efficiently.
problem Suboptimal feature learning in LoRA for large models.
method Different learning rates for LoRA adapter matrices A and B.
result Improves performance and speed by 1-2% and up to 2X, respectively.
Edge devices adapt pre-trained models to local data without backpropagation.
problem Adapting pre-trained models to edge devices' local data distributions.
method Feed-forward latent domain adaptation using cross-attention.
result Consistent improvements over ERM baselines and domain-supervised adaptation.
Paper tackles test-time adaptation for tabular data.
problem Performance degradation due to distribution shifts in testing.
method Proposes FTAT for robustly adapting tabular models during testing.
result FTAT outperforms state-of-the-art methods on benchmark datasets.
New method adapts without backprop, faster and better.
problem Efficient domain adaptation without source data.
method Computing class prototypes from pre-trained model.
result Significant accuracy improvements over pre-trained model.
New adaptive test for NPIV models controls size and has superior power.
problem Testing inequality and equality restrictions in nonparametric IV models.
method Adaptive hypothesis test based on modified leave-one-out sample quadratic distance.
result Adaptive test attains the adaptive minimax rate of testing in .
A nonlinear wave alternative for the standard Black-Scholes option-pricing model is presented. The adaptive-wave model, representing 'controlled Brownian behavior' of financial markets, is formally defined by adaptive nonlinear Schrödinger (NLS) equations, defining the option-pricing wave function in terms of the stock…
DoubleAdapt improves stock trend forecasting by adapting models to evolving data.
problem Incremental learning for stock trend forecasting is challenging due to distribution shifts.
method DoubleAdapt framework with two adapters for data and model adaptation.
result DoubleAdapt achieves state-of-the-art predictive performance on real-world stock datasets.
Sparse Meta Networks for Sequential Adaptation and its Application to Adaptive Language Modellingcs.NE
Sparse Meta Networks adapt deep neural networks incrementally for fast learning.
problem Training deep neural networks is slow and impractical for complex, changing environments.
method Sparse Meta Networks use a memory layer to learn online sequential adaptation, accumulating fast-weights incrementally.
result Sparse Meta Networks achieve strong performance in various sequential adaptation scenarios.
Complementary Domain Adaptation and Generalization for Unsupervised Continual Domain Shift Learningcs.LG
CoDAG combines domain adaptation and generalization for unsupervised continual domain shift learning.
problem Acquiring knowledge in unsupervised continual domain shift learning.
method Complementary Domain Adaptation and Generalization (CoDAG) framework.
result CoDAG outperforms state-of-the-art models in all datasets and evaluation metrics.
This work proposes a meta-learning approach for better adaptation of source code models.
problem Adapting source code models to unseen local contexts.
method Formulated as a meta-learning problem, selecting targeted information for adaptation.
result Improved performance in code auto-completion tasks, especially for identifiers and literals.
Learn to automatically plug domain-specific modules into a common network.
problem Learning inflexibility and computational intensiveness in multi-domain learning.
method Neural Architecture Search (NAS) for data-driven adapter plugging and structure design.
result NAS-driven MDL model achieves comparable performance to existing approaches.
ADAPT improves robustness of Vision Transformers without full model fine-tuning.
problem Vulnerability of Vision Transformers to adversarial attacks.
method Parameter-efficient prompt tuning with ADAPT framework for adaptive adversarial training.
result ADAPT achieves robust accuracy of ~40% w.r.t. SOTA methods using only ~1% of the parameters.
A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one…
New method learns models to adapt to domain shifts at test time.
problem Learning models robust to distribution shifts in practical applications.
method Adaptive Risk Minimization (ARM) framework.
result Performance gains of 1-4% on image classification problems.
HireVAE adapts to market regimes for online stock prediction.
problem Building an online and adaptive factor model for stock prediction.
method HireVAE uses a hierarchical latent space to estimate latent factors from historical market information.
result HireVAE outperforms previous methods in active returns across benchmarks.
Adaptive Wave Models for Option Pricing Evolution: Nonlinear and Quantum Schrödinger Approachesq-fin.PR
Adaptive wave model for financial option pricing is proposed, as a high-complexity alternative to the standard Black--Scholes model. The new option-pricing model, representing a controlled Brownian motion, includes two wave-type approaches: nonlinear and quantum, both based on (adaptive form of) the Schrödinger equatio…
Develops a generic two-layer framework for adaptive ABMs.
problem Bi-level adaptation problem in ABMs: agents adapt to environment, and environment adapts to agents.
method Formalizes bi-level problem as a Stackelberg game with conditional policies, solving coupled non-linear equations.
result Unified framework for adaptive ABMs, addressing traditional ABM limitations.