Efficient inference for adaptive data with directional stability condition.
problem Efficient inference on scalar targets after adaptive data collection.
method Introduces directional stability, a weaker condition than i.i.d. data, and shows asymptotic normality and efficiency of estimators.
result Estimators remain asymptotically normal and semiparametrically efficient under directional stability.
New method improves treatment effect estimation in adaptive experiments with noncompliance.
problem Estimating average treatment effect in adaptive experiments with binary instrumental variable.
method AMRIV estimator that balances outcome noise and compliance variability.
result AMRIV achieves semiparametric efficiency bound and is robust to noncompliance.
FLoE adapts LLMs by selectively deploying LoRA adapters based on layer importance and task requirements.
problem Uniform LoRA deployment across all layers leads to inefficient and redundant parameter allocation.
method FLoE uses Fisher information to dynamically identify task-critical layers and optimizes LoRA ranks.
result FLoE achieves significant efficiency-accuracy trade-offs, especially in resource-constrained environments.
Improved LoRA+ adapts large models more efficiently.
problem Suboptimal feature learning in LoRA for large models.
method Different learning rates for LoRA adapter matrices A and B.
result Improves performance and speed by 1-2% and up to 2X, respectively.
In this paper we investigate the adaptive market efficiency of the agricultural commodity futures market, using a sample of eight futures contracts. Using a battery of nonlinear tests, we uncover the nonlinear serial dependence in the returns series. We run the Hinich portmanteau bicorrelation test to uncover the momen…
ELSA efficiently adapts to label shift without post-prediction calibrations.
problem Domain adaptation with label shift across training and testing datasets.
method Moment-matching framework based on influence function geometry; solves linear systems for adaptation weights.
result ELSA estimator is n \sqrt{n} n -consistent and asymptotically normal, achieving state-of-the-art estimation performance. PLoP optimizes LoRA placement for efficient large model finetuning.
problem Improving efficiency of LoRA adaptation for large models.
method Intuitive theoretical analysis for automatic adapter placement.
result PLoP consistently outperforms existing placement strategies.
This study optimizes covariate density and propensity score for efficient ATE estimation.
problem Efficiently estimating average treatment effects (ATEs) with minimal variance.
method Adaptive experiment optimizing both covariate density and propensity score.
result Proposed method minimizes the semiparametric efficiency bound for ATE estimation.
Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself a…
The paper introduces GAER to assess market feasibility under geopolitical and institutional constraints.
problem Feasibility of adaptive market efficiency under heterogeneous institutional and geopolitical conditions.
method Structural framework integrating adaptive market theory, institutional economics, and political economy.
result GAER as a diagnostic indicator for portfolio construction feasibility.
Develops adaptive framework for estimating survival effects with censoring.
problem Estimating causal effects in survival data with censoring.
method Derives semiparametric efficiency bound, proposes efficiency-optimal allocation policy, and develops Adaptive Survival Estimator (ASE).
result ASE achieves asymptotic normality via martingale central limit theorem and demonstrates efficiency gains over uniform randomization.
Paper proposes an adaptive gradient method for federated learning.
problem Communication efficiency in federated learning.
method Adaptive gradient method for federated learning.
result Guaranteed convergence and communication efficiency in federated learning.
Adaptive sampling method improves efficiency in complex target distributions.
problem Efficiency of importance sampling in complex target distributions, especially multimodal distributions in high-dimensional spaces.
method Proposes an adaptive scheme combining global sampling with delayed weighting to promote efficient exploration of target distributions.
result The proposed algorithm is geometrically convergent under mild assumptions and demonstrates improved efficiency in various numerical experiments.
We study how to efficiently estimate average treatment effects (ATEs) using adaptive experiments. In adaptive experiments, experimenters sequentially assign treatments to experimental units while updating treatment assignment probabilities based on past data. We start by defining the efficient treatment-assignment prob…
GeLoRA optimizes LoRA fine-tuning by dynamically adjusting ranks based on intrinsic dimensionality.
problem Efficient fine-tuning of large language models with limited computational resources.
method GeLoRA computes intrinsic dimensionality to adaptively select LoRA ranks, balancing expressivity and efficiency.
result GeLoRA consistently outperforms recent baselines within the same parameter budget on multiple tasks.
CoLoRA leverages task similarity to boost fine-tuning efficiency.
problem Efficiently fine-tuning large foundation models with scarce labeled data.
method CoLoRA trains a shared adapter for task similarity and personalized adapters for user-specific tasks.
result CoLoRA significantly boosts fine-tuning performance when tasks are similar.
A new federated learning method reduces communication costs and improves adaptivity.
problem Large communication overhead and lack of adaptivity in federated learning.
method FedCAMS: A novel communication-efficient adaptive federated learning method with theoretical guarantees.
result FedCAMS achieves the same convergence rate as non-compressed federated learning methods.
Optimizes biomolecular simulations by ranking adaptive sampling policies.
problem Efficiently sampling biomolecular systems to capture complex dynamical behaviors.
method Metric-driven ranking of adaptive sampling policies to identify the optimal policy for each round.
result Different adaptive sampling policies lead to faster convergence and improved sampling performance.
New study shows non-adaptive trials can be outperformed by adaptive designs in treatment selection.
problem Determining the best allocation of resources in clinical trials.
method Analysis of batched arm elimination designs and comparison with completely randomized trials.
result Simple adaptive designs universally and strictly dominate non-adaptive completely randomized trials for at least three treatment arms.
Efficient inference method for adaptive experiments with tighter confidence sequences.
problem Efficient inference of Average Treatment Effect in a changing policy sequential experiment.
method Semiparametric efficient inference using Adaptive Augmented Inverse-Probability Weighted estimator and asymptotic confidence sequences.
result Derives tighter confidence sequences for adaptive experiments under data-dependent stopping times.
Efficient CF approach using fast adaptive PCA for recommender systems.
problem Matrix completion problem in recommender systems.
method Fast adaptive randomized singular value decomposition (SVD) and termination mechanism for latent factors.
result The approach achieves near optimal prediction accuracy with high runtime efficiency.
CPATTA uses conformal prediction for efficient test-time adaptation.
problem Low data selection efficiency in existing ATTA methods.
method Conformal Prediction, online weight-update algorithm, domain-shift detector, staged update scheme.
result CPATTA consistently outperforms state-of-the-art methods by 5% in accuracy.
Adaptive HMC improves sampling efficiency by optimizing mass matrix.
problem Inefficient HMC performance due to mass matrix choice.
method Gradient-based adaptation of mass matrix to maximize proposal entropy.
result Adaptation method outperforms HMC variants by optimizing mass matrix.
DANCE improves prediction set efficiency for deep learning models.
problem Inefficient, overly conservative prediction sets for pre-trained models.
method DANCE combines adaptive kernel regression and nearest-neighbor approach.
result DANCE produces more efficient and robust prediction sets.
A new method for efficient neural network fine-tuning using queryable low-rank update atoms.
problem Rigidity of static low-rank adaptation methods when input and depth-wise computation vary.
method A shared queryable memory of low-rank update atoms, allowing dynamic and context-sensitive adaptation.
result Improves final test performance and training stability compared to standard low-rank adaptation.
Credit assignment in Meta-reinforcement learning (Meta-RL) is still poorly understood. Existing methods either neglect credit assignment to pre-adaptation behavior or implement it naively. This leads to poor sample-efficiency during meta-training as well as ineffective task identification strategies. This paper provide…
EVA adapts LoRA for faster, more efficient fine-tuning.
problem Fast and efficient fine-tuning of large models for specific tasks.
method EVA uses directions capturing most activation variance for initialization, maximizing gradient signal and reducing parameters.
result EVA achieves faster convergence and higher average scores across tasks, reducing parameters.
TensorGuide improves LoRA efficiency and expressivity through joint tensor-train optimization.
problem Limited expressivity and generalization of standard LoRA.
method TensorGuide uses a unified tensor-train structure with controlled Gaussian noise to generate correlated low-rank matrices.
result TensorGuide achieves superior accuracy and scalability with fewer parameters compared to standard LoRA and TT-LoRA.
Study optimizes health incentives to balance efficiency and fairness.
problem Designing health incentives to balance efficiency and fairness.
method Inverse behavioral optimization framework integrating QALY-based incentives and adaptive learning.
result Modern health systems operate near an efficiency-saturated frontier, with small fairness adjustments yielding diminishing returns.
Enhances physics-informed neural networks with adaptive sampling and weighting.
problem Challenges in training physics-informed neural networks on complex problems.
method Hybrid adaptive sampling and weighting method.
result Consistently improves prediction accuracy and training efficiency.
ADCMs adaptively discretize CMs for efficient training.
problem Manual discretization schemes cause repeated adjustments for different noise schedules and datasets.
method Unified framework with optimization problem, local and global consistency constraints, and Gauss-Newton method.
result Significantly improve training efficiency and generative performance of CMs.
Adaptive quantization improves SGD accuracy in data-parallel settings.
problem Fixed gradient quantization schemes lead to suboptimal performance in deep learning.
method Developed adaptive quantization schemes ALQ and AMQ that update compression schemes based on gradient statistics.
result Improved validation accuracy on CIFAR-10 and ImageNet datasets by 2% and 1% respectively.
New model-based methods adapt pre-trained policies to unseen environments efficiently.
problem High sample complexity in reinforcement learning limits practical applications.
method Combines online learning and adaptive control to adapt policies in unseen environments.
result Proves policies can quickly recover trajectories from source to target environments.
New method combines HQR and WACI for better time series prediction intervals.
problem Challenges in creating reliable prediction intervals for time series forecasting.
method Combining Heteroscedastic Quantile Regression (HQR) with Width-Adaptive Conformal Inference (WACI).
result Combined approach meets or surpasses typical benchmarks for validity and efficiency.
Kriging is an efficient machine-learning tool, which allows to obtain an approximate response of an investigated phenomenon on the whole parametric space. Adaptive schemes provide a the ability to guide the experiment yielding new sample point positions to enrich the metamodel. Herein a novel adaptive scheme called Mon…
Adaptive batch sizes improve local gradient methods in distributed training.
problem Communication bottlenecks in distributed deep learning.
method Adaptive batch size strategies for local gradient methods.
result Adaptive batch sizes reduce minibatch gradient variance and improve training efficiency.
Optimal transport between Gaussian Mixture Models improves domain adaptation efficiency.
problem Adapting machine learning models to new data distributions with minimal access.
method Optimal transport between Gaussian Mixture Models (GMMs) for domain adaptation.
result Our methods are more efficient and scalable with sample size and dimensions.
A new method improves EEG classification across subjects efficiently.
problem Challenges in adapting and retaining knowledge for EEG classifiers across different subjects.
method Meta UPdate Strategy (MUPS-EEG) for continuous EEG classification.
result Outperforms current state-of-the-art methods in adapting to new subjects and retaining knowledge of learned subjects.
We analyze the computational limits of LoRA for transformer models using fine-grained complexity theory.
problem Computational efficiency of LoRA fine-tuning for transformer models.
method Fine-grained complexity theory, identifying phase transitions, almost linear algorithms.
result Existence of almost linear algorithms for LoRA adaptation based on specific norms.
Adaptive teacher improves sample efficiency and mode coverage in sampling tasks.
problem Efficient exploration and mode coverage in sampling tasks with amortized inference.
method An adaptive training distribution (teacher) guides the training of the primary sampler (student).
result Improves sample efficiency and mode coverage across various tasks.
Estimates isotonic functions under unknown permutations, achieving optimal statistical and computational efficiency.
problem Estimating isotonic functions with unknown permutations in multiway comparison data.
method Mirsky partition estimator for minimax optimal and adaptive estimation.
result Achieves optimal worst-case statistical performance and computational efficiency.
Efficiently selects seed nodes to maximize content influence in unknown social networks.
problem Maximizing content spread in social networks with unknown network model.
method Formulated as an infinite-horizon discounted MDP, uses model-based reinforcement learning to select seed users adaptively.
result Established a regret bound of O ~ ( T ) \widetilde O(\sqrt{T}) O ( T ) for the algorithm. Adaptive sparse GP model for non-stationary data.
problem Non-stationary environments require efficient model updates.
method Reformulated variational sparse GP with adaptive forgetting factor and single inducing point update.
result Fast convergence and efficient model update in non-stationary environments.
LoRA enhances model adaptability without increasing parameters.
problem Theoretical understanding of LoRA's adaptability.
method Theoretical analysis of LoRA's expressive power.
result LoRA can adapt any model to a smaller target model with a specific rank.
Adaptive regularization methods pre-multiply a descent direction by a preconditioning matrix. Due to the large number of parameters of machine learning problems, full-matrix preconditioning methods are prohibitively expensive. We show how to modify full-matrix adaptive regularization in order to make it practical and e…
Prototype extraction framework for domain adaptation.
problem Statistical distance minimization issues in unsupervised domain adaptation.
method Memory and computation-efficient probabilistic framework for class prototype extraction and feature alignment.
result Competitive performance with state-of-the-art methods, no additional model parameters required.
Paper proposes a new method to optimize robot body structure and control policy.
problem Optimizing robot body structure and control policy in a coupled manner.
method Revisits co-design problem as a Stackelberg game, incorporating control adaptation dynamics.
result Stackelberg PPO outperforms standard PPO in stability and performance.
Meta-Reinforcement learning approaches aim to develop learning procedures that can adapt quickly to a distribution of tasks with the help of a few examples. Developing efficient exploration strategies capable of finding the most useful samples becomes critical in such settings. Existing approaches towards finding effic…