Federated learning framework improves model generalization and privacy.
problem Communication overhead and statistical heterogeneity in FL.
method Prototypes and lightweight adapters for local model refinement.
result Improves classification accuracy over baseline algorithms.
We introduce an adaptive output-sensitive Metropolis-Hastings algorithm for probabilistic models expressed as programs, Adaptive Lightweight Metropolis-Hastings (AdLMH). The algorithm extends Lightweight Metropolis-Hastings (LMH) by adjusting the probabilities of proposing random variables for modification to improve c…
A lightweight framework improves convergence and stability of PINNs for complex PDEs.
problem Training instability and reduced accuracy in PINNs for complex PDEs.
method Adaptive curvature correction using secant information to optimize first-order optimizers.
result Consistent improvements in convergence speed, stability, and accuracy over standard optimizers.
ELF improves FM forecasts by efficiently using online feedback.
problem Fixed FMs fail to adapt to current data characteristics.
method ELF-Forecaster learns current data distribution, ELF-Weighter combines forecasts.
result ELF improves FM forecasts across various datasets.
We study the problem of using low computational cost to automate the choices of learners and hyperparameters for an ad-hoc training dataset and error metric, by conducting trials of different configurations on the given training data. We investigate the joint impact of multiple factors on both trial cost and model erro…
DriftLite improves inference quality of diffusion models without retraining.
problem Adapting pre-trained diffusion models to new target distributions without retraining.
method Lightweight, training-free particle-based approach that steers inference dynamics with optimal stability control.
result Consistently reduces variance and improves sample quality over existing methods.
Bayesian approach for policy search in stochastic domains.
problem Policy search in stochastic domains.
method Nested probabilistic programs, Lightweight Metropolis-Hastings (LMH) adaptation.
result Similar quality policies learned with simpler algorithm.
Learn to automatically plug domain-specific modules into a common network.
problem Learning inflexibility and computational intensiveness in multi-domain learning.
method Neural Architecture Search (NAS) for data-driven adapter plugging and structure design.
result NAS-driven MDL model achieves comparable performance to existing approaches.
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
ReRe detects anomalies in real-time for time series data.
problem Real-time anomaly detection for time series data requires human intervention or domain knowledge and high computation complexity.
method ReRe uses two lightweight LSTM models to predict and determine anomalies based on historical data and adaptive thresholds.
result ReRe detects anomalies in real-time without requiring human intervention or domain knowledge.
DBQ quantizes lightweight networks efficiently for resource-constrained devices.
problem High computational and storage complexity of deep neural networks on resource-constrained devices.
method A differentiable non-uniform quantizer that can be mapped onto efficient ternary-based dot product engines.
result Achieves state-of-the-art results with minimal training overhead and best accuracy-complexity trade-off.
Deep neural networks and decision trees operate on largely separate paradigms; typically, the former performs representation learning with pre-specified architectures, while the latter is characterised by learning hierarchies over pre-specified features with data-driven architectures. We unite the two via adaptive neur…
PPI++ uses machine learning predictions to improve inference from small datasets.
problem Efficient inference from small labeled datasets with high-quality predictions.
method Adapts prediction-powered inference (PPI) to compute confidence sets for any parameter dimensionality.
result Improves classical intervals using only labeled data, always yielding better results.
RSAC improves lightweight continuous learning efficiency.
problem Excessive training time and memory usage in continuous learning.
method Regularized subspace approximation classifier with feature reduction and regularization.
result RSAC achieves more efficient continuous learning than prior methods.
ALT transforms time series data for better classification.
problem Efficiently classifying time series data with varying temporal scales.
method ALT algorithm using variable-length shifted time windows.
result State-of-the-art performance with minimal computational overhead.
The success of deep learning models is heavily tied to the use of massive amount of labeled data and excessively long training time. With the emergence of intelligent edge applications that use these models, the critical challenge is to obtain the same inference capability on a resource-constrained device while providi…
DLC enhances distillation-based continual learning with lightweight plugins.
problem Stability-plasticity dilemma in distillation-based continual learning.
method DLC deploys lightweight residual plugins into the classifier-proximal layer of a shared feature extractor.
result Significant 8% accuracy gain on large-scale benchmarks with minimal parameter increase.
iSTFTNet2 improves iSTFTNet's speed and lightness with 1D-2D CNN.
problem Efficiently synthesizing high-fidelity speech.
method Improved iSTFTNet using 1D-2D CNNs for temporal and spectrogram structures.
result iSTFTNet2 is faster and more lightweight with comparable speech quality.
Enhances GNNs for causal relationship learning.
problem Lack of robust causal modeling in GNNs.
method Synthesized dataset with known causal relationships, lightweight GNN module.
result Empirically validated GNN module improves causal learning.
Method adapts frozen models for few-shot tasks without training.
problem Deployment constraints limit model updates, necessitating new adaptation methods.
method Exponential tilting of latent distribution for inference.
result Method outperforms parameter-update methods across benchmarks.
Augment small datasets with synthetic backgrounds to train lightweight CNNs for human pose estimation.
problem Training CNNs from limited real-world data for human pose estimation.
method Synthetic background substitution for data augmentation.
result Improves generalization to unseen environments.
PLoP optimizes LoRA placement for efficient large model finetuning.
problem Improving efficiency of LoRA adaptation for large models.
method Intuitive theoretical analysis for automatic adapter placement.
result PLoP consistently outperforms existing placement strategies.
Improves regression models' performance on covariate shift.
problem Out-of-distribution generalization for regression.
method Spectrally adapting the weights of a pre-trained neural regression model.
result Spectral adaptation improves out-of-distribution performance.
CMoS improves time series forecasting with minimal parameters.
problem Efficiently forecasting time series data with limited resources.
method CMoS directly models chunk-wise spatial correlations, using Correlation Mixing and Periodicity Injection techniques.
result CMoS outperforms state-of-the-art models with minimal parameters.
We present a machine learning-based approach to lossy image compression which outperforms all existing codecs, while running in real-time. Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of generic images across all …
LoRA fine-tuning on CPUs without GPUs achieves comparable performance to GPU-based methods.
problem Limited computational resources for fine-tuning LLMs.
method Developed a CPU-efficient method to learn meta-operators for LoRA weights.
result LoRA adapters trained on CPUs outperform base models on downstream tasks.
LaLoRA prevents forgetting in LoRA fine-tuning.
problem Catastrophic forgetting in fine-tuned models.
method LaLoRA applies Laplace approximation to LoRA weights for regularization.
result Improved learning-forgetting trade-off with controllable regularization strength.
Mapper-GIN simplifies 3D point cloud classification with lightweight structure.
problem Robust 3D point cloud classification under corruption.
method Mapper algorithm for structural decomposition, GIN for graph classification.
result Mapper-GIN achieves competitive accuracy with minimal parameters.
GIBLy adds geometric priors to 3D segmentation models, improving performance with minimal overhead.
problem Lack of explicit geometric information in 3D semantic segmentation models.
method Introduces GIBLy, a lightweight geometric inductive bias layer that integrates learnable geometric priors into existing 3D segmentation pipelines.
result Consistent performance gains across multiple benchmarks, including up to +11.5% mIoU on TS40K with PTV3.
Simplified image clustering achieves competitive results without text-based embeddings.
problem Complexity and resource requirements of state-of-the-art clustering methods.
method SCP: trains a small cluster head using pre-trained vision model features and positive data pairs.
result SCP achieves highly competitive performance on various benchmark datasets.
Paper tackles unknown participation in FL, proposing FedAU for better performance.
problem Unknown participation statistics in federated learning impact performance.
method Adapting aggregation weights in FedAvg based on participation history.
result FedAU converges to optimal solution and has desirable properties.
Improved neural ODEs learn adaptable flows.
problem Neural ODEs struggle with expressive power and adaptability.
method Introduce N-CODE modules with dynamic parameters controlled by a trainable map.
result N-CODE modules enhance expressivity of neural ODEs.
ALT improves TSC by capturing complex patterns in time series data.
problem Challenges in traditional TSC methods with time series complexity and variability.
method ALT incorporates variable-length shifted time windows to enhance LLT for better feature representation.
result ALT achieves state-of-the-art performance with few hyperparameters.
BERTino is a lightweight Italian DistilBERT model for NLP tasks.
problem High computational and memory demands of large language models.
method Developed a DistilBERT model tailored for Italian language.
result F1 scores comparable to BERTBASE with significant speed improvements.
New method restores source features for SFDA without source data.
problem Domain adaptation without access to source data.
method Feature Restoration (FR) and Bottom-Up Feature Restoration (BUFR).
result BUFR outperforms existing SFDA methods in accuracy, calibration, and data efficiency.
FADE adapts machine learning models to evolving data efficiently.
problem Sequential covariate shift in dynamic environments.
method FADE uses Fisher information geometry for robust learning under SCS.
result FADE achieves up to 19% higher accuracy under severe shifts.
We develop a progressive training approach for neural networks which adaptively grows the network structure by splitting existing neurons to multiple off-springs. By leveraging a functional steepest descent idea, we derive a simple criterion for deciding the best subset of neurons to split and a splitting gradient for …
TSLANet improves time series models by capturing long-term and short-term interactions.
problem Noise sensitivity, computational efficiency, and overfitting in Transformer-based models for time series data.
method Adaptive Spectral Block and Interactive Convolution Block for robust feature representation and noise mitigation.
result TSLANet outperforms state-of-the-art models in various time series tasks.
AMS improves video inference on edge devices by adapting a small model with online knowledge distillation.
problem High computation cost of Deep Neural Networks for real-time video inference on edge devices.
method AMS uses a remote server to continually train and adapt a small model on edge devices, using online knowledge distillation from a large model.
result 0.4--17.8 percent mean Intersection-over-Union improvement in video semantic segmentation.
MELO predicts electricity loads by adapting to shifts without external indicators.
problem Adapting to non-stationary prediction challenges in online settings.
method MELO combines multiple forgetting factors and aggregation rules to adaptively predict.
result MELO reduces RMSE by 34.7% compared to base predictors and external covariates.
Automates design of lightweight neural networks for image classification.
problem Designing efficient neural networks for edge devices with limited computational resources.
method Uses the Mesh Adaptive Direct Search (MADS) algorithm to optimize network architecture.
result Achieves comparable performance to standard methods with fewer design trials.
Improved Markov models learn from their mistakes and adapt to problem complexity.
problem Limitations of standard masked discrete diffusion models in reasoning tasks.
method Learning a Markov transition kernel trained on its own outputs, allowing remasking and adaptation.
result Significant improvement in solving reasoning problems, especially Sudoku-Extreme and Countdown-4.
TinyXRA assesses financial risks from 10-K reports using a lightweight transformer model.
problem Comprehensive risk assessment from financial reports, distinguishing between upside and downside risk.
method Lightweight transformer model with dynamic attention, incorporating skewness, kurtosis, and Sortino ratio.
result State-of-the-art predictive accuracy and transparent risk assessments.
Plug-in robust NPE method adapts summaries independently of pretrained NPE.
problem Misspecification of neural posterior estimators under test data distribution.
method Minimum-distance summaries using maximum mean discrepancy (MMD).
result Substantial robustness gains with minimal additional overhead.
Coresets are compact representations of data sets such that models trained on a coreset are provably competitive with models trained on the full data set. As such, they have been successfully used to scale up clustering models to massive data sets. While existing approaches generally only allow for multiplicative appro…
QTMRL uses RL with multi-indicators to improve trading adaptability.
problem Traditional trading models fail in volatile markets due to rigid assumptions.
method Combines multi-indicators with RL for adaptive portfolio management.
result QTMRL outperforms baselines in profitability and risk control.
Executing deep neural networks for inference on the server-class or cloud backend based on data generated at the edge of Internet of Things is desirable due primarily to the limited compute power of edge devices and the need to protect the confidentiality of the inference neural networks. However, such a remote inferen…
Token-adaptive FFN design improves LLM expressivity.
problem Fixed activation functions limit FFN expressivity in LLMs.
method Mixture of Activations (MoA) and learnable activations (LA).
result MoA achieves lower terminal loss and better scaling than baselines.