Packed-Ensembles improve uncertainty estimation in constrained hardware.
problem Hardware limitations restrict the size of ensembles and network capacity, degrading performance.
method Packed-Ensembles (PE) design and train lightweight structured ensembles by modulating encoding space and parallelizing into a single backbone.
result PE accurately preserves diversity and maintains performance on key metrics like accuracy, calibration, and out-of-distribution detection.
Paper proposes GP-NAS-ensemble for fast neural architecture performance prediction.
problem Estimating neural network performance without training time-consuming evaluations.
method GP-NAS-ensemble framework using ensemble learning improvements.
result Ranked second in a NAS performance prediction challenge.
A lightweight model predicts IT system KPIs from historical data.
problem Predicting future KPIs of interconnected IT systems is hard due to diverse and changing components.
method A weighted heterogeneous ensemble method combining neural network and mean predictor.
result Achieved R2 scores of 0.10 and 0.15 on test data. AdaNet is a lightweight TensorFlow-based (Abadi et al., 2015) framework for automatically learning high-quality ensembles with minimal expert intervention. Our framework is inspired by the AdaNet algorithm (Cortes et al., 2017) which learns the structure of a neural network as an ensemble of subnetworks. We designed it…
Ensemble methods for supervised machine learning have become popular due to their ability to accurately predict class labels with groups of simple, lightweight "base learners." While ensembles offer computationally efficient models that have good predictive capability they tend to be large and offer little insight into…
Binarization is an attractive strategy for implementing lightweight Deep Convolutional Neural Networks (CNNs). Despite the unquestionable savings offered, memory footprint above all, it may induce an excessive accuracy loss that prevents a widespread use. This work elaborates on this aspect introducing TentacleNet, a n…
Emotions play a crucial role in human interaction, health care and security investigations and monitoring. Automatic emotion recognition (AER) using electroencephalogram (EEG) signals is an effective method for decoding the real emotions, which are independent of body gestures, but it is a challenging problem. Several …
KOD detects outliers in high-dimensional data.
problem Challenges in outlier detection in high-dimensional settings.
method Kernel transformation followed by projection pursuit approach with ensemble of directions and result combination.
result Empirical evaluations show effectiveness on various datasets.
This paper connects RND, deep ensembles, and Bayesian inference, providing a unified theoretical perspective.
problem Uncertainty quantification in deep learning models.
method Analysis of Random Network Distillation (RND) within the neural tangent kernel framework.
result The uncertainty signal from RND is equivalent to the predictive variance of a deep ensemble and can be made to mirror the centered posterior predictive distribution of Bayesian inference.
KineticSim: A lightweight, high-performance execution engine for real-time market simulators
problem Simulating financial markets at scale with multi-agent models
method Reusable parallel design pattern: persistent, state-carrying clearing for iterative multi-agent reductions
result Reduces per-step critical-path depth from Theta(L+A) to Theta(log L + ceil(A/L))
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
DBQ quantizes lightweight networks efficiently for resource-constrained devices.
problem High computational and storage complexity of deep neural networks on resource-constrained devices.
method A differentiable non-uniform quantizer that can be mapped onto efficient ternary-based dot product engines.
result Achieves state-of-the-art results with minimal training overhead and best accuracy-complexity trade-off.
RSAC improves lightweight continuous learning efficiency.
problem Excessive training time and memory usage in continuous learning.
method Regularized subspace approximation classifier with feature reduction and regularization.
result RSAC achieves more efficient continuous learning than prior methods.
DLC enhances distillation-based continual learning with lightweight plugins.
problem Stability-plasticity dilemma in distillation-based continual learning.
method DLC deploys lightweight residual plugins into the classifier-proximal layer of a shared feature extractor.
result Significant 8% accuracy gain on large-scale benchmarks with minimal parameter increase.
Two new methods reduce random forest latency and improve accuracy.
problem High latency and memory demands in deep random forest models.
method DiNo and RanBu convert shallow random forests into efficient predictors.
result RanBu matches or exceeds full-depth random forest accuracy with up to 95% reduction in time.
Framework for precise recall control in spatial conflation tasks.
problem Precise recall control in large-scale spatial conflation tasks to avoid downstream analytics failures and excessive manual review.
method End-to-end framework using equigrid bounding-box filter, CSR representation, neural ranker, and inverse-variance weighted ensemble of threshold estimators.
result Achieves exact recall with sub-percent variance over tens of millions of geometry pairs, runs on a single TPU v3 core.
iSTFTNet2 improves iSTFTNet's speed and lightness with 1D-2D CNN.
problem Efficiently synthesizing high-fidelity speech.
method Improved iSTFTNet using 1D-2D CNNs for temporal and spectrogram structures.
result iSTFTNet2 is faster and more lightweight with comparable speech quality.
Augment small datasets with synthetic backgrounds to train lightweight CNNs for human pose estimation.
problem Training CNNs from limited real-world data for human pose estimation.
method Synthetic background substitution for data augmentation.
result Improves generalization to unseen environments.
CMoS improves time series forecasting with minimal parameters.
problem Efficiently forecasting time series data with limited resources.
method CMoS directly models chunk-wise spatial correlations, using Correlation Mixing and Periodicity Injection techniques.
result CMoS outperforms state-of-the-art models with minimal parameters.
Federated learning framework improves model generalization and privacy.
problem Communication overhead and statistical heterogeneity in FL.
method Prototypes and lightweight adapters for local model refinement.
result Improves classification accuracy over baseline algorithms.
Mantis improves time series classification using a transformer model trained on synthetic data.
problem Insufficient application of foundation models to time series classification.
method Pre-trained transformer model on synthetic data, enhanced test-time methodology.
result Mantis achieves state-of-the-art performance across diverse datasets.
KineticSim accelerates financial market simulations 3406x over CPU.
problem Simulating financial markets at scale with multi-agent models is bottlenecked by sequential processing and GPU kernel overhead.
method Formalized and implemented a reusable parallel design pattern for iterative multi-agent reductions in thread-block shared memory.
result Achieved a peak throughput of over 54.7 billion agent-events per second, delivering 3406x speedup over CPU.
Mapper-GIN simplifies 3D point cloud classification with lightweight structure.
problem Robust 3D point cloud classification under corruption.
method Mapper algorithm for structural decomposition, GIN for graph classification.
result Mapper-GIN achieves competitive accuracy with minimal parameters.
GIBLy adds geometric priors to 3D segmentation models, improving performance with minimal overhead.
problem Lack of explicit geometric information in 3D semantic segmentation models.
method Introduces GIBLy, a lightweight geometric inductive bias layer that integrates learnable geometric priors into existing 3D segmentation pipelines.
result Consistent performance gains across multiple benchmarks, including up to +11.5% mIoU on TS40K with PTV3.
BERTino is a lightweight Italian DistilBERT model for NLP tasks.
problem High computational and memory demands of large language models.
method Developed a DistilBERT model tailored for Italian language.
result F1 scores comparable to BERTBASE with significant speed improvements.
A new method calibrates scientific models by adding randomness to their predictions.
problem Current scientific foundation models lack calibrated uncertainty.
method Stochastic Attention, which randomizes attention weights using multinomial samples.
result Stochastic Attention achieves the strongest native calibration and sharpest prediction intervals.
TinyXRA assesses financial risks from 10-K reports using a lightweight transformer model.
problem Comprehensive risk assessment from financial reports, distinguishing between upside and downside risk.
method Lightweight transformer model with dynamic attention, incorporating skewness, kurtosis, and Sortino ratio.
result State-of-the-art predictive accuracy and transparent risk assessments.
Coresets are compact representations of data sets such that models trained on a coreset are provably competitive with models trained on the full data set. As such, they have been successfully used to scale up clustering models to massive data sets. While existing approaches generally only allow for multiplicative appro…
Estimates interactions between modalities for multimodal data.
problem Accurately quantifying interactions between different data types.
method Developed Lightweight Sample-wise Multimodal Interaction (LSMI) estimator using pointwise information theory.
result LSMI reveals fine-grained dynamics in multimodal data.
We introduce an adaptive output-sensitive Metropolis-Hastings algorithm for probabilistic models expressed as programs, Adaptive Lightweight Metropolis-Hastings (AdLMH). The algorithm extends Lightweight Metropolis-Hastings (LMH) by adjusting the probabilities of proposing random variables for modification to improve c…
Running high-resolution physical models is computationally expensive and essential for many disciplines. Agriculture, transportation, and energy are sectors that depend on high-resolution weather models, which typically consume many hours of large High Performance Computing (HPC) systems to deliver timely results. Many…
A lightweight framework improves convergence and stability of PINNs for complex PDEs.
problem Training instability and reduced accuracy in PINNs for complex PDEs.
method Adaptive curvature correction using secant information to optimize first-order optimizers.
result Consistent improvements in convergence speed, stability, and accuracy over standard optimizers.
Executing deep neural networks for inference on the server-class or cloud backend based on data generated at the edge of Internet of Things is desirable due primarily to the limited compute power of edge devices and the need to protect the confidentiality of the inference neural networks. However, such a remote inferen…
This thesis enhances ML reliability by selectively abstaining from predictions when uncertain.
problem Improving reliability in machine learning systems, especially in high-stakes domains.
method Exploiting uncertainty signals from training trajectories to develop lightweight, post-hoc abstention methods compatible with differential privacy.
result A robust trajectory-based approach to selective prediction that maintains high accuracy under privacy noise.
Optimizes neural architecture search to generate novel lightweight models.
problem Over-reliance on expert knowledge limits NAS to local optima, preventing architectural breakthroughs.
method Casts NAS as an optimization problem, introduces a hierarchical graph-based search space, and uses Bayesian optimization.
result Generates extremely lightweight yet competitive models on six benchmark datasets.
We study the problem of using low computational cost to automate the choices of learners and hyperparameters for an ad-hoc training dataset and error metric, by conducting trials of different configurations on the given training data. We investigate the joint impact of multiple factors on both trial cost and model erro…
ELF improves FM forecasts by efficiently using online feedback.
problem Fixed FMs fail to adapt to current data characteristics.
method ELF-Forecaster learns current data distribution, ELF-Weighter combines forecasts.
result ELF improves FM forecasts across various datasets.
Framework disentangles deep feature uncertainty for efficient inference.
problem Inference-time uncertainty estimation for reliable decision-making.
method Uncertainty-Guided Inference-Time Selection framework.
result Significantly tighter prediction intervals and 60% compute reduction.
Interpretable additive models outperform complex DL and hybrid pipelines for air quality forecasting.
problem Accurate forecasting of urban air pollution for public health and policy guidance.
method Investigated lightweight additive models (FBP, NP) vs. deep learning and hybrid pipelines on Beijing PM2.5 and PM10 data.
result Facebook Prophet consistently outperformed NeuralProphet and traditional models, achieving high R2 values. Kolmogorov-Arnold network improves GW catalog posterior construction.
problem Efficiently constructing posterior distributions for GW catalogs.
method Using the Kolmogorov-Arnold network to create lightweight neural density estimators.
result Kolmogorov-Arnold network achieves superior interpretability and accuracy in posterior construction.
Flatland is a simple, lightweight environment for fast prototyping and testing of reinforcement learning agents. It is of lower complexity compared to similar 3D platforms (e.g. DeepMind Lab or VizDoom), but emulates physical properties of the real world, such as continuity, multi-modal partially-observable states with…
ReRe detects anomalies in real-time for time series data.
problem Real-time anomaly detection for time series data requires human intervention or domain knowledge and high computation complexity.
method ReRe uses two lightweight LSTM models to predict and determine anomalies based on historical data and adaptive thresholds.
result ReRe detects anomalies in real-time without requiring human intervention or domain knowledge.
Method transfers feature representation from large to small models using perception coherence.
problem Transfer feature representation from large to small models.
method Defines perception coherence, proposes loss function to minimize.
result Method outperforms or achieves on-par performance compared to strong baseline methods.
FEDS distills LIC model knowledge into a lightweight student for efficient compression.
problem Efficiently compress images with high performance and low resources.
method FEDS combines feature alignment and entropy-based loss for lightweight compression.
result Student model matches teacher's performance while reducing parameters and speeding up.
iSTFTNet speeds up mel-spectrogram vocoders without sacrificing quality.
problem Efficiently converting mel-spectrograms to speech with minimal computation.
method Replaces convolutional layers with iSTFT after frequency dimension reduction.
result Significant reduction in computational cost with comparable quality.
SWIFT improves time series forecasting on edge devices with wavelet decomposition.
problem Efficient time series forecasting on resource-constrained devices.
method Wavelet decomposition, cross-band fusion, and shared linear mapping.
result SWIFT achieves state-of-the-art performance on multiple datasets.
This work optimizes DNN inference for energy-harvesting devices by compressing and selectively executing neural network exits.
problem Inference delays and energy inefficiency in energy-harvesting devices.
method Developed a power trace-aware and exit-guided network compression algorithm for multi-exit neural networks.
result Superior accuracy and reduced latency compared to state-of-the-art techniques.
A novel framework combines LLMs and RL for financial portfolio optimization.
problem Optimizing financial portfolios using sentiment analysis and market indicators.
method Hierarchical RL structure with base, meta, and super-agents.
result Achieved a 26% annualized return and Sharpe ratio of 1.2.