DBQ quantizes lightweight networks efficiently for resource-constrained devices.
problem High computational and storage complexity of deep neural networks on resource-constrained devices.
method A differentiable non-uniform quantizer that can be mapped onto efficient ternary-based dot product engines.
result Achieves state-of-the-art results with minimal training overhead and best accuracy-complexity trade-off.
Optimizes neural architecture search to generate novel lightweight models.
problem Over-reliance on expert knowledge limits NAS to local optima, preventing architectural breakthroughs.
method Casts NAS as an optimization problem, introduces a hierarchical graph-based search space, and uses Bayesian optimization.
result Generates extremely lightweight yet competitive models on six benchmark datasets.
GIBLy adds geometric priors to 3D segmentation models, improving performance with minimal overhead.
problem Lack of explicit geometric information in 3D semantic segmentation models.
method Introduces GIBLy, a lightweight geometric inductive bias layer that integrates learnable geometric priors into existing 3D segmentation pipelines.
result Consistent performance gains across multiple benchmarks, including up to +11.5% mIoU on TS40K with PTV3.
BERTino is a lightweight Italian DistilBERT model for NLP tasks.
problem High computational and memory demands of large language models.
method Developed a DistilBERT model tailored for Italian language.
result F1 scores comparable to BERTBASE with significant speed improvements.
Mapper-GIN simplifies 3D point cloud classification with lightweight structure.
problem Robust 3D point cloud classification under corruption.
method Mapper algorithm for structural decomposition, GIN for graph classification.
result Mapper-GIN achieves competitive accuracy with minimal parameters.
A lightweight network and NAS method improve ASC tasks.
problem Heavy computational burden in acoustic scene classification.
method Inspired by MobileNetV2, unidirectional convolutions; dynamic NAS with evolutionary algorithm.
result 90.3% F1-score on DCASE2018 task 5, 25% fewer FLOPs.
A novel framework combines LLMs and RL for financial portfolio optimization.
problem Optimizing financial portfolios using sentiment analysis and market indicators.
method Hierarchical RL structure with base, meta, and super-agents.
result Achieved a 26% annualized return and Sharpe ratio of 1.2.
Machine Reading Comprehension (MRC) is an important topic in the domain of automated question answering and in natural language processing more generally. Since the release of the SQuAD 1.1 and SQuAD 2 datasets, progress in the field has been particularly significant, with current state-of-the-art models now exhibiting…
Paper proposes GP-NAS-ensemble for fast neural architecture performance prediction.
problem Estimating neural network performance without training time-consuming evaluations.
method GP-NAS-ensemble framework using ensemble learning improvements.
result Ranked second in a NAS performance prediction challenge.
In this paper, we integrate VAEs and flow-based generative models successfully and get f-VAEs. Compared with VAEs, f-VAEs generate more vivid images, solved the blurred-image problem of VAEs. Compared with flow-based models such as Glow, f-VAE is more lightweight and converges faster, achieving the same performance und…
DeepDyve uses simpler neural networks to verify DNNs for faults.
problem Faults in DNNs, especially those not crafted by attackers.
method Pre-trained, simpler neural networks for dynamic verification.
result Reduces 90% of risks with 10% overhead.
A new data-level recombination strategy improves RGB-D salient object detection.
problem RGB-D salient object detection struggles with depth information.
method Proposes a novel data-level recombination strategy to fuse RGB and depth data before feature extraction.
result Achieves a new state-of-the-art performance in RGB-D salient object detection.
Deep neural networks and decision trees operate on largely separate paradigms; typically, the former performs representation learning with pre-specified architectures, while the latter is characterised by learning hierarchies over pre-specified features with data-driven architectures. We unite the two via adaptive neur…
VALAN is a framework for navigation agents in photo-realistic environments.
problem Developing agents for indoor navigation tasks.
method Deep reinforcement learning with SEED RL architecture.
result VALAN framework can solve a variety of RL problems.
We develop a progressive training approach for neural networks which adaptively grows the network structure by splitting existing neurons to multiple off-springs. By leveraging a functional steepest descent idea, we derive a simple criterion for deciding the best subset of neurons to split and a splitting gradient for …
A new framework for lightweight BNNs learns heteroscedastic uncertainties efficiently.
problem Learning heteroscedastic uncertainties from BNNs for lightweight networks.
method Embedding heteroscedastic variances into BNN parameters and using moment propagation for inference.
result Improves predictive performance for lightweight BNNs without increasing parameter count.
Neuro-inspired recurrent neural network algorithms, such as echo state networks, are computationally lightweight and thereby map well onto untethered devices. The baseline echo state network algorithms are shown to be efficient in solving small-scale spatio-temporal problems. However, they underperform for complex task…
Even though probabilistic treatments of neural networks have a long history, they have not found widespread use in practice. Sampling approaches are often too slow already for simple networks. The size of the inputs and the depth of typical CNN architectures in computer vision only compound this problem. Uncertainty in…
RSAC improves lightweight continuous learning efficiency.
problem Excessive training time and memory usage in continuous learning.
method Regularized subspace approximation classifier with feature reduction and regularization.
result RSAC achieves more efficient continuous learning than prior methods.
DLC enhances distillation-based continual learning with lightweight plugins.
problem Stability-plasticity dilemma in distillation-based continual learning.
method DLC deploys lightweight residual plugins into the classifier-proximal layer of a shared feature extractor.
result Significant 8% accuracy gain on large-scale benchmarks with minimal parameter increase.
iSTFTNet2 improves iSTFTNet's speed and lightness with 1D-2D CNN.
problem Efficiently synthesizing high-fidelity speech.
method Improved iSTFTNet using 1D-2D CNNs for temporal and spectrogram structures.
result iSTFTNet2 is faster and more lightweight with comparable speech quality.
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD): the variance in predicted probability of the correct class across iterations of …
Augment small datasets with synthetic backgrounds to train lightweight CNNs for human pose estimation.
problem Training CNNs from limited real-world data for human pose estimation.
method Synthetic background substitution for data augmentation.
result Improves generalization to unseen environments.
A new RNN architecture reduces model size and improves performance.
problem Overparameterization and resource limitations in RNNs.
method Jointly encodes weight matrices using tensor-train factorization.
result Reduces model size by several orders of magnitude without sacrificing performance.
MetaPerturb learns to improve generalization across different tasks and architectures.
problem Improving generalization on unseen data for diverse tasks and architectures.
method MetaPerturb is a meta-learned set-based perturbation function that improves generalization performance across heterogeneous tasks and architectures.
result MetaPerturb significantly outperforms baselines on most tasks and architectures with minimal increase in parameter size and no hyperparameters to tune.
CMoS improves time series forecasting with minimal parameters.
problem Efficiently forecasting time series data with limited resources.
method CMoS directly models chunk-wise spatial correlations, using Correlation Mixing and Periodicity Injection techniques.
result CMoS outperforms state-of-the-art models with minimal parameters.
Federated learning framework improves model generalization and privacy.
problem Communication overhead and statistical heterogeneity in FL.
method Prototypes and lightweight adapters for local model refinement.
result Improves classification accuracy over baseline algorithms.
When using reinforcement learning (RL) algorithms it is common, given a large state space, to introduce some form of approximation architecture for the value function (VF). The exact form of this architecture can have a significant effect on an agent's performance, however, and determining a suitable approximation arch…
The deployment of deep neural networks in real-world applications is mostly restricted by their high inference costs. Extensive efforts have been made to improve the accuracy with expert-designed or algorithm-searched architectures. However, the incremental improvement is typically achieved with increasingly more expen…
CogScale benchmarks AI architectures for sequential processing.
problem Evaluating AI architectures' ability to process sequential information efficiently.
method 14 scalable synthetic tasks designed to isolate cognitive and memory abilities at different scales.
result Attention mechanisms and modern state-space models consistently maintain high performance as task difficulty scales.
Learn to automatically plug domain-specific modules into a common network.
problem Learning inflexibility and computational intensiveness in multi-domain learning.
method Neural Architecture Search (NAS) for data-driven adapter plugging and structure design.
result NAS-driven MDL model achieves comparable performance to existing approaches.
Paper presents a lightweight, unobtrusive method to protect edge device data privacy.
problem Protecting inference data privacy in IoT edge devices with limited compute power.
method A lightweight neural network at edge devices to obfuscate inference data without indicating obfuscation.
result Effectively protects inference data confidentiality while preserving backend accuracy.
Winograd convolutions are used to improve quantized neural networks.
problem Improving quantized neural networks using Winograd convolutions.
method Proposed a Winograd-aware formulation of convolution layers to expose numerical inaccuracies to learning.
result Up to 10% higher classification accuracy on CIFAR-10 with Winograd-aware layers.
DecompKAN improves time series forecasting accuracy and transparency.
problem Accurate and transparent time series forecasting in scientific domains.
method Combines decomposition, patching, normalization, and B-spline KAN edge functions.
result Achieves best or tied-best MSE on 20 of 36 comparisons across 9 datasets.
Optimizes neural networks by removing unnecessary layers, improving performance and speed.
problem Finding the optimal depth of neural networks to improve performance and speed.
method Develops a fast end-to-end method for training lightweight neural networks with multiple classifier heads, allowing the model to determine the importance of each head and choosing a single shallow classifier.
result Significantly reduces the number of parameters and accelerates inference, outperforming many standard pruning methods.
We present a machine learning-based approach to lossy image compression which outperforms all existing codecs, while running in real-time. Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of generic images across all …
TinyXRA assesses financial risks from 10-K reports using a lightweight transformer model.
problem Comprehensive risk assessment from financial reports, distinguishing between upside and downside risk.
method Lightweight transformer model with dynamic attention, incorporating skewness, kurtosis, and Sortino ratio.
result State-of-the-art predictive accuracy and transparent risk assessments.
P-OCS detects OOD samples in a low-dimensional subspace, outperforming existing methods.
problem Efficient OOD detection for deep learning models in open-world environments.
method P-OCS operates in the orthogonal complement of the principal subspace, applying a single projected perturbation.
result P-OCS achieves state-of-the-art OOD detection with negligible computational cost and without requiring model retraining.
Coresets are compact representations of data sets such that models trained on a coreset are provably competitive with models trained on the full data set. As such, they have been successfully used to scale up clustering models to massive data sets. While existing approaches generally only allow for multiplicative appro…
When confronted with a substance of unknown identity, researchers often perform mass spectrometry on the sample and compare the observed spectrum to a library of previously-collected spectra to identify the molecule. While popular, this approach will fail to identify molecules that are not in the existing library. In r…
We introduce an adaptive output-sensitive Metropolis-Hastings algorithm for probabilistic models expressed as programs, Adaptive Lightweight Metropolis-Hastings (AdLMH). The algorithm extends Lightweight Metropolis-Hastings (LMH) by adjusting the probabilities of proposing random variables for modification to improve c…
New model mimics neural next item recommendation using Hankel matrices.
problem Next item recommendation efficiency and structural knowledge capture.
method Tensor factorization with Hankel matrix representation.
result Model performs competitively with neural networks but is simpler.
ALIEN improves uncertainty estimation of language models by refining entropy-based methods.
problem Overconfidence in uncertainty estimation for language models, especially for difficult inputs.
method ALIEN refines entropy-based uncertainty by aligning it with prediction reliability, using a lightweight uncertainty head.
result ALIEN consistently outperforms strong baselines in detecting incorrect predictions and achieving the lowest calibration error.
A lightweight framework improves convergence and stability of PINNs for complex PDEs.
problem Training instability and reduced accuracy in PINNs for complex PDEs.
method Adaptive curvature correction using secant information to optimize first-order optimizers.
result Consistent improvements in convergence speed, stability, and accuracy over standard optimizers.
Our goal is to design architectures that retain the groundbreaking performance of CNNs for landmark localization and at the same time are lightweight, compact and suitable for applications with limited computational resources. To this end, we make the following contributions: (a) we are the first to study the effect of…
Deploying trained convolutional neural networks (CNNs) to mobile devices is a challenging task because of the simultaneous requirements of the deployed model to be fast, lightweight and accurate. Designing and training a CNN architecture that does well on all three metrics is highly non-trivial and can be very time-con…
Paper presents a hybrid framework combining sentiment analysis and market indicators for financial portfolio optimization.
problem Improving financial portfolio optimization through better integration of sentiment and market data.
method A three-tier hierarchical RL framework integrating LLMs, DRL, and market data.
result Achieved a 26% annualized return and Sharpe ratio of 1.2, outperforming benchmarks.
Detect knots from photos using machine learning and traditional algorithms.
problem Automatically recognize knots from images.
method Combining CNN and transformer architectures for image recognition with traditional knot invariants.
result Lightweight machine learning models can recover meaningful structural information from images.