RCAM-based ensemble combines binary classifiers using similarity and vote scheme.
problem Improving binary classification accuracy through ensemble methods.
method RCAM-based ensemble combining classifiers using similarity and recurrent consult-vote scheme.
result RCAM-based ensemble outperforms individual classifiers and majority voting.
A new method reduces memory usage for deep neural networks ensembles.
problem Reducing memory footprint of ensemble methods for deep learning.
method Extract multiple sub-networks from a single neural network through end-to-end optimization.
result Achieves higher or comparable accuracy with significantly less memory usage.
This work investigates power laws in deep neural network ensembles and predicts their performance.
problem Understanding the performance of deep neural network ensembles and their optimal structure.
method Investigated the behavior of negative log-likelihood (CNLL) of a deep ensemble as a function of ensemble size and member network size, identifying power law dependencies.
result One large network may perform worse than an ensemble of several medium-size networks, known as a memory split.
Memory split advantage: thinner networks outperform a single wide network.
problem Optimizing deep learning models with limited memory.
method Investigated training a single wide network vs. an ensemble of thinner networks with the same total number of parameters.
result An ensemble of several thinner networks outperforms a single wide network for large memory budgets.
Ensembles, where multiple neural networks are trained individually and their predictions are averaged, have been shown to be widely successful for improving both the accuracy and predictive uncertainty of single neural networks. However, an ensemble's cost for both training and testing increases linearly with the numbe…
Binarization is an attractive strategy for implementing lightweight Deep Convolutional Neural Networks (CNNs). Despite the unquestionable savings offered, memory footprint above all, it may induce an excessive accuracy loss that prevents a widespread use. This work elaborates on this aspect introducing TentacleNet, a n…
EnLSTM network improves log generation from small datasets.
problem Generating well logs from small datasets with high accuracy.
method Combining ENN and C-LSTM networks with perturbation methods.
result 34% reduction in mean-square-error compared to existing models.
We study dynamical behavior of the Chinese stock markets by investigating the statistical properties of daily ensemble returns and varieties defined respectively as the mean and the standard deviation of the ensemble daily price returns of a portfolio of stocks traded in China's stock markets on a given day. The distri…
Ensembles of models have been empirically shown to improve predictive performance and to yield robust measures of uncertainty. However, they are expensive in computation and memory. Therefore, recent research has focused on distilling ensembles into a single compact model, reducing the computational and memory burden o…
N-BEATS(P) efficiently forecasts millions of time series with reduced memory and time.
problem Efficiently forecasting millions of time series with high accuracy.
method Global parallel variant of N-BEATS model designed for multi-step time series forecasting.
result Significant reduction in training time and memory usage with comparable accuracy.
Deep neural networks are vulnerable to adversarial attacks.
Quantum algorithm improves ensemble classification with reduced memory and time requirements.
problem High memory and computational time requirements in ensemble methods.
method Quantum superposition, entanglement, and interference to build an ensemble of classification models.
result Exponential growth of ensemble size with linear increase in depth of circuit.
Paper uses AI methods to forecast Bitcoin prices.
problem Inaccurate Bitcoin price predictions in previous studies.
method Combines EEMD and LSTM for next-day price forecast.
result Improves Bitcoin price prediction accuracy.
HD algorithm simulates dynamics on random matrix ensembles without generating full matrices.
problem Simulating dynamics on dense random matrix ensembles with high space and time complexity.
method Householder reflectors for adaptive and recursive construction, deferring decisions.
result Significant reductions in runtime and memory footprint for practical T≪n. The study shows how modular learning can adapt to new tasks.
problem Adapting to new tasks in an ever-changing environment.
method Task segmentation, modular learning, memory-based ensembling.
result The system demonstrates robustness to catastrophic forgetting and increasing positive transfer.
ForestPrune optimizes tree ensemble pruning for compactness and speed.
problem Large tree ensembles in predictive models consume excessive memory and reduce interpretability.
method Developed a specialized optimization algorithm to efficiently prune tree ensembles by depth layers.
result ForestPrune produces compact, high-performing models that outperform existing post-processing methods.
New hyperparameter ensembles boost neural network performance and uncertainty.
problem Improving neural network robustness and uncertainty quantification.
method Designing ensembles over both weights and hyperparameters, stratified across random initializations.
result Hyper-deep and hyper-batch ensembles outperform deep and batch ensembles on various architectures.
Embedded ensembles improve neural network performance efficiently.
problem Improving neural network performance with fewer resources.
method Analyzing the wide network limit of gradient descent dynamics using Neural-Tangent-Kernel.
result Embedded ensembles exhibit two regimes: independent and collective, affecting performance.
Q-Ensembles are a model-free approach where input images are fed into different Q-networks and exploration is driven by the assumption that uncertainty is proportional to the variance of the output Q-values obtained. They have been shown to perform relatively well compared to other exploration strategies. Further, mode…
Packed-Ensembles improve uncertainty estimation in constrained hardware.
problem Hardware limitations restrict the size of ensembles and network capacity, degrading performance.
method Packed-Ensembles (PE) design and train lightweight structured ensembles by modulating encoding space and parallelizing into a single backbone.
result PE accurately preserves diversity and maintains performance on key metrics like accuracy, calibration, and out-of-distribution detection.
Long Short-Term Memory networks trained with gradient descent and back-propagation have received great success in various applications. However, point estimation of the weights of the networks is prone to over-fitting problems and lacks important uncertainty information associated with the estimation. However, exact Ba…
A method to reduce memory usage in deep learning models by adding inducing weights.
problem Memory inefficiency in Bayesian neural networks and deep ensembles.
method Augmenting the weight matrix with inducing weights and using Matheron's conditional Gaussian sampling rule.
result Reduces parameter size to 24.3% of a single neural network while maintaining competitive performance.
Efficient neural network ensembles improve image classification reliability and uncertainty quantification.
problem Uncertainty in neural network predictions for industrial image classification.
method Investigated efficient neural network ensembles (snapshot, batch, multi-input multi-output) for image classification reliability and uncertainty quantification.
result Batch ensemble is a cost-effective and competitive alternative to deep ensembles, offering savings in training and test time.
LSTM neural networks improve stock price prediction for Stockholm OMX30.
problem Forecasting stock price movement in financial markets.
method Ensemble of parallel long short-term memory (LSTM) neural networks trained on binary classification of stock returns.
result The LSTM ensemble outperforms traditional portfolios in terms of average daily returns, cumulative returns, and risk-adjusted performance.
SliceOut speeds up deep learning training without sacrificing accuracy.
problem Frequent model re-training and large model training workloads in deep learning.
method SliceOut uses dropout-inspired scheme to drop contiguous sets of units at random, leveraging GPU memory layout.
result 10-40% speedups and memory reduction with minimal accuracy loss.
Ensembles of models often yield improvements in system performance. These ensemble approaches have also been empirically shown to yield robust measures of uncertainty, and are capable of distinguishing between different \emph{forms} of uncertainty. However, ensembles come at a computational and memory cost which may be…
Efficiently builds diverse sub-model ensembles for robust self-supervised learning.
problem Challenges in diversity and efficiency of deep ensembles for self-supervised representation learning.
method Ensemble of independent sub-networks with a new loss function for diversity.
result Significantly improves prediction reliability and model calibration.
In this paper, we solve a semi-supervised regression problem. Due to the lack of knowledge about the data structure and the presence of random noise, the considered data model is uncertain. We propose a method which combines graph Laplacian regularization and cluster ensemble methodologies. The co-association matrix of…
This paper studies recursive ensembles driven by Fibonacci updates, improving learning dynamics.
problem Improving learning dynamics in recursive ensemble learning.
method Develops second-order recursive architectures with Fibonacci-type update flows.
result Establishes global convergence conditions and generalization bounds for recursive ensembles.
Data-driven predictive analytics are in use today across a number of industrial applications, but further integration is hindered by the requirement of similarity among model training and test data distributions. This paper addresses the need of learning from possibly nonstationary data streams, or under concept drift,…
The number of component classifiers chosen for an ensemble greatly impacts the prediction ability. In this paper, we use a geometric framework for a priori determining the ensemble size, which is applicable to most of existing batch and online ensemble classifiers. There are only a limited number of studies on the ense…
Study characterizes memory capacity of quantum reservoirs using transmon qubits.
problem Understanding the memory capacity of quantum reservoirs built with transmon qubits.
method Characterized memory capacity of quantum reservoirs using transmon qubits from IBM, focusing on NMSE and topology complexity.
result Found a peak in memory capacity for configurations with n-1 self-loops, suggesting optimal design for forecasting tasks.
The stock market prediction has always been crucial for stakeholders, traders and investors. We developed an ensemble Long Short Term Memory (LSTM) model that includes two-time frequencies (annual and daily parameters) in order to predict the next-day Closing price (one step ahead). Based on a four-step approach, this …
Deep RL ensemble strategy outperforms individual algorithms in stock trading.
problem Designing profitable stock trading strategies in a complex market.
method Ensemble of three deep reinforcement learning algorithms (PPO, A2C, DDPG) for stock trading.
result Deep ensemble strategy outperforms individual algorithms and traditional min-variance portfolio.
Paper proposes a deep learning model to predict stock prices using sentiment analysis.
problem Predicting future stock movement using financial textual and numerical data.
method A blending ensemble deep learning model with two levels of RNNs, LSTM, and GRU followed by a fully connected neural network.
result The model improves prediction accuracy compared to traditional methods.
An ensemble of neural networks is known to be more robust and accurate than an individual network, however usually with linearly-increased cost in both training and testing. In this work, we propose a two-stage method to learn Sparse Structured Ensembles (SSEs) for neural networks. In the first stage, we run SG-MCMC wi…
GDML learns effective CG models from all-atom data.
problem Learning effective coarse-grained force fields efficiently.
method Ensemble learning with stratified sampling and GDML.
result GDML yields smaller free energy error than neural networks.
Combines Laplace approximations of deep networks for better uncertainty quantification.
problem Overconfident predictions on outliers in deep learning models.
method Gaussian mixture model posterior using weighted sum of Laplace approximations of pre-trained deep networks.
result Mitigates overconfidence 'far away' from training data.
Recent studies have revealed the vulnerability of deep neural networks: A small adversarial perturbation that is imperceptible to human can easily make a well-trained deep neural network misclassify. This makes it unsafe to apply neural networks in security-critical applications. In this paper, we propose a new defense…
We introduce the C++ application and R package ranger. The software is a fast implementation of random forests for high dimensional data. Ensembles of classification, regression and survival trees are supported. We describe the implementation, provide examples, validate the package with a reference implementation, and …
New distributed EnKF method for non-sequential assimilation of large datasets.
problem Computational intensity and order dependencies in traditional EnKF.
method Distributed computing for full model error covariance matrix.
result Non-sequential assimilation outperforms sequential in performance.
Binary neural networks (BNN) have been studied extensively since they run dramatically faster at lower memory and power consumption than floating-point networks, thanks to the efficiency of bit operations. However, contemporary BNNs whose weights and activations are both single bits suffer from severe accuracy degradat…
Deep learning models predict mutual funds' performance better than traditional methods.
problem Predicting mutual funds' performance accurately.
method Deep learning models (LSTM, GRUs) trained with Bayesian optimization and ensemble methods.
result Ensemble method of LSTM and GRUs achieves the highest accuracy in forecasting mutual funds' Sharpe ratios.
Resource-efficient oblique trees reduce neural signal classification costs.
problem Implementing efficient neural signal classifiers on resource-constrained devices.
method Integrating model compression, probabilistic routing, and cost-aware learning.
result Significant reduction in model size and feature extraction cost compared to state-of-the-art models.
The dynamic ensemble selection of classifiers is an effective approach for processing label-imbalanced data classifications. However, such a technique is prone to overfitting, owing to the lack of regularization methods and the dependence of the aforementioned technique on local geometry. In this study, focusing on bin…
Deep learning ensembles improve COVID-19 detection from chest X-rays.
problem Detecting COVID-19 from chest X-rays using machine learning.
method Custom CNN and ImageNet models, transfer learning, iterative pruning, ensemble learning.
result 99.01% accuracy in detecting COVID-19 from chest X-rays.
Paper proposes novel hedging strategies using LSTM models for diversified investment portfolios.
problem Hedging risky asset portfolios in turbulent financial markets.
method Four diverse models (LSTM, ARIMA-GARCH, momentum, contrarian) generate price forecasts for diversified AIS.
result LSTM-based strategies outperform other models, with Bitcoin being the best diversifier for S&P 500 index.
The Analog Ensemble (AnEn) method tries to estimate the probability distribution of the future state of the atmosphere with a set of past observations that correspond to the best analogs of a deterministic Numerical Weather Prediction (NWP). This model post-processing method has been successfully used to improve the fo…