We present a transductive deep learning-based formulation for the sparse representation-based classification (SRC) method. The proposed network consists of a convolutional autoencoder along with a fully-connected layer. The role of the autoencoder network is to learn robust deep features for classification. On the othe…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Deep Sparse Autoencoders”
We demonstrate a new deep learning autoencoder network, trained by a nonnegativity constraint algorithm (NCAE), that learns features which show part-based representation of data. The learning algorithm is based on constraining negative weights. The performance of the algorithm is assessed based on decomposing data into…
Increasing volume of Electronic Health Records (EHR) in recent years provides great opportunities for data scientists to collaborate on different aspects of healthcare research by applying advanced analytics to these EHR clinical data. A key requirement however is obtaining meaningful insights from high dimensional, sp…
Increasing volume of Electronic Health Records (EHR) in recent years provides great opportunities for data scientists to collaborate on different aspects of healthcare research by applying advanced analytics to these EHR clinical data. A key requirement however is obtaining meaningful insights from high dimensional, sp…
This work combines deep learning and sparse coding for CT image reconstruction.
problem Improving image quality in low-dose CT scans.
method Sparse signal representation using learned dictionaries, inspired by variational autoencoders and deep learning techniques.
result Regularization with learned dictionaries achieves competitive performance in CT reconstruction.
Develops SGP-VAE for efficient sparse GP inference in multi-dimensional datasets.
problem Sparse GP approximations and missing data in multi-dimensional spatio-temporal datasets.
method Leverages partial inference networks for sparse GP approximations and amortized variational inference.
result Outperforms multi-output GPs and structured VAEs in various experiments.
Evidential Softmax preserves multimodality in sparse probability distributions for generative models.
problem Sparse probability distributions in deep generative models make exact marginalization computationally intractable.
method Introduce ev-softmax, a sparse normalization function that preserves multimodality and can be trained with probabilistic loss functions.
result ev-softmax outperforms existing techniques in distributional accuracy and dimensionality reduction.
Proposes a Bayesian Autoencoder with sparse Gaussian process priors to capture data correlations.
problem Autoencoders' i.i.d. assumption of latent representations fails to capture data correlations.
method Imposes fully Bayesian sparse Gaussian Process priors on the latent space of a Bayesian Autoencoder and uses stochastic gradient Hamiltonian Monte Carlo for posterior estimation.
result Consistently outperforms alternatives relying on Variational Autoencoders on various tasks.
Algorithm learns latent variables for thermodynamically-consistent deep neural networks.
problem Predicting time evolution of large-scale physical systems with thermodynamic consistency.
method Sparse autoencoders and structure-preserving neural networks.
result Method conserves total energy and entropy inequality for both conservative and dissipative systems.
This paper proposes a novel model for the rating prediction task in recommender systems which significantly outperforms previous state-of-the art models on a time-split Netflix data set. Our model is based on deep autoencoder with 6 layers and is trained end-to-end without any layer-wise pre-training. We empirically de…
DSAEE FS selects features for imbalanced data.
problem Feature selection for imbalanced data.
method DSAEE ensemble for feature ranking based on reconstruction error.
result DSAEE FS outperforms other methods in selecting relevant features.
Novel deep learning method predicts reaction coordinates and future MD trajectories.
problem Identifying optimal reaction coordinates for chemical reactions.
method Regularized Sparse Autoencoder (RSE) for discovering reaction coordinates and predicting MD trajectory evolution.
result RSE helps in choosing a small but important set of reaction coordinates.
Deep learning models reconstruct volatility surfaces from noisy data under no-arbitrage constraints.
problem Reconstructing implied volatility surfaces from sparse and noisy option quotes.
method Compared multiple neural architectures including Transformers, U-Nets, and variational autoencoders.
result Transformer and U-Net architectures achieve strong reconstruction accuracy, especially under sparse observation regimes.
New framework improves generative models with prediction and consistency constraints.
problem Improving generative models with sparse labeled data.
method Optimizes variational autoencoders with prediction and consistency constraints.
result Promising image classification performance, especially in semi-supervised scenarios.
Autoencoders have been successful in learning meaningful representations from image datasets. However, their performance on text datasets has not been widely studied. Traditional autoencoders tend to learn possibly trivial representations of text documents due to their confounding properties such as high-dimensionality…
Study analyzes feedback complexity for sparse feature retrieval in deep networks.
problem Learning sparse superposed features with feedback.
method Analysis of feedback complexity in sparse settings, including triplet comparisons.
result Establishes tight bounds and strong upper bounds for feature recovery.
Combining simple elements from the literature, we define a linear model that is geared toward sparse data, in particular implicit feedback data for recommender systems. We show that its training objective has a closed-form solution, and discuss the resulting conceptual insights. Surprisingly, this simple model achieves…
Generates high-quality images using sparse DCT representations.
problem Challenges in generating images due to high dimensionality.
method Transformers trained on sparse DCT block sequences.
result Competitive image generation quality with state-of-the-art methods.
Sparse code multiple access (SCMA) has been one of non-orthogonal multiple access (NOMA) schemes aiming to support high spectral efficiency and ubiquitous access requirements for 5G wireless communication networks. Conventional SCMA approaches are confronting remarkable challenges in designing low complexity high accur…
Improved SINDy autoencoder for identifying noisy dynamical systems.
problem Robust identification of noisy dynamical systems from data.
method Incorporates noise-separating neural network structures into SINDy autoencoder architecture.
result Accurately recovers latent dynamics and estimates measurement noise from noisy observations.
Deep dynamic modeling with just two time points: Can we still allow for individual trajectories?stat.ML
Deep learning models can infer individual trajectories from sparse data.
problem Learning individual dynamics from limited data points.
method Combining variational autoencoders (VAEs) with ordinary differential equations (ODEs) for dynamic modeling.
result Deep learning can recover individual trajectories from sparse data, but requires careful adaptation.
New method learns sparse distributions by thresholding samples, improving performance and efficiency.
problem Sparse coding optimization in high-dimensional problems is computationally expensive and inefficient.
method Proposes a new variational sparse coding approach that learns sparse distributions by thresholding samples.
result Shows superior performance, statistical efficiency, and gradient estimation compared to other sparse distributions.
We propose novel deep learning based chemometric data analysis technique. We trained L2 regularized sparse autoencoder end-to-end for reducing the size of the feature vector to handle the classic problem of the curse of dimensionality in chemometric data analysis. We introduce a novel technique of automatic selection o…
Enhances FAVAR models with autoencoder for better economic forecasting and interpretability.
problem Limitations of linear FAVAR models in forecasting and structural analysis.
method Introduces Grouped Sparse autoencoder with time-varying parameters.
result The Grouped Sparse autoencoder produces more interpretable factors and superior forecasting performance.
AEN-SAEs address feature starvation in sparse autoencoders by stabilizing the geometric alignment of sparse coding.
problem Feature starvation in sparse autoencoders, leading to unstable and misaligned representations.
method Adaptive Elastic Net SAEs (AEN-SAEs) combine and terms to stabilize the sparse coding map and control feature interactions.
result AEN-SAEs mitigate feature starvation without heuristic resampling, maintaining competitive reconstruction abilities.
SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selectionq-fin.CP
SAE-FiRE extracts key financial info from long documents, improving earnings surprise predictions.
problem Predicting earnings surprises from long, redundant financial documents.
method Sparse Autoencoder feature selection to filter out noise and identify key dimensions.
result SAE-FiRE significantly outperforms baseline approaches in financial datasets.
Binary autoencoder with sparse hidden layer preserves information and zero reconstruction error.
problem Preserving information and zero reconstruction error in binary neural networks.
method Binary autoencoder with random binary weights, sparse hidden layer, and varying neuron thresholds.
result Zero reconstruction error for any input with a large hidden layer and varying neuron thresholds.
Autoencoders fail to capture sparse structure in 1-bit data compression.
problem Proving the performance of shallow autoencoders on sparse data compression.
method Gradient descent analysis and approximate message passing.
result Gradient descent minimizer for sparse data is the identity (up to permutation) above critical sparsity.
This paper introduces structure learning for autoencoder recommenders to improve performance and generalization.
problem Efficient training and generalization in sparse collaborative filtering data.
method Learn groups of related items and use this information to determine the connectivity structure of an auto-encoding neural network.
result The proposed structure learning method results in a sparse network that converges to a local optimum with smaller spectral norm and generalization error.
Regularization improves stability and consistency of sparse autoencoders.
problem Varying features across random seeds and training choices in SAEs.
method Added L1 or L2 penalties on encoder and decoder weights.
result L2 regularization increases cross-seed feature consistency.
It has recently been observed that certain extremely simple feature encoding techniques are able to achieve state of the art performance on several standard image classification benchmarks including deep belief networks, convolutional nets, factored RBMs, mcRBMs, convolutional RBMs, sparse autoencoders and several othe…
We introduce a neural-network architecture, termed the constrained recurrent sparse autoencoder (CRsAE), that solves convolutional dictionary learning problems, thus establishing a link between dictionary learning and neural networks. Specifically, we leverage the interpretation of the alternating-minimization algorith…
A new method selects features efficiently for high-dimensional data.
problem High computational costs and memory requirements in high-dimensional data.
method QuickSelection uses the strength of neurons in sparse autoencoders to select features.
result QuickSelection achieves the best trade-off of accuracy, speed, and memory usage.
Deep Autoencoder outperforms in anomaly detection for building energy data.
problem Automated detection of faulty data in learning applications.
method Training and comparison of Simple, Deep, and Supervised Deep Autoencoders on ASHRAE building energy dataset.
result Supervised Deep Autoencoder outperforms in total anomalies detected.
ALF reduces network parameters and operations by 70% and 61%, respectively, on embedded hardware.
problem Efficient deployment of deep learning models on resource-constrained hardware.
method Autoencoder-based low-rank filter-sharing technique.
result ALF achieves significant compression with minimal accuracy loss.
We propose rectified factor networks (RFNs) to efficiently construct very sparse, non-linear, high-dimensional representations of the input. RFN models identify rare and small events in the input, have a low interference between code units, have a small reconstruction error, and explain the data covariance structure. R…
New GP-VAE model improves scalability and performance.
problem Inability of conventional VAEs to model correlations between data points.
method Principled sparse inference approaches to improve scalability of GP-VAEs.
result New model outperforms existing approaches in runtime and memory usage.
In this paper we propose a Deep Autoencoder MIxture Clustering (DAMIC) algorithm based on a mixture of deep autoencoders where each cluster is represented by an autoencoder. A clustering network transforms the data into another space and then selects one of the clusters. Next, the autoencoder associated with this clust…
In this paper, we develop a new framework for sensing and recovering structured signals. In contrast to compressive sensing (CS) systems that employ linear measurements, sparse representations, and computationally complex convex/greedy algorithms, we introduce a deep learning framework that supports both linear and mil…
Feature extraction becomes increasingly important as data grows high dimensional. Autoencoder as a neural network based feature extraction method achieves great success in generating abstract features of high dimensional data. However, it fails to consider the relationships of data samples which may affect experimental…
Parameterized mathematical models play a central role in understanding and design of complex information systems. However, they often cannot take into account the intricate interactions innate to such systems. On the contrary, purely data-driven approaches do not need explicit mathematical models for data generation an…
A new deep clustering model learns both clustering and embedding simultaneously.
problem Optimizing autoencoders for clustering and embedding separately.
method Integrating clustering module into a deep autoencoder.
result Joint learning of clustering and embedding improves performance.
BatchTopK SAEs improve GPT-2 and Gemma activations with adjustable sparsity.
problem Interpreting language model activations using sparse autoencoders.
method Adapting the TopK constraint to the batch-level, allowing variable number of active latents per sample.
result BatchTopK SAEs consistently outperform TopK SAEs in reconstructing GPT-2 and Gemma activations.
New neural operators learn structured patterns efficiently.
problem Learning and representing complex, structured patterns in data.
method Sparse autoencoder neural operators (SAE-NOs) parameterize concepts as functions, enabling efficient and structured representation.
result SAE-FNOs learn localized patterns and generalize across different scales and discretizations.
Bayesian autoencoders discover physics from noisy data.
problem Challenges in identifying governing equations and coordinates from noisy, low-data real-world data.
method Bayesian SINDy autoencoders with hierarchical Bayesian sparsifying prior and adaptive empirical Bayesian method.
result Better physics discovery with lower data and fewer training epochs, along with valid uncertainty quantification.
Detecting changes in asset co-movements is of much importance to financial practitioners, with numerous risk management benefits arising from the timely detection of breakdowns in historical correlations. In this article, we propose a real-time indicator to detect temporary increases in asset co-movements, the Autoenco…
Automated anomaly detection is essential for managing information and communications technology (ICT) systems to maintain reliable services with minimum burden on operators. For detecting varying and continually emerging anomalies as differences from normal states, learning normal relationships inherent among cross-dom…
Deep learning using multi-layer neural networks (NNs) architecture manifests superb power in modern machine learning systems. The trained Deep Neural Networks (DNNs) are typically large. The question we would like to address is whether it is possible to simplify the NN during training process to achieve a reasonable pe…