The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribut…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,982 papers · 148 categories
Trend · papers per month
48 results for “Importance Weighted Autoencoders”
New methods improve gradient estimation in autoencoders, enhancing generative network performance.
problem Improving gradient estimation in autoencoders to enhance learning.
method Developed and studied three methods: PIWAE, MIWAE, CIWAE.
result Generated approximate posterior distributions closer to true posterior distribution.
Improved neural spike inference from calcium imaging data.
problem Neural spike inference from calcium imaging data.
method Importance weighted adversarial variational autoencoders (IWAE) with adversarial training.
result Adversarial IWAE methods outperform VAEs in inferring neural spikes.
The variational autoencoder (VAE; Kingma, Welling (2014)) is a recently proposed generative model pairing a top-down generative network with a bottom-up recognition network which approximates posterior inference. It typically makes strong assumptions about posterior inference, for instance that the posterior distributi…
Hierarchical IWAE reduces sample redundancy for better inference.
problem Improving variational inference by reducing sample redundancy.
method Introduces a hierarchical structure to induce correlation among proposals.
result Maximizing the lower bound implicitly minimizes variance, improving inference performance.
Boltzmann machines are powerful distributions that have been shown to be an effective prior over binary latent variables in variational autoencoders (VAEs). However, previous methods for training discrete VAEs have used the evidence lower bound and not the tighter importance-weighted bound. We propose two approaches fo…
Deep neural network for cancer classification using autoencoders.
problem Cancer classification using molecular information.
method Using a Denoising Autoencoder (DAE) as weight initialization for a deep neural network, comparing two approaches: fixed weights and fine-tuning. Embedding strategies included encoding layers and complete autoencoder.
result Best F1 score of 98.04% for identifying thyroid cancer samples.
AISLE framework improves on IWAE by directly optimising proposal distribution.
problem IWAE's multi-sample objective leads to inference-network gradients that break down with increasing samples.
method Introduces AISLE framework, which optimises proposal distribution directly.
result AISLE admits IWAE-STL and IWAE-DREG as special cases, avoiding breakdown.
The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…
Study on dynamics of non-linear autoencoders learning principal components.
problem Technical difficulty in studying non-linear autoencoders due to non-trivial correlations.
method Derive asymptotically exact equations for SGD training of shallow, non-linear autoencoders.
result Autoencoders learn principal components sequentially and tie weights are ineffective.
Multi-sample, importance-weighted variational autoencoders (IWAE) give tighter bounds and more accurate uncertainty estimates than variational autoencoders (VAE) trained with a standard single-sample objective. However, IWAEs scale poorly: as the latent dimensionality grows, they require exponentially many samples to r…
The autoencoder is an effective unsupervised learning model which is widely used in deep learning. It is well known that an autoencoder with a single fully-connected hidden layer, a linear activation function and a squared error cost function trains weights that span the same subspace as the one spanned by the principa…
Bayesian methods improve OoD detection in deep networks.
problem Detecting Out-of-Distribution (OoD) inputs in deep neural networks.
method Three Bayesian inference approaches applied to VAE weights.
result Improved OoD detection scores over benchmarks.
Regularization improves stability and consistency of sparse autoencoders.
problem Varying features across random seeds and training choices in SAEs.
method Added L1 or L2 penalties on encoder and decoder weights.
result L2 regularization increases cross-seed feature consistency.
Variational autoencoders learn deep latent models.
problem Learning deep latent-variable models.
method Principled framework using variational inference.
result Introduction to variational autoencoders and extensions.
We provide a series of results for unsupervised learning with autoencoders. Specifically, we study shallow two-layer autoencoder architectures with shared weights. We focus on three generative models for data that are common in statistical machine learning: (i) the mixture-of-gaussians model, (ii) the sparse coding mod…
Beyond the Kolmogorov Barrier: A Learnable Weighted Hybrid Autoencoder for Model Order Reductioncs.LG
A new autoencoder combines deep learning with SVD to reduce model complexity.
problem Overcoming the Kolmogorov barrier in high-dimensional systems.
method Learnable weighted hybrid autoencoder combining SVD and deep learning.
result Empirically, the model exhibits a sharpness thousands of times smaller than other models.
Geometry and Generalization: Eigenvalues as predictors of where a network will fail to generalizecs.LG
Autoencoder performance is predicted by eigenvalues of weight matrices.
problem Predicting an autoencoder's generalization ability without dataset knowledge.
method Analyze Jacobian matrices' eigenvalues to bound mean squared errors.
result Eigenvalues are good predictors of MSE on test points.
Probabilistic Autoencoder learns latent space weights' distribution.
problem Nonlinear model reconstruction error and sample quality.
method Normalizing flow for latent space weights' probability distribution.
result PAE achieves small reconstruction errors, high sample quality, and good performance.
Improved autoencoders show joint training benefits over weak training.
problem Improving unsupervised learning performance with over-parameterized networks.
method Analyzing gradient dynamics of two-layer autoencoders with ReLU activation, proving linear convergence in weakly-trained and jointly-trained regimes.
result Joint training leads to better global optima and requires less over-parameterization.
A new modal autoencoder improves feature extraction by constraining the decoder.
problem Improving autoencoder performance through regularization.
method Proposed a modal autoencoder (MAE) by orthogonalizing the readout weight matrix.
result Extracted functionally independent features that perform better in classification tasks.
We extend Stochastic Gradient Variational Bayes to perform posterior inference for the weights of Stick-Breaking processes. This development allows us to define a Stick-Breaking Variational Autoencoder (SB-VAE), a Bayesian nonparametric version of the variational autoencoder that has a latent representation with stocha…
Deep model predicts wildfire data in incomplete regions.
problem Predicting distributions of wildfire frequency and size in missing data regions.
method Conditional Missing data Importance-Weighted Autoencoder (CMIWAE) model trained on incomplete data.
result Maximizes log-likelihood of observed wildfire information.
Feature extraction becomes increasingly important as data grows high dimensional. Autoencoder as a neural network based feature extraction method achieves great success in generating abstract features of high dimensional data. However, it fails to consider the relationships of data samples which may affect experimental…
Proposes a model for generating survival trajectories and data.
problem Generating survival trajectories and data for new feature vectors.
method Autoencoder with specific structure, incorporating weighting scheme for robustness.
result Generates a prototype time-dependent trajectory for objects.
We formulate learning of a binary autoencoder as a biconvex optimization problem which learns from the pairwise correlations between encoded and decoded bits. Among all possible algorithms that use this information, ours finds the autoencoder that reconstructs its inputs with worst-case optimal loss. The optimal decode…
The paper identifies five extreme learning regimes for large linear autoencoders.
problem Understanding the learning dynamics of large weight-tied linear autoencoders.
method Formal loss-expansion hierarchy and analysis of gradient flow.
result Five extreme regimes associated with faces of a triangular prism.
Improved IFA with Generative Adversarial Networks for high-dimensional latent variables.
problem Limited expressiveness of traditional VAEs in high-dimensional latent variable modeling.
method Introducing Adversarial Variational Bayes (AVB) and Importance-weighted Adversarial Variational Bayes (IWAVB) algorithms.
result IWAVB demonstrated superior expressiveness and higher likelihood compared to IWAE.
BAE uses boosting to improve autoencoder ensembles for robust outlier detection.
problem Overfitting in autoencoders limits their effectiveness in unsupervised outlier detection.
method Boosting-based Autoencoder Ensemble (BAE) trains autoencoders sequentially with weighted sampling to reduce outliers and inject diversity.
result BAE outperforms state-of-the-art approaches in various outlier detection conditions.
Autoencoder Treescs.LG
We discuss an autoencoder model in which the encoding and decoding functions are implemented by decision trees. We use the soft decision tree where internal nodes realize soft multivariate splits given by a gating function and the overall output is the average of all leaves weighted by the gating values on their path. …
Study on autoencoder denoising in high dimensions.
problem Denoising data from Gaussian mixtures.
method Two-layer non-linear autoencoder with skip connection in high-dimensional limit.
result Closed-form expressions for denoising mean-squared test error.
SoftAdapt: Techniques for Adaptive Loss Weighting of Neural Networks with Multi-Part Loss Functionscs.LG
SoftAdapt dynamically adjusts loss weights for multi-part functions.
problem Slow convergence and poor weight selection for multi-part loss functions.
method SoftAdapt dynamically changes weights based on live performance statistics.
result Improved convergence and better weight selection for multi-part loss functions.
Binary autoencoder with sparse hidden layer preserves information and zero reconstruction error.
problem Preserving information and zero reconstruction error in binary neural networks.
method Binary autoencoder with random binary weights, sparse hidden layer, and varying neuron thresholds.
result Zero reconstruction error for any input with a large hidden layer and varying neuron thresholds.
Alternating optimization method based on nonnegative matrix factorizations for deep neural networkscs.LG
The backpropagation algorithm for calculating gradients has been widely used in computation of weights for deep neural networks (DNNs). This method requires derivatives of objective functions and has some difficulties finding appropriate parameters such as learning rate. In this paper, we propose a novel approach for c…
DGPs with variational inference suffer from SNR issues that degrade gradient estimates, leading to unreliable training.
problem SNR issues in gradient estimates for DGPs with variational inference.
method Adapted doubly reparameterized gradient estimators for DGP training.
result Fix improves predictive performance of DGP models.
Generative models simulate S&P 500 returns for financial analysis.
problem Modeling the joint distribution of S&P 500 equities.
method Conditional importance weighted autoencoders and conditional normalizing flows.
result Generative models accurately capture the complex joint distribution of S&P 500 returns.
Proposes a neural network autoencoder for smoothing and representation learning of functional data.
problem Lack of sufficient nonlinear representations in existing methods for functional data analysis.
method Develops a neural network autoencoder architecture to process functional data directly, learning both smoothing and representation.
result Outperforms traditional methods in prediction, classification, and computational efficiency.
We demonstrate a new deep learning autoencoder network, trained by a nonnegativity constraint algorithm (NCAE), that learns features which show part-based representation of data. The learning algorithm is based on constraining negative weights. The performance of the algorithm is assessed based on decomposing data into…
This work analyzes autoencoders to understand feature learning, revealing different phases of learning.
problem Understanding the feature extraction mechanism in weight-tied autoencoders.
method Mean field analysis of two-layer weight-tied autoencoders trained with stochastic gradient descent.
result The training dynamics of autoencoders exhibit different phases corresponding to learning different principal subspaces of the data.
New method disentangles shared and private latent factors in multimodal data.
problem Challenges in disentangling shared and private latent factors in multimodal data.
method Proposes a modification to existing multimodal Variational Autoencoders (MMVAE) to better handle modality-specific variation.
result Demonstrates improved robustness of modified MMVAE to modality-specific variation.
Simpler method for separating and manipulating latent attributes in autoencoders.
problem Separating and manipulating latent attributes in autoencoders.
method Matrix subspace projection
result Our method allows for changing selected attributes while preserving other information.
Bayesian autoencoders quantify anomaly uncertainty for safer machine learning.
problem Lack of uncertainty quantification in autoencoders for anomaly detection.
method Formulated Bayesian autoencoders to quantify epistemic and aleatoric anomalies.
result Demonstrated effectiveness of BAEs on benchmark and real datasets.
SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selectionq-fin.CP
SAE-FiRE extracts key financial info from long documents, improving earnings surprise predictions.
problem Predicting earnings surprises from long, redundant financial documents.
method Sparse Autoencoder feature selection to filter out noise and identify key dimensions.
result SAE-FiRE significantly outperforms baseline approaches in financial datasets.
AI learns to classify and represent univariate distributions in a 2D latent space.
problem Classifying and representing univariate empirical distributions.
method Unsupervised beta variational autoencoder (beta-VAE) to separate and represent distributions in a 2D latent space.
result The latent space representation separates distributions of different shapes while overlapping similar ones.
A deep learning method for XML with autoencoder and ranking loss.
problem XML with large label collections, high complexity, inter-label and feature dependencies, and noisy labels.
method Word-vector-based self-attention, ranking-based AutoEncoder architecture.
result Competitive performance on benchmark datasets.
Machine learning factors outperform traditional portfolio optimization methods.
problem Comparing machine learning and traditional portfolio optimization methods.
method Examined machine learning and factor-based portfolio optimization using autoencoder neural networks and dimensionality reduction techniques.
result Minimum-variance portfolios using latent factors derived from autoencoders and sparse methods outperform simpler benchmarks in risk minimization.
AnomalyDAE detects anomalies in networks by learning cross-modality interactions.
problem Detecting anomalies in attributed networks where structure and attributes interact.
method Dual autoencoder framework with attention mechanism for joint learning of structure and attribute embeddings.
result AnomalyDAE effectively detects anomalies by reconstructing node attributes and structures.
A hybrid framework prices options using neural networks and VAE latent space.
problem Lack of explicit asset dynamics information in compressed volatility surfaces.
method Combining Weighted Monte Carlo with neural networks trained on VAE latent space.
result Effective pricing of vanilla and exotic options on idealized vol surface.