Develops VAEs for learning complex physical systems from data.
problem Learning low-dimensional representations of nonlinear physical systems.
method Variational Autoencoders with manifold latent spaces.
result Effective in learning nonlinear Burgers equation and constrained mechanical systems.
We solve a high-dimensional model where nonlinear autoencoders detect hidden structure missed by PCA.
problem Hidden structure in high-dimensional data not detected by PCA.
method Tractable spiked model with two latent factors, one visible and one uncorrelated.
result Nonlinear autoencoders can extract hidden structure missed by PCA, even if reconstruction loss is higher.
Novel autoencoder method approximates Koopman operator in low dimensions.
problem Challenges in approximating finite Koopman operators using data-driven methods.
method Mori-Zwanzig autoencoder (MZ-AE) for robust Koopman operator approximation.
result Improved predictive capability and robust long-term statistical performance.
GD-VAEs learn dynamics from observations using geometric and topological information.
problem Learning parsimonious representations of nonlinear dynamics from observations.
method Develops data-driven methods incorporating geometric and topological information using Variational Autoencoders (VAEs).
result GD-VAEs provide methods for learning reduced dimensional representations of nonlinear dynamics.
Proposes LAE-EnKF for improved nonlinear data assimilation.
problem Performance of EnKF deteriorates for strongly nonlinear dynamics.
method Reformulates assimilation in a learned latent space with linear dynamics.
result LAE-EnKF yields more accurate and stable assimilation.
This work analyzes autoencoders to understand feature learning, revealing different phases of learning.
problem Understanding the feature extraction mechanism in weight-tied autoencoders.
method Mean field analysis of two-layer weight-tied autoencoders trained with stochastic gradient descent.
result The training dynamics of autoencoders exhibit different phases corresponding to learning different principal subspaces of the data.
New method identifies latent components in nonlinear mixtures without stringent assumptions.
problem Unraveling latent components in nonlinearly mixed data.
method Constrained autoencoder-based algorithm for identifiability under relaxed assumptions.
result Comprehensive sample complexity results and new identifiability conditions.
Gene expression profiles have been widely used to characterize patterns of cellular responses to diseases. As data becomes available, scalable learning toolkits become essential to processing large datasets using deep learning models to model complex biological processes. We present an autoencoder to capture nonlinear …
Enhances FAVAR models with autoencoder for better economic forecasting and interpretability.
problem Limitations of linear FAVAR models in forecasting and structural analysis.
method Introduces Grouped Sparse autoencoder with time-varying parameters.
result The Grouped Sparse autoencoder produces more interpretable factors and superior forecasting performance.
In this paper we propose a Deep Autoencoder MIxture Clustering (DAMIC) algorithm based on a mixture of deep autoencoders where each cluster is represented by an autoencoder. A clustering network transforms the data into another space and then selects one of the clusters. Next, the autoencoder associated with this clust…
Proposes CoDEAL for estimating heterogeneous treatment effects in panel data models.
problem Estimating heterogeneous treatment effects in causal panel data models with covariate effects.
method Covariate-Adjusted Deep Causal Learning (CoDEAL) integrating neural networks and autoencoders.
result Establishes theoretical guarantees and demonstrates compelling performance in simulations and real data.
A new method for separating mixed signals in space and time.
problem Nonlinear and nonstationary spatio-temporal data challenges.
method Identifiable autoregressive variational autoencoder.
result The method outperforms existing techniques in blind source separation and spatio-temporal prediction.
A new model for multiview data analysis using graph autoencoders.
problem Nonlinear multiview canonical correlation analysis for large datasets.
method Variational approach with graph convolutional neural networks.
result Competitive performance on classification, clustering, and recommendation tasks.
Unsupervised clustering is one of the most fundamental challenges in machine learning. A popular hypothesis is that data are generated from a union of low-dimensional nonlinear manifolds; thus an approach to clustering is identifying and separating these manifolds. In this paper, we present a novel approach to solve th…
Paper uses autoencoders for efficient reduced-order modeling of eigenvalue problems.
problem Efficiently modeling eigenvalue problems in high dimensions.
method Autoencoder-based reduced-order modeling for eigenvalue problems.
result Autoencoder-based models outperform standard POD-Galerkin methods in neutron diffusion applications.
New method learns nonlinear projections for reduced-order modeling of complex dynamical systems.
problem Modeling transient dynamics near a manifold in nonlinear systems.
method Constrained autoencoder neural networks with invertible activation functions and biorthogonal weight matrices.
result Demonstrated effectiveness on a vortex shedding model, learning oblique fibers for fast dynamics.
Enhanced GARCH model uses autoencoder for volatility forecasting.
problem Selecting optimal realised volatility estimator for forecasting.
method Proposes an autoencoder-enhanced Realised GARCH model combining multiple realised measures.
result The model outperforms traditional linear methods in one-step-ahead rolling volatility forecasting.
The paper explores how the depth of neural networks affects their ability to represent data accurately.
problem Understanding the implicit bias and rank of neural networks with large depth.
method Analyzing the convergence of representation cost to a notion of rank as network depth increases, and investigating conditions for recovering the true rank of data.
result There is a range of network depths where the true rank of data is recovered, and this affects the topology of class boundaries.
The ability of deep neural networks to generalize well in the overparameterized regime has become a subject of significant research interest. We show that overparameterized autoencoders exhibit memorization, a form of inductive bias that constrains the functions learned through the optimization process to concentrate a…
Proposes a neural network autoencoder for smoothing and representation learning of functional data.
problem Lack of sufficient nonlinear representations in existing methods for functional data analysis.
method Develops a neural network autoencoder architecture to process functional data directly, learning both smoothing and representation.
result Outperforms traditional methods in prediction, classification, and computational efficiency.
MMbeddings reduces categorical embeddings by treating them as latent effects, significantly decreasing parameters and mitigating overfitting.
problem Large cardinalities in categorical embeddings lead to high parameter counts and overfitting.
method MMbeddings treats embeddings as latent random effects in a variational autoencoder framework, reducing parameter count and mitigating overfitting.
result MMbeddings consistently outperforms traditional embeddings across various tasks, demonstrating its potential in machine learning applications.
Proposes a new method to learn operators for stochastic problems using DeepONet with autoencoder.
problem Efficiently solve forward and inverse stochastic problems with limited data.
method MultiAuto-DeepONet, a multi-resolution autoencoder DeepONet model.
result The model effectively handles high-dimensional stochastic inputs and reduces the number of trainable parameters.
Proposes a variational autoencoder for long-term customer revenue forecasting.
problem Predicting long-term customer revenue from sparse and irregular transaction data.
method Variational Autoencoder (VAE) with flexible latent representation.
result Improves upon latest benchmarks in multiple real-world datasets.
VAELLS learns latent manifold structure to improve VAE model accuracy.
problem VAEs struggle with mismatched latent structure and global structure.
method Integrates learnable manifold model into latent space of VAE.
result Improves model accuracy by matching prior to data manifold structure.
WeldNet reduces complex dynamics to simpler, manageable segments.
problem Complex, high-dimensional time-dependent datasets from physical processes are costly to simulate.
method Windowed Encoders for Learning Dynamics, splitting time domain into windows for nonlinear dimension reduction and propagator training.
result WeldNet captures nonlinear latent structures and dynamics, outperforming existing methods.
Novel KAN-based autoencoder improves asset pricing models' accuracy and interpretability.
problem Improving asset pricing models' accuracy and interpretability.
method Kolmogorov-Arnold Networks (KANs) inspired autoencoder for latent factor exposures.
result Outperforms Multilayer Perceptrons in both accuracy and interpretability.
New method reconstructs missing variables in time series using autoencoders and automatic differentiation.
problem Reconstruct missing variables in time series with flexible input and output combinations.
method Train an autoencoder with all features, optimize missing variables as inputs, and use automatic differentiation.
result Flexible input and output combinations can be achieved without retraining the autoencoder.
Improved SINDy autoencoder for identifying noisy dynamical systems.
problem Robust identification of noisy dynamical systems from data.
method Incorporates noise-separating neural network structures into SINDy autoencoder architecture.
result Accurately recovers latent dynamics and estimates measurement noise from noisy observations.
Graph regularized autoencoder improves anomaly detection performance.
problem Unsupervised anomaly detection in high-dimensional data.
method Developed a graph regularized autoencoder using MST-based distances.
result Outperforms alternative methods on 20 benchmark anomaly detection datasets.
Paired autoencoders solve inverse problems using latent space projections.
problem Solving inverse problems in scientific computing.
method Paired autoencoder framework that projects data and quantity of interest into a latent space.
result Paired autoencoders generate multiple reconstruction metrics and enable latent-space refinement for accurate data fitting.
Conformal Autoencoders infer intrinsic dimensionality and impose invariance.
problem Detecting intrinsic dimensionality and imposing invariance in nonlinear manifold data.
method Imposing orthogonality conditions on latent variables to infer intrinsic dimensionality and build coordinate invariance.
result The method can infer intrinsic dimensionality and build coordinate invariance on submanifolds.
InVA models image outcomes from multiple modalities, outperforming standard VAEs.
problem Understanding relationships across multiple imaging modalities in neuroimaging.
method Integrative Variational Autoencoder (InVA) framework for image-on-image regression.
result InVA accurately predicts PET scans from structural MRI, outperforming conventional models.
High-dimensional data in many areas such as computer vision and machine learning tasks brings in computational and analytical difficulty. Feature selection which selects a subset from observed features is a widely used approach for improving performance and effectiveness of machine learning models with high-dimensional…
Bayesian autoencoders discover physics from noisy data.
problem Challenges in identifying governing equations and coordinates from noisy, low-data real-world data.
method Bayesian SINDy autoencoders with hierarchical Bayesian sparsifying prior and adaptive empirical Bayesian method.
result Better physics discovery with lower data and fewer training epochs, along with valid uncertainty quantification.
Principal component analysis (PCA) is largely adopted for chemical process monitoring and numerous PCA-based systems have been developed to solve various fault detection and diagnosis problems. Since PCA-based methods assume that the monitored process is linear, nonlinear PCA models, such as autoencoder models and kern…
RAEUFS selects features from data without labels, improving robustness to outliers.
problem Feature selection in high-dimensional data, especially in the presence of outliers.
method RAEUFS uses a deep autoencoder to learn nonlinear feature representations, improving robustness to outliers.
result RAEUFS outperforms state-of-the-art UFS methods in both clean and outlier-contaminated data settings.
Time-lagged autoencoders (TAEs) have been proposed as a deep learning regression-based approach to the discovery of slow modes in dynamical systems. However, a rigorous analysis of nonlinear TAEs remains lacking. In this work, we discuss the capabilities and limitations of TAEs through both theoretical and numerical an…
Machine learning is used to compute achievable information rates (AIRs) for a simplified fiber channel. The approach jointly optimizes the input distribution (constellation shaping) and the auxiliary channel distribution to compute AIRs without explicit channel knowledge in an end-to-end fashion.
RVRAE combines deep learning and dynamic factor models for better stock returns prediction.
problem Improving stock returns prediction in volatile markets.
method Combines dynamic factor modeling with variational recurrent autoencoder (VRAE). Uses prior-posterior learning for optimal factor model.
result RVRAE outperforms traditional methods in predicting stock returns and estimating variances.
AEALT uses autoencoders to reduce text embedding dimensions for improved efficiency.
problem High dimensionality of text embeddings hinders downstream tasks.
method Factor-augmented supervised learning with autoencoders.
result AEALT outperforms conventional deep-learning approaches.
A new approach for blind channel equalization and decoding, variational inference, and variational autoencoders (VAEs) in particular, is introduced. We first consider the reconstruction of uncoded data symbols transmitted over a noisy linear intersymbol interference (ISI) channel, with an unknown impulse response, with…
Bayesian model updating uses VAEs to approximate likelihood with small data.
problem Approximating likelihood for small data sets in structural analysis.
method Uses multimodal VAEs to approximate likelihood, suitable for high-dimensional correlated observations.
result Demonstrates computational efficiency and accuracy compared to original VAE approach.
Probabilistic Autoencoder learns latent space weights' distribution.
problem Nonlinear model reconstruction error and sample quality.
method Normalizing flow for latent space weights' probability distribution.
result PAE achieves small reconstruction errors, high sample quality, and good performance.
Macromolecular and biomolecular folding landscapes typically contain high free energy barriers that impede efficient sampling of configurational space by standard molecular dynamics simulation. Biased sampling can artificially drive the simulation along pre-specified collective variables (CVs), but success depends crit…
Denoising autoencoders (DAEs) have proven useful for unsupervised representation learning, but a thorough theoretical understanding is still lacking of how the input noise influences learning. Here we develop theory for how noise influences learning in DAEs. By focusing on linear DAEs, we are able to derive analytic ex…
A new method learns complex dynamical systems from data efficiently.
problem Learning complex dynamical systems from large-scale data efficiently.
method Low-rank structured variational autoencoding framework for nonlinear Gaussian state-space models.
result Consistently demonstrates better predictive capabilities compared to other models.
Study shows double and triple descent in unsupervised autoencoders, improving performance in various tasks.
problem Exploring the phenomenon of double descent in unsupervised learning.
method Analytical demonstration and extensive experiments on synthetic and real datasets.
result Over-parameterized unsupervised autoencoders exhibit double and triple descent, enhancing performance in downstream tasks.
The paper discovers a hidden component in data using an autoencoder with a discriminator.
problem Discovering a single independent latent variable in data.
method An autoencoder with a discriminator is used to recover the hidden component.
result The approach can recover the hidden component up to entropy-preserving transformations.