Framework learns encodings to protect private attributes from inference.
problem Protect private attributes from inference in image encodings.
method Adversarial training of deep neural networks to inhibit classifier learning.
result Stable optimization approach yields encoders resistant to privacy inference.
A new approach to generative modeling using optimal transport.
problem Improving unsupervised generative modeling techniques.
method Relaxing constrained optimization of probabilistic encoders to a penalized optimal transport objective.
result The POT objective for 2-Wasserstein distance matches AAEs, providing theoretical justification.
Principal components analysis (PCA) is the optimal linear auto-encoder of data, and it is often used to construct features. Enforcing sparsity on the principal components can promote better generalization, while improving the interpretability of the features. We study the problem of constructing optimal sparse linear a…
SELO model predicts link signs better than SDGNN using subgraph encoding and linear optimization.
problem Inferring the sign of links in signed networks with limited sign data.
method Subgraph Encoding via Linear Optimization (SELO) approach to learn edge embeddings.
result SELO model outperforms state-of-the-art methods on multiple real-world signed networks.
End-to-end auto-encoder for neural image compression with variable bit rates.
problem Efficient image compression with variable bit rates.
method Block-based auto-encoder system with novel contributions.
result Incremental performance improvement of each contribution.
Direct optimization of discrete variational auto-encoders using arg max.
problem Optimizing discrete latent variables in variational auto-encoders.
method Direct optimization through arg max without softmax relaxations.
result Empirical effectiveness of direct loss minimization in discrete latent variables.
MUTE improves neural network performance with efficient target encoding.
problem Improving neural network performance with limited resources.
method MUTE optimizes Hamming distances among target encoding by understanding class confusion.
result MUTE offers better generalization and robustness with minimal overhead.
Develops an analytical method for filtering point process observations.
problem Intractability of dynamic state estimation based on point process observations.
method Bayesian approximation to optimal filtering, introducing distributional assumptions.
result Analytic framework provides insights into optimal encoding strategies.
New data encoding method speeds up distributed optimization.
problem Slow tasks (stragglers) slow down distributed optimization.
method Embed redundancy in data, allowing computation without waiting for stragglers.
result Deterministic linear convergence to approximate solution.
WWAE improves deep generative models by minimizing penalized optimal transport.
problem Challenges in learning deep generative models, such as blurriness in variational auto-encoders and instability in generative adversarial networks.
method WWAE formulates as minimization of penalized optimal transport between target and generated distributions. It uses closed-form squared Wasserstein-2 distance for Gaussians.
result WWAE learns better latent structures and generates higher quality samples than VAEs and GANs.
Iterative models improve inference efficiency in deep latent variable models.
problem Inference models in deep latent variable models are computationally inefficient and have an amortization gap.
method Proposes iterative models that learn to perform inference optimization through repeated encoding of gradients.
result Iterative models outperform standard inference models on benchmark data sets of images and text.
Improved VAE models avoid posterior collapse in text modeling.
problem Posterior collapse in VAEs leads to poor data manifold parameterization.
method Coupled-VAE couples a VAE with a deterministic autoencoder to improve encoder and decoder parameterizations.
result Coupled-VAE consistently improves results in probability estimation and latent space richness.
Unified framework explains geometric properties of CNNs.
problem Understanding why encoder-decoder CNNs perform well.
method Unified mathematical framework based on recent neural network theories.
result Encoder-decoder CNNs are related to nonlinear basis representation using combinatorial convolution frames.
Paper develops a decoder for sparse codes without encoder matrix, achieving optimal recovery.
problem Designing a decoder for sparse codes from linear measurements alone.
method Matrix factorization to recover encoder and sparse coding matrices from measurements.
result Decoder-Expander Based Factorisation recovers encoder and sparse coding matrix at optimal measurement rate with high probability.
Redundancy helps speed up slow nodes in distributed learning.
problem Slow nodes (stragglers) bottleneck distributed optimization and learning performance.
method Encode data with redundancy, dynamically exclude stragglers, and compensate losses.
result Optimization algorithms converge to solutions even with straggling nodes.
Novel framework reduces slow nodes' impact on distributed computing.
problem Mitigating slow nodes (stragglers) in distributed computing.
method Developed mathematical understanding and analyzed convergence behavior of encoded optimization algorithms.
result Characterized trade-offs between various parameters in encoded optimization.
New auto-encoder handles varying noise levels without retraining.
problem Auto-encoders degrade in noisy conditions.
method Formalized auto-encoders as transform learning, derived new architecture.
result Models generalize well to different noise levels.
The paper identifies saddlepoints in unsupervised auto-encoding neural nets.
problem The risk landscape of unsupervised least squares in auto-encoding neural nets.
method Established an equivalence between unsupervised least squares and principal manifolds, discussed regularization strategies for auto-encoders.
result All non-trivial critical points in auto-encoding are saddlepoints, which are degenerate in overcomplete auto-encoding.
DDPM encoder matches optimal transport for natural images.
problem Understanding theoretical properties of DDPM latent space.
method Showed DDPM encoder matches optimal transport for common distributions.
result DDPM encoder map coincides with optimal transport map for natural images.
New bounds for quantum circuits depend on how data is encoded.
problem Lack of explicit dependence on data encoding in generalization bounds for PQCs.
method Derived generalization bounds that depend on data encoding strategies using Rademacher complexity and metric entropy.
result Optimal data-encoding strategies can be selected via structural risk minimization.
Proposes SLCVAE to improve text diversity by self-labeling.
problem KL-Vanishing problem in CVAE for diverse text generation.
method Explicit optimizing objective to guide encoder towards best encoder, using a labeling network.
result Improves text diversity while maintaining comparable accuracy.
We propose computationally efficient encoders and decoders for lossy compression using a Sparse Regression Code. The codebook is defined by a design matrix and codewords are structured linear combinations of columns of this matrix. The proposed encoding algorithm sequentially chooses columns of the design matrix to suc…
A new method for Gaussian process regression with categorical inputs.
problem Challenges in building a predictive and computationally efficient Gaussian process with categorical inputs.
method Distributional encoding based on maximum mean discrepancy and Wasserstein distance.
result State-of-the-art predictive performance on various datasets.
Gradient flow autoencoder improves data efficiency over traditional autoencoders.
problem Sub-optimal latent space representations in autoencoders.
method Gradient flow through ODE with adaptive step size for optimization.
result Gradient flow autoencoder achieves higher data efficiency.
ZegOT uses optimal transport to zero-shot segment images with text prompts.
problem Zero-shot semantic segmentation with limited image-text alignment knowledge.
method ZegOT uses optimal transport to match multiple text prompts with frozen image embeddings.
result ZegOT achieves state-of-the-art performance in zero-shot semantic segmentation.
A novel generative encoder model for imaging and image processing.
problem Efficiently processing and recovering images with noise.
method A pre-training phase with a GAN and an AE, followed by an optimization phase.
result The GE model outperforms state-of-the-art algorithms in image recovery.
New research shows encoder-decoder GANs can still fail even on real data.
problem Theoretical limitations of Encoder-Decoder GAN architectures.
method Rigorous analysis of Encoder-Decoder GAN training objectives.
result The training objectives cannot prevent mode collapse or learning meaningless codes.
BANANAS uses Bayesian optimization with neural predictors to improve neural architecture search.
problem Improving neural architecture search (NAS) performance.
method BANANAS combines Bayesian optimization with neural predictors, focusing on architecture encoding, predictor, uncertainty calibration, acquisition function, and optimization strategy.
result BANANAS achieves state-of-the-art performance on NAS search spaces.
CRsAE auto-encoder recovers convolutional dictionary from noisy signals.
problem Recovering a convolutional dictionary from noisy signals.
method Constrained recurrent sparse auto-encoder (CRsAE) architecture.
result CRsAE successfully recovers the underlying dictionary in the presence of noise.
Paper proposes continual learning for sentence encoders.
problem Optimize sentence encoders for new corpora while maintaining old corpus accuracy.
method Initialize encoders with corpus-independent features, update using Boolean operations of conceptor matrices.
result Proposed sentence encoder can continually learn features from new corpora.
Proposes DC3-GAN for diverse unsupervised conditional generation.
problem Low diversity in unsupervised conditional generation.
method Integrates encoder-generator pair with generator-encoder pair to enhance diversity.
result Improves clustering performance and disentanglement of latent variables.
Novel PCA method for high-dimensional inverse problems.
problem Optimizing large-scale random fields with gradient information.
method Gradient-Sensitive Principal Component Analysis (Gradient-SPCA) that modifies PCA using objective function gradients.
result Improvements in encoding quality for objective function minimization and field distribution.
Extends graph encoder embedding to weighted graphs and matrices.
problem Classifying vertices in various graph types efficiently.
method Graph encoder embedding applied to weighted graphs, distance matrices, and kernel matrices.
result The method achieves asymptotic normality, enabling optimal classification.
New method extracts relevant information from X for predicting Y.
problem Finding optimal bottleneck variable in IB optimization problem.
method Non-parametric upper bound for mutual information, neural network implementation.
result Method achieves better performance than variational IB on real-world datasets.
Despite its nonconvex nature, ℓ0 sparse approximation is desirable in many theoretical and application cases. We study the ℓ0 sparse approximation problem with the tool of deep learning, by proposing Deep ℓ0 Encoders. Two typical forms, the ℓ0 regularized problem and the M-sparse problem, are …
The study compares ASP encodings for sequential pattern mining tasks.
problem Efficiency of Answer Set Programming (ASP) encodings for sequential pattern mining.
method Two representations of embeddings (fill-gaps vs skip-gaps) and various types of patterns were tested.
result Fill-gaps strategy is more efficient on real problems due to lower memory consumption.
A scalable deep generative model using variational auto-encoding and deep Gaussian processes.
problem Deep unsupervised learning and Bayesian optimization challenges.
method Augmenting deep Gaussian processes with a recognition model and variational framework to prevent parameter proliferation.
result Efficacy demonstrated on deep unsupervised learning and Bayesian optimization challenges.
Boosting trees predict Twitch subscriptions from user activity.
problem Predicting Twitch user subscriptions from activity data.
method Used boosting trees and target-encodings for high cardinality categoricals.
result User activity can be better predicted than content alone.
Improves energy efficiency of neuromorphic hardware by optimizing memory organization and encoding schemes.
problem Energy inefficiency in neuromorphic hardware, especially in digital accelerators.
method Synthesized controller and memory for different encoding schemes, introduced functional encoding for structured connectivity.
result Functional encoding offers a 58% reduction in energy for weight updates in convolutional layers.
This paper integrates auto-encoders and GANs using variational inference.
problem Preventing mode collapse in generative models.
method Develops a principle to combine variational auto-encoders and GANs, using synthetic likelihoods and implicit posterior distributions.
result Unified objective for optimizing the fusion of variational auto-encoders and GANs.
This work addresses encoding biases in neural networks by tailoring models with unsupervised losses.
problem Improving neural network representations and reducing the generalization gap.
method Inspired by transductive learning, the authors propose tailoring and meta-tailoring to optimize unsupervised losses during prediction time.
result Models trained with tailoring and meta-tailoring perform better on the task objective after adapting to unsupervised losses.
Auto-encoders can recover true signals under certain conditions.
problem Signal recovery from auto-encoders.
method Highly incoherent weight matrices and unit ℓ2 row length, negative bias vectors equal to data mean. result True hidden representation can be approximately recovered with increasing sparsity.
Enhances drug discovery by optimizing molecular structures.
problem Accelerate drug discovery through better optimization of precursor molecules.
method Integrates substructure components with atom-level encoding in a fully autoregressive graph decoder.
result Significantly outperforms previous state-of-the-art baselines on molecular optimization tasks.
This paper analyzes VAE approximation errors in conditional exponential families.
problem Posterior collapse and approximation errors in VAEs.
method Analysis of ELBO objective and conditional exponential families.
result The ELBO optimizer pulls away from the likelihood optimizer towards a consistent subset of models.
CVAEs improve VAEs by accounting for correlations in latent representations.
problem VAEs fail to account for correlations between data points, limiting their effectiveness.
method CVAEs incorporate correlation structure into VAEs using a prior and tractable approximations.
result CVAEs outperform baseline algorithms in matching and link prediction tasks.
The process of dynamic state estimation (filtering) based on point process observations is in general intractable. Numerical sampling techniques are often practically useful, but lead to limited conceptual insight about optimal encoding/decoding strategies, which are of significant relevance to Computational Neuroscien…
MIM learns joint distributions with mutual information and low divergence.
problem Learning joint distributions over observations and latent variables.
method Probabilistic auto-encoder with three design principles: low divergence, high mutual information, and low marginal entropy.
result MIM learns representations with high mutual information, consistent encoding and decoding distributions, effective latent clustering, and comparable data log likelihood to VAE.
Two-step process generates molecules from latent vectors.
problem Generating valid molecules from latent representations.
method Two-step decoding: first formula, then bonds.
result Highest reconstruction rate of 90.5%.