New neural net learns time-reversible symplectic dynamics.
problem Lack of time-reversibility in neural networks for symplectic systems.
method Proposes a new neural network architecture for time-reversible symplectic systems.
result Demonstrates learning of time-reversible symplectic dynamics from data.
Develops reversible neural networks for deep learning, improving efficiency and performance.
problem Training and deploying deep neural networks with limited resources and memory.
method Interprets deep residual networks as ODEs, derives reversible architectures, and analyzes stability.
result Reversible neural networks can be trained deeper with less memory and computational resources.
Improves neural network mapping functionality using latent feature generation.
problem Improving neural network performance in visual recognition tasks.
method Reversible learning for generating and learning latent features.
result The proposed method outperforms existing state-of-the-art methods in visual recognition.
Adds layers to NNs to protect them from reverse engineering.
problem Extracting the underlying model of a Neural Network.
method Introducing parasitic layers that approximate a noisy identity mapping with a Convolutional NN.
result The protected NN's predictions remain mostly unchanged while making reverse-engineering more complex.
New method trains generative models by reversing generator maps.
problem Training deep neural network generators.
method Non-parametrically estimate flexible code distributions by reversing generator maps.
result More powerful generative models, better latent structure modeling, explicit generalization control.
Neural networks combining multiple data sources can reverse preferences, affecting decision reliability.
problem Preference reversals in neural networks under pooled data.
method Formalized through Case-Based Decision Theory, analyzed Gram geometry, introduced regularization, and developed auditing methods.
result Pooled refitting can reverse shared preferences, and conditions for preserving preferences are derived.
Reversible RNNs reduce memory usage in training without sacrificing performance.
problem Memory-intensive training of RNNs limits model flexibility.
method Developed a scheme for perfect reversible RNNs with forgetting, reducing memory by 10-15x.
result Achieved comparable performance to traditional models with reduced activation memory cost.
Paper connects neural network score approximation to reverse diffusion model distribution approximation.
problem Quantifying the relationship between neural network score approximation and the distribution generated by reverse diffusion models.
method Combines Hornik's universal approximation theorem, Girsanov's theorem, and data processing inequality.
result Neural network score approximation guarantees distribution approximation in reverse diffusion models.
Paper extends method to detect reverse engineering attacks on DNN classifiers.
problem Detecting reverse engineering attacks on deep neural network image classifiers.
method Extend ADA approach to detect RE attacks (ADA-RE).
result Successfully detects 'stealthy' RE attacks before they launch test-time evasion attacks.
Researchers can reverse-engineer deep ReLU networks from outputs.
problem Recovering a deep ReLU network from its outputs.
method Dissecting region boundaries to identify neuron states and weights.
result Weights and neuron arrangement of a deep ReLU network can be recovered.
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
Graph normalizing flows use neural networks for graph prediction and generation.
problem Efficiently processing and generating graph data with reduced memory usage.
method Reversible graph neural network model combining auto-encoder and normalizing flows.
result Graph normalizing flows achieve competitive results in graph generation and prediction.
Paper combines QRM and CNN for better stock option price forecasting.
problem Forecasting stock option prices in a complex market.
method Solves Black-Scholes equation using QRM, trains CNN models on data.
result CNN models improve option price prediction accuracy.
NSGLD improves SGLD for non-convex optimization problems.
problem Optimizing non-convex objectives efficiently.
method Introducing non-reversible SGLD by adding an anti-symmetric matrix to the drift term of the Langevin diffusion.
result NSGLD converges faster to the same stationary distribution with non-asymptotic guarantees.
Paper demystifies AD techniques for efficient deep learning.
problem Generalizing backpropagation for complex neural networks.
method Uncovering reverse-mode AD and its connection to delimited continuations, implementing it via operator overloading.
result Efficient reverse-mode AD without auxiliary data structures, combining deep learning and pure library approaches.
New method trains Markov kernels for efficient sampling.
problem Efficient sampling from complex probability distributions.
method Adversarial learning of involutive Metropolis-Hastings kernels.
result Minimizes total variation distance to empirical data.
Two gradient-based methods for hyperparameter optimization are introduced, with applications in machine learning.
problem Optimizing hyperparameters for machine learning models.
method Forward and reverse-mode procedures for computing gradients of validation error with respect to hyperparameters.
result Forward-mode procedure suitable for real-time hyperparameter updates, potentially speeding up optimization on large datasets.
A new interpolation method speeds up neural ODE training.
problem Efficiently approximating gradients in neural ODEs.
method Interpolation-based technique to approximate gradients.
result Our method trains neural ODEs faster than the reverse dynamic method.
Paper optimizes stock option forecasting using ML models and improved trading strategies.
problem Improving accuracy of stock option predictions and trading decisions.
method Application of Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Quasi-Reversibility Method (QRM).
result Optimized stock option investment results through improved trading strategies and model combination.
Bayesian Layers adds uncertainty to neural networks, enabling faster experimentation and scalability.
problem Enabling neural networks to quantify uncertainty in predictions.
method Drop-in replacements for common layers, capturing uncertainty over weights, activations, etc.
result Bayesian Layers can fit large models like 5-billion parameter Bayesian Transformers.
Reverse engineered RNNs reveal line attractor dynamics for sentiment classification.
problem Understanding how recurrent neural networks solve sequential tasks like sentiment classification.
method Dynamical systems analysis to reverse engineer trained RNNs, identifying fixed points and linearized dynamics.
result Trained RNNs converge to low-dimensional line attractor dynamics, providing interpretable solutions.
GRAD simplifies making neural networks fair.
problem No methods for making neural networks fair.
method Gradient Reversal Against Discrimination (GRAD)
result Improved individual and group fairness.
RNNs learn device models from input/output data.
problem Learning complex device models from limited data.
method Empirical study using RNNs to model six different devices.
result RNNs can generate functional software-only models of hardware devices.
New method bypasses time-reversal for diffusion-based generative models.
problem Diffusion-based generative models require time-reversal, limiting flexibility.
method Constructs diffusion processes without time-reversal through mixtures of diffusion bridges.
result Exact transport without time-reversal, greater flexibility in dynamics.
Quantum machine learns faster by reverse annealing on AQCs.
problem Training RBMs on AQCs is hard due to low qubit connectivity.
method Embedding RBM nodes to virtual qubits, semantic quantum search, reverse annealing schedule.
result Reverse annealing accelerates RBM training and improves reconstruction scores.
Deep neural network improves ultrasound imaging quality.
problem Rapid performance degradation of traditional beamformers in poor data acquisition conditions.
method Developed a deep learning-based beamformer.
result Significant performance gain in contrast-to-noise ratio and structural similarity for focused and planar imaging.
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
Tuning hyperparameters of learning algorithms is hard because gradients are usually unavailable. We compute exact gradients of cross-validation performance with respect to all hyperparameters by chaining derivatives backwards through the entire training procedure. These gradients allow us to optimize thousands of hyper…
New method trains neural samplers to sample from multi-modal distributions efficiently.
problem Mode-seeking behavior of reverse KL divergence hinders effective sampling from multi-modal target distributions.
method Minimizing reverse diffusive KL divergence along diffusion trajectories of model and target densities.
result Demonstrated enhanced sampling performance across various multi-modal distributions.
Unified understanding of neural networks on group operations verified.
problem Understanding and verifying neural networks trained on group operations.
method Investigated one-hidden-layer neural networks trained on binary operation of finite groups, revealing structure and providing a compact proof of model performance.
result Verified explanation applies to a large fraction of networks trained on the symmetric group S5, providing a >=95% accuracy bound for 45% of models.
BPN defends against adversarial attacks by generating beneficial perturbations.
problem Adversarial attacks cause deep neural networks to misclassify clean inputs.
method BPN generates beneficial perturbations during training to neutralize future adversarial attacks.
result BPN is robust to adversarial examples and more efficient than classical adversarial training.
A new method relaxes molecules without needing non-equilibrium data.
problem Molecular relaxation requires understanding non-equilibrium structures.
method MoreRed: molecular relaxation by reverse diffusion with time step prediction.
result MoreRed learns a simpler pseudo potential energy surface.
Adversarial domain adaptation reduces sample bias in high energy physics classifier.
problem Sample bias in high energy physics classifier training.
method Adversarial domain adaptation using neural networks with gradient reversal layer.
result Successful bias removal on simulated events at the LHC.
We propose a novel reversible jump Markov chain Monte Carlo (MCMC) simulated annealing algorithm to optimize radial basis function (RBF) networks. This algorithm enables us to maximize the joint posterior distribution of the network parameters and the number of basis functions. It performs a global search in the joint …
RevNets improve generative performance on CelebA dataset.
problem Training autoencoders is complicated by needing separate encoder and decoder models.
method Used adversarial autoencoder framework with RevNets in latent space.
result RevNets generate coherent faces with similar quality to Variational Autoencoders.
Recurrent neural networks' hidden state can be reconstructed from its past, providing a theoretical framework for stability and tracking.
problem Hidden-state stability in RNNs
method Backward coherence analysis
result Almost-sure convergence, rates under mixing, interpretable limiting representation, finite pathwise stopping times, and theoretical framework for time-uniform confidence sequences.
TRS-ODENs learn dynamics with time-reversal symmetry for more efficient learning.
problem Learning dynamics with time-reversal symmetry for more efficient learning.
method Proposed a loss function and a new framework (TRS-ODENs) to learn dynamics efficiently.
result TRS-ODENs can learn dynamics from noisy and complex trajectories efficiently.
New method trains networks to avoid using protected concepts in decisions.
problem Avoiding use of protected concepts in neural network decisions.
method Domain-Adversarial Neural Network with gradient reversal layer.
result Trains networks to make decisions agnostic to protected concepts.
A new Monte Carlo sampling method derived from reverse diffusion.
problem Sampling from complex distributions, especially multi-modal ones.
method Transforming score matching into mean estimation; estimating means of regularized posterior distributions.
result rdMC can approximate sampling with any desired accuracy and is significantly faster than MCMC for complex distributions.
AEGR method improves anomaly detection in autoencoders without needing anomaly-free training data.
problem Challenges in anomaly detection, especially high dimensionality and noise in training sets.
method Gradient-reversal method for autoencoders, using reconstruction error and Local Outlier Factor.
result The proposed AEGR model outperforms other methods in detecting network anomalies.
A taxonomy classifies memory networks based on their memory organization.
problem Classifying and understanding the expressive power of different memory networks.
method Developed a taxonomy including RNN, LSTM, neural stack, and neural RAM, analyzing their differences and commonality.
result Showed the relative expressive power of memory networks and how they relate to specific tasks.
Reverse Experience Replay improves Deep Q-learning for sparse rewards.
problem Sparse rewards and reward-maximizing tasks in Deep Q-learning.
method Sampling transitions in reverse order for training.
result Significantly increased performance in tasks with limited experience and memory capacity.
DPS uses PINNs to estimate drift in diffusion models for sampling.
problem Accurately estimating drift term in reverse SDE from unnormalized density.
method Diffusion-PINN Sampler (DPS) solves PINN for log-density of SDE marginals.
result DPS achieves convergence guarantees and accurately samples complex distributions.
Enhances RJMCMC efficiency with non-linear transport-based proposals.
problem Designing efficient RJMCMC proposals for complex models.
method Applies non-linear transport-based approach to construct efficient transdimensional jumps.
result Acceptance probability depends only on model probabilities when exact transports are used.
New invertible transformations improve flow-based generative models.
problem Improving flow-based generative models for better performance.
method Proposed new invertible transformations and coupling layers.
result New coupling layers achieve better results in IDF.
A new neural network framework ADNN improves financial feature construction.
problem Constructing highly informative financial features.
method Neural network (ADNN) with domain knowledge, pre-training, and data augmentation.
result ADNN produces more diversified and informative features than genetic programming.
Bayesian symbolic regression uncovers missing physics from data with uncertainty quantification.
problem Incomplete knowledge of physical laws from experimental data.
method Bayesian symbolic regression using Reversible Jump Markov Chain Monte Carlo.
result Uncertainty quantification in recovered model structures.
ABIPNN improves neural network performance by processing vectors in each neuron.
problem Traditional neural networks fail to model associations among adjacent scalars.
method ABIPNN uses arbitrary bilinear products to process vector-valued neurons.
result ABIPNN outperforms conventional neural networks in multispectral image denoising and singing voice separation.