Adds layers to NNs to protect them from reverse engineering.
problem Extracting the underlying model of a Neural Network.
method Introducing parasitic layers that approximate a noisy identity mapping with a Convolutional NN.
result The protected NN's predictions remain mostly unchanged while making reverse-engineering more complex.
Bayesian Layers adds uncertainty to neural networks, enabling faster experimentation and scalability.
problem Enabling neural networks to quantify uncertainty in predictions.
method Drop-in replacements for common layers, capturing uncertainty over weights, activations, etc.
result Bayesian Layers can fit large models like 5-billion parameter Bayesian Transformers.
New invertible transformations improve flow-based generative models.
problem Improving flow-based generative models for better performance.
method Proposed new invertible transformations and coupling layers.
result New coupling layers achieve better results in IDF.
RevDEQs improve performance on tasks with exact gradients and fewer function evaluations.
problem Inexact gradient calculation in DEQs leads to unstable training and requires regularisation or many function evaluations.
method Introduce Reversible Deep Equilibrium Models (RevDEQs) that allow for exact gradient calculation, no regularisation, and far fewer function evaluations.
result RevDEQs significantly improve performance on language modelling and image classification tasks.
Reformer improves Transformer efficiency for long sequences.
problem Prohibitively costly training of large Transformer models.
method Locality-sensitive hashing for attention and reversible residual layers.
result Memory-efficient and faster performance on long sequences.
PETRA enables parallel training of deep models with reversible architectures.
problem Challenges in parallelizing deep model training.
method Introduces PETRA, a novel approach for parallelizing gradient computations in reversible architectures.
result Achieves competitive accuracies on CIFAR-10, ImageNet32, and ImageNet using ResNet models.
Recently, deep residual networks have been successfully applied in many computer vision and natural language processing tasks, pushing the state-of-the-art performance with deeper and wider architectures. In this work, we interpret deep residual networks as ordinary differential equations (ODEs), which have long been s…
A new perspective on self-attention models using MLPs.
problem Improving sequence modeling with self-attention mechanisms.
method Introducing HyperMLP and HyperGLU, which use dynamic two-layer MLPs with reverse-offset layout.
result HyperMLP/HyperGLU consistently outperform softmax-attention baselines.
Flood extent mapping plays a crucial role in disaster management and national water forecasting. Unfortunately, traditional classification methods are often hampered by the existence of noise, obstacles and heterogeneity in spectral features as well as implicit anisotropic spatial dependency across class labels. In thi…
Memory-efficient learning for large-scale imaging systems.
problem Memory limitations in GPUs for real-world large-scale inverse problems.
method Exploits reversibility of network layers to enable data-driven design.
result Demonstrated on small-scale and large-scale real-world systems.
Self-supervised and supervised methods learn similar intermediate visual representations but diverge in final layers.
problem Comparing self-supervised and supervised methods for visual learning.
method Comparison of contrastive self-supervised and supervised methods on simple image data.
result Contrastive and supervised methods learn similar intermediate representations but diverge in final layers.
We analyse a multiplex of networks between OECD countries during the decade 2002-2010, which consists of five financial layers, given by foreign direct investment, equity securities, short-term, long-term and total debt securities, and five environmental layers, given by emissions of N O x, P M 10 SO 2, CO 2 equivalent…
Adversarial domain adaptation reduces sample bias in high energy physics classifier.
problem Sample bias in high energy physics classifier training.
method Adversarial domain adaptation using neural networks with gradient reversal layer.
result Successful bias removal on simulated events at the LHC.
A new method for image translation using disentangled style and content preservation.
problem Difficulty in maintaining original content during reverse diffusion in diffusion-based image translation.
method Disentangled style and content representation using intermediate keys from ViT model, CLIP loss, semantic divergence loss, and resampling strategy.
result Outperforms state-of-the-art models in text-guided and image-guided translation tasks.
Classifies reversible and strongly reversible elements in quaternionic groups.
problem Classifying reversible and strongly reversible elements in quaternionic groups.
method Proves elements are reversible if and only if they are products of skew-involutions (resp. involutions).
result Proves elements are reversible if and only if they are products of skew-involutions (resp. involutions).
The paper classifies reversible and strongly reversible elements in Hermitian isometry groups.
problem Classifying reversible and strongly reversible elements in Hermitian isometry groups.
method Classification through group theory and algebraic manipulation.
result New classification of strongly reversible elements in Sp(n).
The paper classifies reversible and strongly reversible elements in quaternionic hyperbolic spaces.
problem Classifying reversible and strongly reversible elements in quaternionic hyperbolic spaces.
method Analyzing conjugacy classes and using properties of quaternionic hyperbolic spaces and their isometry groups.
result All elements of the isometry group of quaternionic hyperbolic spaces are strongly reversible.
This paper classifies reversible and strongly reversible elements in affine groups.
problem Classifying reversible and strongly reversible elements in affine groups.
method Identifying affine transformations and using conjugacy by involutions.
result Classification of reversible and strongly reversible elements in affine groups.
Study finds mean reversion strategies perform well on historical data but fail in recent market conditions.
problem Performance of mean reversion strategies in recent market data.
method Empirical investigation of three mean reversion strategies (PAMR, OLMAR, TCO) on historical S&P 500 data and benchmark datasets.
result Mean reversion strategies may fail in recent market conditions, especially with transaction costs.
New neural net learns time-reversible symplectic dynamics.
problem Lack of time-reversibility in neural networks for symplectic systems.
method Proposes a new neural network architecture for time-reversible symplectic systems.
result Demonstrates learning of time-reversible symplectic dynamics from data.
A new trading strategy using reinforcement learning for statistical arbitrage.
problem Traditional statistical arbitrage models rely on model assumptions and price deviations from a long-term mean.
method Empirical reversion time metric, reinforcement learning framework, and state space optimization.
result Optimal mean reversion strategy identified through reinforcement learning.
Algebraic method reveals criterion for quaternionic Möbius group reversibility.
problem Characterizing reversibility in quaternionic Möbius group elements.
method Purely algebraic approach using matrix entries and conjugacy invariants.
result Explicit criterion for reversibility in terms of matrix entries.
A Finsler space is said to be geodesically reversible if each oriented geodesic can be reparametrized as a geodesic with the reverse orientation. A reversible Finsler space is geodesically reversible, but the converse need not be true. In this note, building on recent work of LeBrun and Mason, it is shown that a geodes…
Sharp stability results for reverse isoperimetric inequalities in 2D.
problem Reverse isoperimetric inequalities in the plane.
method Stability analysis of λ-convex bodies and convex bodies with smooth boundaries. result Sharp stability results for reverse isoperimetric inequalities, including inradius and Cheeger inequalities.
On-line portfolio selection has attracted increasing interests in machine learning and AI communities recently. Empirical evidences show that stock's high and low prices are temporary and stock price relatives are likely to follow the mean reversion phenomenon. While the existing mean reversion strategies are shown to …
Boundary rigidity proven for non-reversible Finsler metrics.
problem Recovering non-reversible Finsler metrics from boundary distance data.
method Sum of reversible Finsler norm and closed 1-form, boundary rigidity results.
result 1-form can be uniquely recovered from boundary distance data.
Develops LSTM for predicting neuronal dynamics over long time-horizons.
problem Understanding and controlling complex brain behaviors.
method Long Short-Term Memory (LSTM) neural network architecture for multi-time step predictions.
result LSTM improves short time-horizon prediction accuracy and multi-time step predictions of neuronal dynamics.
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
Reverse Experience Replay improves Deep Q-learning for sparse rewards.
problem Sparse rewards and reward-maximizing tasks in Deep Q-learning.
method Sampling transitions in reverse order for training.
result Significantly increased performance in tasks with limited experience and memory capacity.
New knots not rationally concordant to their reverses found.
problem Identifying knots not rationally concordant to their reverses.
method Infinite family of knots constructed, rational knot concordance group analyzed.
result Infinite rank subgroup in rational knot concordance group.
The paper classifies reversible elements in Seifert-fibered spaces and braid groups.
problem Classifying reversible elements in Seifert-fibered spaces and braid groups.
method Classification of reversible elements in Fuchsian groups, application to Seifert-fibered groups, and analysis of 3-torsion elements.
result Classification and analysis of reversible and 3-torsion elements in Seifert-fibered spaces and braid groups.
Reverse annealing boosts quantum matrix factorization performance.
problem Improving quantum matrix factorization performance.
method Combining forward and reverse annealing for nonnegative/binary matrix factorization.
result Combination of forward and reverse annealing significantly improves performance.
Unified understanding of neural networks on group operations verified.
problem Understanding and verifying neural networks trained on group operations.
method Investigated one-hidden-layer neural networks trained on binary operation of finite groups, revealing structure and providing a compact proof of model performance.
result Verified explanation applies to a large fraction of networks trained on the symmetric group S5, providing a >=95% accuracy bound for 45% of models.
TRS-ODENs learn dynamics with time-reversal symmetry for more efficient learning.
problem Learning dynamics with time-reversal symmetry for more efficient learning.
method Proposed a loss function and a new framework (TRS-ODENs) to learn dynamics efficiently.
result TRS-ODENs can learn dynamics from noisy and complex trajectories efficiently.
Study on Finsler spaces with specific metric changes and reversible geodesics.
problem Characterizing Finsler spaces with reversible geodesics.
method Analyzing a Finsler space with a Randers change of Quartic metric and deriving conditions for reversible geodesics.
result The Finsler metric F induces a generalized weighted quasi-distance on the space.
Paper proves rigidity theorems for geodesically reversible Finsler metrics.
problem Understanding geodesically reversible Finsler metrics in closed manifolds.
method Applied theory of volumes and areas on Finsler spaces to establish rigidity theorems.
result Partial explanation of the scarcity of geodesically reversible Finsler metrics in closed manifolds.
Reduces identity testing of reversible Markov chains to simpler symmetric chain tests.
problem Testing identity of reversible Markov chains from a single trajectory.
method Using lumping-congruent Markov embeddings, the problem is simplified to testing symmetric chains over a larger state space.
result Achieves state-of-the-art sample complexity for identity testing.
Proof of reverse isoperimetric inequality for black holes.
problem Reverse isoperimetric inequality for black holes in Einstein gravity.
method Geometric-analytic approach.
result Reversal of the usual isoperimetric inequality is explained by curved backgrounds governed by Einstein's equations.
A novel feedbackward approach for semantic segmentation reduces parameter count.
problem Semantic segmentation challenges in terms of model complexity and performance.
method Reverse-direction encoder-decoder architecture using the same network for both encoding and decoding.
result Significantly outperforms other models using only 13 VGG-16 layers, achieving higher IoU scores.
New algorithm learns bridged diffusion processes without time-reversals.
problem Learning bridged diffusion processes efficiently and accurately.
method Score matching with Doob's h-transform, avoiding time-reversals.
result Outperforms existing methods in learning bridged diffusion processes.
Consider the problem of pricing options on forwards in energy markets, when spot prices follow a geometric multi-factor model in which several rates of mean reversion appear. In this paper we investigate the role played by slow mean reversion when pricing and hedging options. In particular, we determine both upper and …
Characterizes isometries between non-reversible Finsler manifolds.
problem Understanding isometries in non-reversible Finsler manifolds.
method Generalization of Myers-Nakai Theorem for Riemannian manifolds, modification of function spaces to accommodate asymmetric structure.
result Functional characterization of isometries between non-reversible Finsler manifolds.
State-of-the-art machine learning models frequently misclassify inputs that have been perturbed in an adversarial manner. Adversarial perturbations generated for a given input and a specific classifier often seem to be effective on other inputs and even different classifiers. In other words, adversarial perturbations s…
New method trains neural samplers to sample from multi-modal distributions efficiently.
problem Mode-seeking behavior of reverse KL divergence hinders effective sampling from multi-modal target distributions.
method Minimizing reverse diffusive KL divergence along diffusion trajectories of model and target densities.
result Demonstrated enhanced sampling performance across various multi-modal distributions.
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
Classifies orientation-reversing homeomorphisms of even periods on surfaces.
problem Classifying orientation-reversing homeomorphisms of even periods on surfaces.
method Following the approach of [1] and correcting errors in [1] for the case of periods multiple of 4.
result Classification for orientation-reversing homeomorphisms of periods multiple of 4.
A new sampler speeds up Bayesian mixture models.
problem Sampling from Bayesian finite mixture models is slow and hard.
method Introduces a non-reversible sampling scheme for Bayesian finite mixture models.
result The new sampler outperforms classical samplers in many scenarios, especially during convergence.
Study shows knots can be reversed without becoming trivially concordant.
problem Understanding when knots can be reversed without changing their concordance class.
method Constructed an infinite family of knots, analyzed their concordance properties.
result Found an infinitely generated free subgroup in the concordance group of topologically slice knots.